• 15 Posts
  • 45 Comments
Joined 1 year ago
cake
Cake day: June 4th, 2023

help-circle


  • While true for the component itself, there’s material difference for any caps surrounding it. Sure the chipset would work fine at 40, 50, 70°C. However electrolytic capacitors lifespan is halved with every 10°C temperature increase. From a brief search it seems solid caps also crap out much faster at higher temps but can outlast electrolytic at lower temps. This is a consideration for a long lifespan system. The one in my case is expected to operate till 2032 or beyond.

    I don’t think other components degrade in any significant fashion whether they run at 40 or 60°C.










  • Hard to say. She’s been in 24/7 service since 2017. Never had stability issues and I’ve tested it with Prime95 plenty of times upon upgrades. Last week I ran a Llama model and the computer froze hard. Even holding the power button wouldn’t turn it off. Did the PSU power flip, came back up. Prime95 stable. Llama -> rip. Perhaps it’s been cooked for a while and only trips by this workload. She’s an old board, a Gigabyte with B350 running a 5950X, so it’s not super surprising that the power section has been a bit overused. 😅 Replacing with an X570 as we speak.






  • B350 isn’t a very fast chipset to begin with

    For sure.

    I’m willing to bet the CPU in such a motherboard isn’t exactly current-gen either.

    Reasonable bet, but it’s a Ryzen 9 5950X with 64GB of RAM. I’m pretty proud of how far I’ve managed to stretch this board. 😆 At this point I’m waiting for blown caps, but the case temp is pretty low so it may end up trucking along for surprisingly long time.

    Are you sure you’re even running at PCIe 3.0 speeds too?

    So given the CPU, it should be PCIe 3.0, but that doesn’t remove any of the queues/scheduling suspicions for the chipset.

    I’m now replicating data out of this pool and the read load looks perfectly balanced. Bandwidth’s fine too. I think I have no choice but to benchmark the disks individually outside of ZFS once I’m done with this operation in order to figure out whether any show problems. If not, they’ll go in the spares bin.



  • I put the low IOPS disk in a good USB 3 enclosure, hooked to an on-CPU USB controller. Now things are flipped:

                                            capacity     operations     bandwidth 
    pool                                  alloc   free   read  write   read  write
    ------------------------------------  -----  -----  -----  -----  -----  -----
    storage-volume-backup                 12.6T  3.74T      0    563      0   293M
      mirror-0                            12.6T  3.74T      0    563      0   293M
        wwn-0x5000c500e8736faf                -      -      0    406      0   146M
        wwn-0x5000c500e8737337                -      -      0    156      0   146M
    

    You might be right about the link problem.

    Looking at the B350 diagram, the whole chipset is hooked via PCIe 3.0 x4 link to the CPU. The other pool (the source) is hooked via USB controller on the chipset. The SATA controller is also on the chipset so it also shares the chipset-CPU link. I’m pretty sure I’m also using all the PCIe links the chipset provides for SSDs. So that’s 4GB/s total for the whole chipset. Now I’m probably not saturating the whole link, in this particular workload, but perhaps there’s might be another related bottleneck.


  • Turns out the on-CPU SATA controller isn’t available when the NVMe slot is used. 🫢 Swapped SATA ports, no diff. Put the low IOPS disk in a good USB 3 enclosure, hooked to an on-CPU USB controller. Now things are flipped:

                                            capacity     operations     bandwidth 
    pool                                  alloc   free   read  write   read  write
    ------------------------------------  -----  -----  -----  -----  -----  -----
    storage-volume-backup                 12.6T  3.74T      0    563      0   293M
      mirror-0                            12.6T  3.74T      0    563      0   293M
        wwn-0x5000c500e8736faf                -      -      0    406      0   146M
        wwn-0x5000c500e8737337                -      -      0    156      0   146M