Re: Performance variation in traditional HDDs?

On 4/24/25 06:04, Nick Neumann wrote:
> Just a quick update on initial test results. We started with the two
> "identical" drives (https://pcpartpicker.com/product/BcQG3C) that
> showed the largest difference. Note these are SMR drives so we are
> careful to do an "ATA SECURITY ERASE" before running our benchmark.

So these are drive-managed SMR disks, right ?
All bets are off in terms on performance then. These drives behave like SSDs and
do a lot of internal "grabage collection" to handle the sequential write SMR
constraint. And even doing a purely sequential write workload to the disk does
not guarantee in-order deliveries of the writes to the disk, so that garbage
collection may be trigerring and changing the results. AHCI adapters are
notorious for *not* delivering commands in order...

> Previously we had these drives sitting vertically in 3d-printed trays
> during our benchmarking. We moved to benchmarking them mounted
> horizontally with screws and vibration dampening pads. They were
> mounted to a heavy metal "open air" style case. Drive 1 was
> benchmarked, then Drive 1 was removed and Drive 2 installed in the
> exact same location as Drive 1, and then Drive 2 was benchmarked.

Mounting drives vertically is fine. Plenty of enterprise high-density JBOD
enclosures use such mounting. Vibration dampening is still a requirement of course.

> We didn't see any real difference in results when changing to this
> traditional mounting. Drive 1 went from an average full-write
> throughput of 174.06MB/s to 174.08MB/s. Drive 2's average full write
> throughput stayed at exactly 157.88MB/s.
> 
> Looking at a graph of throughput over time for the
> differently-behaving but identical drives is really interesting:
> https://cdna.pcpartpicker.com/static/forever/images/content/13b2fa7e147ddd5359f5638b4efde7af.png

The general shape of the curve going down with higher LBAs is normal as an HDD
performance is higher at OD (outter diameter, more sectors per track) than ID
(inner diamters, shorter tracks so less sectors per revolution).
However, since these seems to be drive-managed SMR disks, there is no way to
tell how the HDD firmware is mapping LBAs to physical sectors. The noisy results
tend to indicate that there is a lot of HDD internal IOs going on.

> We're working on making sure our HDD mounting is adequate. Do you have
> any specific guidance/tips on what should be done? Is the goal of the
> mounting to prevent outside vibrations from affecting the drive,
> dissipating internal vibrations, or both? We use a number of identical
> 3U rackmount machines for our SSD benchmarking and would also like to
> leverage them for diving into this HDD issue in more depth, but they
> do not have traditional HDD mounting locations. So for now we are more
> slowly conducting experiments on one of our open-air dev benchmark
> machines, which has proper mounting locations.

I do not think this is a mounting issue. Drive-managed SMR and disk internal
state differences are more likely. I recommend you contact your drive vendor to
see if they can explain things. Generally, drive internal FW operation is not
public information though.

Another thing you may want to try is to do a full format of the drive before
benchmarking them. To make sure the LBA mapping is reset and consistant between
the drives.


-- 
Damien Le Moal
Western Digital Research