Re: Disk I/O slowness

> On Jul 29, 2026, at 5:28=E2=80=AFPM, Andy Smith <[email protected]> =
wrote:
>=20
>=20
> On Wed, Jul 29, 2026 at 04:38:55PM -0600, Casey Deccio wrote:
>> - /home is mounted on /dev/md125, which is a RAID 5 consisting of =
three disks, formatted with ext4
>=20
> I did not see anywhere in your email the details of what hardware =
these
> disks are.

Thanks.  Correct, I omitted that.  These are all:

Seagate BarraCuda 5TB Internal Hard Drive HDD =E2=80=93 2.5 Inch SATA =
6Gb/s 5400 RPM 128MB

>=20
>> I'm not really sure what else I might try, and I'm reaching out to =
the community to see if anyone has any suggestions for identifying and =
fixing the problem.
>=20
> I would start by a process of simplification. For example, does the =
sync
> still take a long time when /home is not actually mounted? If so then
> the issue is with your NVMe.

Thanks for the great idea.  I just tested your proposed setup.  I =
unmounted /home and started the file transfer to /root (on the NVMe).  =
While the transfer was happening, I ran `time sync` again.  The times =
were much, much lower than what I was seeing when /home was mounted on =
/dev/md125.

> If it does occur when /home is mounted, then it is the disks in the =
RAID
> setup or the RAID setup itself. Does each individual disk perform
> anomalously?
>=20
> "fio" is a good tool,to carry out performance testing of storage
> devices. Have a look what the random 4k read and write I/O is like in
> terms of IOPS, and the sequential disk transfer rate in MB/s for reads
> or writes. You can try that on the array and then break the array and
> try it on each individual disk. Obviously this is destructive to the
> data.
>=20
> If the performance is as expected on a per-disk basis but terrible =
only
> with RAID, it may be time to post to the linux-raid mailing list as =
this
> is unlikely to be a Debian-specific issue. I would then also be =
checking
> if it happens in other RAID configurations like RAID-1.
>=20
> 150 IOPS of random reads (not coming from RAM) is reasonable for a
> single 7.2kRPM SATA disk. You could expect 2-3x this from a RAID-5 of
> three identical disks.
>=20
> Write IOPS is a bit more difficult to guess and will depend on whether
> you have disabled the write cache in the drives themselves. Due to the
> RAID-5 write penalty (read-modify-write) you may expect only 50-70% of =
a
> single disk's IOPS for random writes.

Thanks for that very helpful information.  I've run fio on the =
still-assembled array, and here's what I get for read/write:

  read: IOPS=3D237
  write: IOPS=3D79

I'm not sure exactly what to make of that...

> Of course, if your sd[abc] are SSDs then there should be plenty of =
IOPS
> available and those sync times are very weird.

Right.  As mentioned above, these are not SSDs.

>=20
> "iostat -x 5" can give some more hints about what is going on.
>=20
> If there is other I/O happening that you aren't expecting then "iotop"
> can help identify it.

Things seem to be idle when the file transfer is not happening.  When it =
is, iotop shows about 100% utilization for /dev/md125 and about 50 -  =
75% utilization of /dev/sd{a,b,c}.  I haven't done comparisons with =
anything else at this point to see how this compares.  Thanks in the =
mean time for the idea.

Casey=