Re: Disk I/O slowness

On 7/29/26 15:38, Casey Deccio wrote:
> Hi all,
> 
> I'm experiencing some unexpected disk I/O latency on a newly built server.  Here are some of the details:
> 
> - Supermicro motherboard with Intel PCH RAID [1]
> - Debian 13.6 installed
> - / is mounted on /dev/nvme0n1p2, which is an NVMe SSD, formatted with ext4
> - /home is mounted on /dev/md125, which is a RAID 5 consisting of three disks, formatted with ext4
> 
> $ sudo mdadm -D /dev/md125
> /dev/md125:
>           Container : /dev/md/imsm1, member 0
>          Raid Level : raid5
>          Array Size : 9279152128 (8.64 TiB 9.50 TB)
>       Used Dev Size : 4639576064 (4.32 TiB 4.75 TB)
>        Raid Devices : 3
>       Total Devices : 3
> 
>               State : active
>      Active Devices : 3
>     Working Devices : 3
>      Failed Devices : 0
>       Spare Devices : 0
> 
>              Layout : left-asymmetric
>          Chunk Size : 64K
> 
> Consistency Policy : resync
> 
> 
>                UUID : 2fbd736b:9b750e2a:e723b80c:df2f2c33
>      Number   Major   Minor   RaidDevice State
>         2       8        0        0      active sync   /dev/sda
>         1       8       16        1      active sync   /dev/sdb
>         0       8       32        2      active sync   /dev/sdc
> 
> 
> I'm getting some weird latency issues with fsync on the RAID disk (/home on /dev/md125):
> (Caveat: I'm doing a file transfer that is writing to /home, so I might expect some latency, but nothing like this.)
> 
> $ for i in {1..5}; do time sync; done 2>&1 | grep real
> real	0m21.647s
> real	0m11.770s
> real	0m5.234s
> real	0m3.023s
> real	0m0.462s
> 
> Even when I turn off the file transfer, I still get something like this:
> 
> $ for i in {1..10}; do time sync; done 2>&1 | grep real
> real	0m0.147s
> real	0m0.077s
> real	0m0.003s
> real	0m0.004s
> real	0m0.003s
> 
> I've tried turning on and off smart monitoring with smartctl to see if that would help, and it didn't seem to help.
> 
> I'm not really sure what else I might try, and I'm reaching out to the community to see if anyone has any suggestions for identifying and fixing the problem.
> 
> Thanks,
> Casey
> 
> [1] https://www.supermicro.com/manuals/other/Intel_PCH_RAID_Config.pdf
> 


What is the model number/ part number of your motherboard?


Have you disabled Intel PCH RAID in Setup -- e.g. so that the 
motherboard disk interface ports appear as non-RAID ports (SATA/SAS)?


How are the disk drives connected to the motherboard?  What is the speed 
rating of the cables?  Do the cables have locking connectors?  If you 
are using backplanes/ racks, what is their speed rating?


Have you run SMART long tests on each of the three disk drives in the 
RAID5?  If not, please do so now.  Please post the extended reports from 
the top through and including the end of the section "SMART Attributes 
Data Structure revision number: ...".


How are you transferring files?  Please post your console session.


David