Re: NetBSD Raid5, slow write speeds, using big disks?!

On 6/23/26 8:30 PM, Robert Elz wrote:
>      Date:        Tue, 23 Jun 2026 16:03:42 +0200
>      From:        smurfd <[email protected]>
>      Message-ID:  <[email protected]>
>
>    | Its tiresome though, doing a change then waiting 8 to 72 hours and
>    | seeing that the change had no effect.
>
> While you're experimenting, make a much smaller raid array, one that is
> just 10GB or so (ok, at least a couple of multiples of the amount of RAM
> in the system) which will init, newfs, ... all very quickly, and is big
> enough to test performance.  You don't need to do the whole thing for that.
> (Doing it just on the hope that this will be the last time, will almost
> certainly have Murphy intervene, and once you have the basics correct,
> just making the number of blocks bigger won't change anything except its
> size.)   So for the raid partitions on wd0 (etc) use 1/2 of the size you
> want (for testing) for each (for a 3 drive raid5).
>
>    | My raid experience is, as you have guessed, small... coming from zfs,
>    | where it was kind of plug and play, i expected some push back here...
>    | not this much though.
>
> Beyond what you seem to be doing, just make sure that the filesystems
> inside the raid are correctly aligned on raid stripes (which will be
> 64 blocks, 32KB, assuming you used the same params as before, except
> with a 3 drive set instead of 4).   That is, everything should be aligned
> on (and sized to be) a multiple of 64 blocks within the raid, and your
> filesystem block size (-b to newfs) should be 32768 or 65536 (you can set
> a smaller frag size, -f, any binary fraction of the block size, down to 1/8,
> ie: 4096 for 32K blocksize, is OK, and uses less space for small files,
> including directories.)   You could also use SectPerSU as 16 if you're
> needing to conserve space (at the expense of some speed) - if you do that
> all the numbers (bigger than 4) above (in this paragraph) can be halved
> as well.  You'd use less RAM for the raidset, and with a smaller blocksize
> potentially have less wasted space in the filesystem (though that is a
> complex thing to evaluate.)  I wouldn't recommend < 16 though, the performance
> loss then is likely to be more noticeable, but you can certainly test it,
> it needs to be a power of 2, and is a number of blocks - the actual raidset
> stripe size is then the number of data drives (don't count the parity)
> multiplied by that number.   [Yes, I know, with raid5, all drives are data
> drives technically, all are also parity drives, but it is simpler to forget
> that optimisation, and just say that 1 of the drives is parity, the remainder
> are data .. just don't try and speculate which drive is which, as they
> really aren't.]
>
> kre
>
A bit of an update. I did not see any big difference in raid5, if i use 
3 or 4 drives.
Not either if i use like 64k blocks / align or 1m... this has me worried...
So, Tried to do a raid10, and got blazing speeds! with these parameters, 
cutting away some parts...
and yes i know i can remove the -s 1g for the log, if im not going ot 
use it... and yes labels are incorrectly named raid5 even with the raid10...
Tried to do a raid5 with these parameters aswell, but no real 
difference, i think it is the speed i pasted below for raid5...
gpt add -t raid -l raid5@wd0 -b 1m -a 1m -s 1000g wd0 ****
gpt add -a 1m -b 1m -t ffs -l island raid1
gpt add -a 1m -b 1m -t ffs -l island raid2
newfs -O2 -a 1m -b 64k -s -1g /dev/dk7
newfs -O2 -a 1m -b 64k -s -1g /dev/dk8

gpt add -a 1m -b 1m -t ffs -l island raid0
dkctl raid0 addwedge island3 1048576 4191152128 ffs
newfs -O2 -a 1m -b 64k -s -1g /dev/dk9

Where for both raid1 and raid0 i used
# sectPerSU SUsPerParityUnit SUsPerReconUnit RAID_level_0
128 1 1 0/1 depeding on raid type

START queue
fifo 127

4194304000 bytes transferred in 8.255 secs (508092549 bytes/sec) CHAZAAAM!
The best i got for raid5 so far, is:
4194304000 bytes transferred in 107.170 secs (39136922 bytes/sec)
So raid10 is the backup plan :)

after re-reading your mail, i will try raid5 a few more attempts...
In raid5.conf you suggest: 16 1 1 5 that would have me use these 
commands while doing the disks and raid
gpt add -t raid -l raid5@wd0 -b 64k -a 32k -s 1000g wd0 ****
gpt add -a 32k -b 64k -t ffs -l island raid5
newfs -O2 -a 32k -b 64k -s -1g /dev/dk7

or did i missunderstand something?
Many many thanks again, for bearing with me!!
br smurfd