Re: striped LV and expected performance

When doing direct, typically it is all about round trip latency.  And
with a VM you add the VM latency on top of the underlying disk
latency.   See what the await is in iostat on the VM vs the physical
(to real disks and physical to SAN) when running the test.  Physical
to local disk has lower latency(seen .05ms) compared to physical to
san(typically once the write gets to the SAN the SAN has to make a
copy in a 2nd SAN nodes cache, around .2ms-depends on array arch), and
vm over either is higher that the pure physical).

On Tue, Jul 22, 2025 at 6:24=E2=80=AFAM Henry, Andrew <[email protected]>=
 wrote:
>
> =C2=B4Thanks for all the tips and info.  I think I've come to the conclus=
ion that the reason I'm not seeing any speed increase from striping across =
the virtual disks is because there's only one communication "path" from the=
 guest OS to the virtual host/SAN.  I've tried all combinations of LVM stri=
ping on a home server and the striping behaves exactly as I expect it to, e=
ven according to the hard drive manufacturer's spec. sheet for read/write s=
peeds that I'm getting on those physical disks when I stripe them, please I=
 see linear speed increases for every disk I add to the striped LVM.
>
> Regarding the very low write speeds with dd in a VM though... On my home =
server, with 4k block sizes in the entire stack, I get about 70MB/s on my m=
echanical disks.  In the virtual environment with solid state, I'm getting =
10MB/s.  This is irrespective of whether I partition those virtual drives o=
r leave them as raw, whether I use 3 striped devices or just one device in =
my VG... it doesn't matter.  By the way, the reason I'm only getting 10 MB/=
s is due to my oflag=3Ddirect.  If I skip that then the speeds are much hig=
her due to caching, but I wanted to test without caching.
>
> So, I've ruled out the Linux guest OS/disk configuration as the problem. =
 It must be something on the VM host side that is causing the issue, or som=
ething to do with "expectations of write patterns in a VM"?  For instance, =
if I create the stripe as 4k block sizes in lvm, but write a test file with=
 dd using a block size of 4MB, then suddenly I'm getting 700MB/s write spee=
ds to disk.  I get it, the higher the write size the better the speed, but =
When I compare like for like with physical mechanical disks and I'm getting=
 higher speeds with the old mechanical disks than in the VM, it just feels =
wrong.  But maybe I'm "comparing it wrong"?
>
> /AH