Understand limitations of lvremove snapshot
Divyam Goel <[email protected]> Thu, 17 Jul 2025 10:11:38 -0700
| Newsgroups | dev.linux.lists.lvm-devel |
|---|---|
| Message-ID | <CACefT4cQ7cAf+iqLUjxCn7aC7Xn+WwNNbHdXrBZ3mouSEcNCeQ@mail.gmail.com> |
Hi Team,
Good day!
I have a setup using lvm thin pool with thin snapshots for the logical
volumes. I noticed that when I delete these snapshots there is a IO
pause during the time lvremove command holds the critical section
lock. This IO pause seems to be pretty significant in the order of 20
seconds on my high-performance NVMe setup. I have appended all the
debug information below.
My questions are:
- Is this IO pause always expected?
- What factors influence the duration of this pause and how can we
mitigate it or completely avoid it?
Thank you very much for all your help.
Best,
Divyam
### Debug information is as follows
## LVM version
```
root@init500-d4:/# lvm version
LVM version: 2.02.176(2) (2017-11-03)
Library version: 1.02.145 (2017-11-03)
Driver version: 4.42.0
Configuration: ./configure --build=3Dx86_64-linux-gnu --prefix=3D/usr
--includedir=3D${prefix}/include --mandir=3D${prefix}/share/man
--infodir=3D${prefix}/share/info --sysconfdir=3D/etc --localstatedir=3D/var
--disable-silent-rules --libdir=3D${prefix}/lib/x86_64-linux-gnu
--libexecdir=3D${prefix}/lib/x86_64-linux-gnu --runstatedir=3D/run
--disable-maintainer-mode --disable-dependency-tracking --exec-prefix=3D
--bindir=3D/bin --libdir=3D/lib/x86_64-linux-gnu --sbindir=3D/sbin
--with-usrlibdir=3D/usr/lib/x86_64-linux-gnu --with-optimisation=3D-O2
--with-cache=3Dinternal --with-clvmd=3Dcorosync --with-cluster=3Dinternal
--with-device-uid=3D0 --with-device-gid=3D6 --with-device-mode=3D0660
--with-default-pid-dir=3D/run --with-default-run-dir=3D/run/lvm
--with-default-locking-dir=3D/run/lock/lvm --with-thin=3Dinternal
--with-thin-check=3D/usr/sbin/thin_check
--with-thin-dump=3D/usr/sbin/thin_dump
--with-thin-repair=3D/usr/sbin/thin_repair --enable-applib
--enable-blkid_wiping --enable-cmdlib --enable-cmirrord
--enable-dmeventd --enable-dbus-service --enable-lvmetad
--enable-lvmlockd-dlm --enable-lvmlockd-sanlock --enable-lvmpolld
--enable-notify-dbus --enable-pkgconfig --enable-readline
--enable-udev_rules --enable-udev_sync
```
## Before deleting the snapshot
```
[email protected]:~# date && fio --blocksize=3D2M
--ioengine=3Dlibaio --readwrite=3Dwrite --numjobs=3D1 --direct=3D1 --iodept=
h=3D1
--end_fsync=3D1 --name=3Dfiowrite
--filename=3D/dev/mapper/pwx0-57414833981247139 --runtime=3D2m
--time_based=3D1
Thu Jul 17 01:08:03 UTC 2025
fiowrite: (g=3D0): rw=3Dwrite, bs=3D(R) 2048KiB-2048KiB, (W)
2048KiB-2048KiB, (T) 2048KiB-2048KiB, ioengine=3Dlibaio, iodepth=3D1
fio-3.7
Starting 1 process
Jobs: 1 (f=3D1): [W(1)][100.0%][r=3D0KiB/s,w=3D1998MiB/s][r=3D0,w=3D998 IOP=
S][eta 00m:00s]
fiowrite: (groupid=3D0, jobs=3D1): err=3D 0: pid=3D68396: Thu Jul 17 01:10:=
03 2025
write: IOPS=3D1824, BW=3D3649MiB/s (3826MB/s)(428GiB/120001msec)
slat (usec): min=3D77, max=3D15965, avg=3D247.70, stdev=3D87.94
clat (nsec): min=3D994, max=3D9362.8k, avg=3D299209.74, stdev=3D180516.=
14
lat (usec): min=3D275, max=3D25329, avg=3D547.07, stdev=3D197.48
clat percentiles (usec):
| 1.00th=3D[ 109], 5.00th=3D[ 133], 10.00th=3D[ 149], 20.00th=3D[=
180],
| 30.00th=3D[ 212], 40.00th=3D[ 235], 50.00th=3D[ 255], 60.00th=3D[=
269],
| 70.00th=3D[ 277], 80.00th=3D[ 379], 90.00th=3D[ 578], 95.00th=3D[=
685],
| 99.00th=3D[ 848], 99.50th=3D[ 955], 99.90th=3D[ 1516], 99.95th=3D[=
2073],
| 99.99th=3D[ 2933]
bw ( MiB/s): min=3D 1796, max=3D 5044, per=3D100.00%, avg=3D3655.49,
stdev=3D642.72, samples=3D239
iops : min=3D 898, max=3D 2522, avg=3D1827.69, stdev=3D321.38, s=
amples=3D239
lat (nsec) : 1000=3D0.01%
lat (usec) : 4=3D0.01%, 20=3D0.01%, 50=3D0.01%, 100=3D0.47%, 250=3D46.5=
8%
lat (usec) : 500=3D39.43%, 750=3D10.89%, 1000=3D2.22%
lat (msec) : 2=3D0.35%, 4=3D0.05%, 10=3D0.01%
cpu : usr=3D8.23%, sys=3D38.62%, ctx=3D219645, majf=3D0, minf=3D=
7292
IO depths : 1=3D100.0%, 2=3D0.0%, 4=3D0.0%, 8=3D0.0%, 16=3D0.0%, 32=3D=
0.0%, >=3D64=3D0.0%
submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, 64=
=3D0.0%, >=3D64=3D0.0%
complete : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, 64=
=3D0.0%, >=3D64=3D0.0%
issued rwts: total=3D0,218931,0,1 short=3D0,0,0,0 dropped=3D0,0,0,0
latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D1
Run status group 0 (all jobs):
WRITE: bw=3D3649MiB/s (3826MB/s), 3649MiB/s-3649MiB/s
(3826MB/s-3826MB/s), io=3D428GiB (459GB), run=3D120001-120001msec
```
## While the snapshot deletion was in progress
```
[email protected]:~# date && fio --blocksize=3D2M
--ioengine=3Dlibaio --readwrite=3Dwrite --numjobs=3D1 --direct=3D1 --iodept=
h=3D1
--end_fsync=3D1 --name=3Dfiowrite
--filename=3D/dev/mapper/pwx0-57414833981247139 --runtime=3D2m
--time_based=3D1
Thu Jul 17 01:12:16 UTC 2025
fiowrite: (g=3D0): rw=3Dwrite, bs=3D(R) 2048KiB-2048KiB, (W)
2048KiB-2048KiB, (T) 2048KiB-2048KiB, ioengine=3Dlibaio, iodepth=3D1
fio-3.7
Starting 1 process
Jobs: 1 (f=3D1): [W(1)][100.0%][r=3D0KiB/s,w=3D2182MiB/s][r=3D0,w=3D1091
IOPS][eta 00m:00s]
fiowrite: (groupid=3D0, jobs=3D1): err=3D 0: pid=3D71680: Thu Jul 17 01:14:=
17 2025
write: IOPS=3D1846, BW=3D3693MiB/s (3872MB/s)(433GiB/120002msec)
slat (usec): min=3D140, max=3D21376k, avg=3D318.43, stdev=3D45413.12
clat (usec): min=3D26, max=3D3981, avg=3D222.19, stdev=3D72.46
lat (usec): min=3D264, max=3D21376k, avg=3D540.77, stdev=3D45412.87
clat percentiles (usec):
| 1.00th=3D[ 103], 5.00th=3D[ 127], 10.00th=3D[ 143], 20.00th=3D[=
167],
| 30.00th=3D[ 192], 40.00th=3D[ 215], 50.00th=3D[ 229], 60.00th=3D[=
247],
| 70.00th=3D[ 262], 80.00th=3D[ 269], 90.00th=3D[ 273], 95.00th=3D[=
277],
| 99.00th=3D[ 363], 99.50th=3D[ 498], 99.90th=3D[ 1012], 99.95th=3D[=
1303],
| 99.99th=3D[ 1926]
bw ( MiB/s): min=3D 1840, max=3D 5484, per=3D100.00%, avg=3D4486.12,
stdev=3D598.22, samples=3D197
iops : min=3D 920, max=3D 2742, avg=3D2243.03, stdev=3D299.11, s=
amples=3D197
lat (usec) : 50=3D0.01%, 100=3D0.81%, 250=3D60.87%, 500=3D37.82%, 750=
=3D0.27%
lat (usec) : 1000=3D0.12%
lat (msec) : 2=3D0.10%, 4=3D0.01%
cpu : usr=3D7.83%, sys=3D34.36%, ctx=3D221768, majf=3D0, minf=3D=
22
IO depths : 1=3D100.0%, 2=3D0.0%, 4=3D0.0%, 8=3D0.0%, 16=3D0.0%, 32=3D=
0.0%, >=3D64=3D0.0%
submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, 64=
=3D0.0%, >=3D64=3D0.0%
complete : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, 64=
=3D0.0%, >=3D64=3D0.0%
issued rwts: total=3D0,221554,0,1 short=3D0,0,0,0 dropped=3D0,0,0,0
latency : target=3D0, window=3D0, percentile=3D100.00%, depth=3D1
Run status group 0 (all jobs):
WRITE: bw=3D3693MiB/s (3872MB/s), 3693MiB/s-3693MiB/s
(3872MB/s-3872MB/s), io=3D433GiB (465GB), run=3D120002-120002msec
```
## Snapshot delete in progress
```
[email protected]:~# date && lvm lvremove -vvvv -dddddd
--noudevsync -y pwx0/672921901357301927
Thu Jul 17 01:12:37 UTC 2025
#lvmcmdline.c:2831 Parsing: lvremove -vvvv -dddddd
--noudevsync -y pwx0/672921901357301927
...
#lvmcmdline.c:3050 Completed: lvremove -vvvv -dddddd
--noudevsync -y pwx0/672921901357301927
```
## From above we can see max latency is 25 ms when no snapshot delete
is running and 21 sec when snapshot delete is in progress.
### More debug information is as follows
```
[email protected]:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:8 0 1.5T 0 disk
=E2=94=9C=E2=94=80nvme0n1p3 259:12 0 1.5T 0 part
=E2=94=82 =E2=94=9C=E2=94=80vgroot-lvswap01 253:1 0 4G 0=
lvm
=E2=94=82 =E2=94=94=E2=94=80vgroot-lvroot 253:0 0 1.5T 0=
lvm /
=E2=94=9C=E2=94=80nvme0n1p1 259:10 0 1G 0 part /=
boot/efi
=E2=94=94=E2=94=80nvme0n1p2 259:11 0 500M 0 part /=
boot
nvme3n1 259:9 0 1.5T 0 disk
=E2=94=94=E2=94=80md127 9:127 0 5.8T 0 raid0
=E2=94=9C=E2=94=80pwx0-pxpool_tmeta 253:2 0 15G 0 lvm
=E2=94=82 =E2=94=94=E2=94=80pwx0-pxpool-tpool 253:4 0 5.8T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-248435070782584467 253:10 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-729796226867285023 253:8 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-681871867032118384 253:17 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-pxMetaFS 253:6 0 64G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-6467409125236334 253:15 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-353177431893931550 253:13 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-369272389739017953 253:9 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-294122483306511515 253:18 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-57414833981247139 253:7 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-393368701089705163 253:16 0 3T 0=
lvm
=E2=94=82 =E2=94=94=E2=94=80pwx0-pxpool 253:5 0 5.8T 0=
lvm
=E2=94=94=E2=94=80pwx0-pxpool_tdata 253:3 0 5.8T 0 lvm
=E2=94=94=E2=94=80pwx0-pxpool-tpool 253:4 0 5.8T 0 lvm
=E2=94=9C=E2=94=80pwx0-248435070782584467 253:10 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-729796226867285023 253:8 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-681871867032118384 253:17 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-pxMetaFS 253:6 0 64G 0 lvm
=E2=94=9C=E2=94=80pwx0-6467409125236334 253:15 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-353177431893931550 253:13 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-369272389739017953 253:9 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-294122483306511515 253:18 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-57414833981247139 253:7 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-393368701089705163 253:16 0 3T 0 lvm
=E2=94=94=E2=94=80pwx0-pxpool 253:5 0 5.8T 0 lvm
nvme6n1 259:7 0 1.5T 0 disk
nvme9n1 259:5 0 1.5T 0 disk
nvme2n1 259:1 0 1.5T 0 disk
=E2=94=94=E2=94=80md127 9:127 0 5.8T 0 raid0
=E2=94=9C=E2=94=80pwx0-pxpool_tmeta 253:2 0 15G 0 lvm
=E2=94=82 =E2=94=94=E2=94=80pwx0-pxpool-tpool 253:4 0 5.8T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-248435070782584467 253:10 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-729796226867285023 253:8 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-681871867032118384 253:17 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-pxMetaFS 253:6 0 64G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-6467409125236334 253:15 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-353177431893931550 253:13 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-369272389739017953 253:9 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-294122483306511515 253:18 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-57414833981247139 253:7 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-393368701089705163 253:16 0 3T 0=
lvm
=E2=94=82 =E2=94=94=E2=94=80pwx0-pxpool 253:5 0 5.8T 0=
lvm
=E2=94=94=E2=94=80pwx0-pxpool_tdata 253:3 0 5.8T 0 lvm
=E2=94=94=E2=94=80pwx0-pxpool-tpool 253:4 0 5.8T 0 lvm
=E2=94=9C=E2=94=80pwx0-248435070782584467 253:10 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-729796226867285023 253:8 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-681871867032118384 253:17 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-pxMetaFS 253:6 0 64G 0 lvm
=E2=94=9C=E2=94=80pwx0-6467409125236334 253:15 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-353177431893931550 253:13 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-369272389739017953 253:9 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-294122483306511515 253:18 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-57414833981247139 253:7 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-393368701089705163 253:16 0 3T 0 lvm
=E2=94=94=E2=94=80pwx0-pxpool 253:5 0 5.8T 0 lvm
pxd!pxd729796226867285023 252:2 0 1G 0 disk
/var/lib/osd/mounts/1gb-d3
nvme5n1 259:3 0 1.5T 0 disk
nvme8n1 259:4 0 1.5T 0 disk
pxd!pxd57414833981247139 252:1 0 3T 0 disk
/var/lib/osd/mounts/3tb-d3
nvme1n1 259:0 0 1.5T 0 disk
=E2=94=94=E2=94=80md127 9:127 0 5.8T 0 raid0
=E2=94=9C=E2=94=80pwx0-pxpool_tmeta 253:2 0 15G 0 lvm
=E2=94=82 =E2=94=94=E2=94=80pwx0-pxpool-tpool 253:4 0 5.8T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-248435070782584467 253:10 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-729796226867285023 253:8 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-681871867032118384 253:17 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-pxMetaFS 253:6 0 64G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-6467409125236334 253:15 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-353177431893931550 253:13 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-369272389739017953 253:9 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-294122483306511515 253:18 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-57414833981247139 253:7 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-393368701089705163 253:16 0 3T 0=
lvm
=E2=94=82 =E2=94=94=E2=94=80pwx0-pxpool 253:5 0 5.8T 0=
lvm
=E2=94=94=E2=94=80pwx0-pxpool_tdata 253:3 0 5.8T 0 lvm
=E2=94=94=E2=94=80pwx0-pxpool-tpool 253:4 0 5.8T 0 lvm
=E2=94=9C=E2=94=80pwx0-248435070782584467 253:10 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-729796226867285023 253:8 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-681871867032118384 253:17 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-pxMetaFS 253:6 0 64G 0 lvm
=E2=94=9C=E2=94=80pwx0-6467409125236334 253:15 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-353177431893931550 253:13 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-369272389739017953 253:9 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-294122483306511515 253:18 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-57414833981247139 253:7 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-393368701089705163 253:16 0 3T 0 lvm
=E2=94=94=E2=94=80pwx0-pxpool 253:5 0 5.8T 0 lvm
nvme4n1 259:2 0 1.5T 0 disk
=E2=94=94=E2=94=80md127 9:127 0 5.8T 0 raid0
=E2=94=9C=E2=94=80pwx0-pxpool_tmeta 253:2 0 15G 0 lvm
=E2=94=82 =E2=94=94=E2=94=80pwx0-pxpool-tpool 253:4 0 5.8T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-248435070782584467 253:10 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-729796226867285023 253:8 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-681871867032118384 253:17 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-pxMetaFS 253:6 0 64G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-6467409125236334 253:15 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-353177431893931550 253:13 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-369272389739017953 253:9 0 1G 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-294122483306511515 253:18 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-57414833981247139 253:7 0 3T 0=
lvm
=E2=94=82 =E2=94=9C=E2=94=80pwx0-393368701089705163 253:16 0 3T 0=
lvm
=E2=94=82 =E2=94=94=E2=94=80pwx0-pxpool 253:5 0 5.8T 0=
lvm
=E2=94=94=E2=94=80pwx0-pxpool_tdata 253:3 0 5.8T 0 lvm
=E2=94=94=E2=94=80pwx0-pxpool-tpool 253:4 0 5.8T 0 lvm
=E2=94=9C=E2=94=80pwx0-248435070782584467 253:10 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-729796226867285023 253:8 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-681871867032118384 253:17 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-pxMetaFS 253:6 0 64G 0 lvm
=E2=94=9C=E2=94=80pwx0-6467409125236334 253:15 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-353177431893931550 253:13 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-369272389739017953 253:9 0 1G 0 lvm
=E2=94=9C=E2=94=80pwx0-294122483306511515 253:18 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-57414833981247139 253:7 0 3T 0 lvm
=E2=94=9C=E2=94=80pwx0-393368701089705163 253:16 0 3T 0 lvm
=E2=94=94=E2=94=80pwx0-pxpool 253:5 0 5.8T 0 lvm
nvme7n1 259:6 0 1.5T 0 disk
[email protected]:~# mdadm -D /dev/md127
/dev/md127:
Version : 1.2
Creation Time : Fri Jul 11 20:17:18 2025
Raid Level : raid0
Array Size : 6250725376 (5.82 TiB 6.40 TB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Fri Jul 11 20:17:18 2025
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Chunk Size : 1024K
Consistency Policy : none
Name : any:pwx0
UUID : f6ae3624:0d629903:d13686a9:146fc8cf
Events : 0
Number Major Minor RaidDevice State
0 259 1 0 active sync /dev/nvme2n1
1 259 9 1 active sync /dev/nvme3n1
2 259 2 2 active sync /dev/nvme4n1
3 259 0 3 active sync /dev/nvme1n1
[email protected]:~# lvs
LV VG Attr LSize Pool Origin
Data% Meta% Move Log Cpy%Sync Convert
248435070782584467 pwx0 Vwi-a-t--- <2.93t pxpool 57414833981247139 99.=
93
294122483306511515 pwx0 Vwi-a-t--- <2.93t pxpool 57414833981247139 99.=
93
353177431893931550 pwx0 Vwi-a-t--- 1.00g pxpool 729796226867285023 4.7=
8
369272389739017953 pwx0 Vwi-a-t--- 1.00g pxpool 729796226867285023 4.7=
8
393368701089705163 pwx0 Vwi-a-t--- <2.93t pxpool 57414833981247139 99.=
93
57414833981247139 pwx0 Vwi-a-t--- <2.93t pxpool 99.=
93
6467409125236334 pwx0 Vwi-a-t--- 1.00g pxpool 729796226867285023 4.7=
8
672921901357301927 pwx0 Vwi-a-t--- <2.93t pxpool 57414833981247139 99.=
93
681871867032118384 pwx0 Vwi-a-t--- 1.00g pxpool 729796226867285023 4.7=
8
729796226867285023 pwx0 Vwi-a-t--- 1.00g pxpool 4.7=
8
84211885115688699 pwx0 Vwi-a-t--- <2.93t pxpool 57414833981247139 99.=
93
pxMetaFS pwx0 Vwi-a-t--- 64.00g pxpool 0.0=
5
pxpool pwx0 twi-aot--- <5.78t
89.58 17.56
pxreserve pwx0 -wi------k 15.00g
lvroot vgroot -wi-ao---- 1.45t
lvswap01 vgroot -wi-a----- 4.00g
[email protected]:~# vgs
VG #PV #LV #SN Attr VSize VFree
pwx0 1 14 0 wz--n- 5.82t 0
vgroot 1 2 0 wz--n- 1.45t 0
[email protected]:~# pvs
PV VG Fmt Attr PSize PFree
/dev/md127 pwx0 lvm2 a-- 5.82t 0
/dev/nvme0n1p3 vgroot lvm2 a-- 1.45t 0
[email protected]:~# lvdisplay pwx0/57414833981247139
--- Logical volume ---
LV Path /dev/pwx0/57414833981247139
LV Name 57414833981247139
VG Name pwx0
LV UUID d12Vj9-J51S-Vsis-rt4X-G53n-n7wd-bzWyio
LV Write Access read/write
LV Creation host, time init500-d3, 2025-07-11 20:22:56 +0000
LV Pool name pxpool
LV Status available
# open 2
LV Size <2.93 TiB
Mapped size 99.93%
Current LE 768000
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 16384
Block device 253:7
[email protected]:~# lvdisplay pwx0/672921901357301927
--- Logical volume ---
LV Path /dev/pwx0/672921901357301927
LV Name 672921901357301927
VG Name pwx0
LV UUID vuZ9j9-lhN9-FGxe-qCVp-Tchf-zJsd-gvhJNt
LV Write Access read/write
LV Creation host, time init500-d3, 2025-07-16 22:51:10 +0000
LV Pool name pxpool
LV Thin origin name 57414833981247139
LV Status available
# open 2
LV Size <2.93 TiB
Mapped size 99.93%
Current LE 768000
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 16384
Block device 253:14
```