Unexpected behavior of reencrypt performed after partition resize
Aleksander Łukasz <[email protected]> Fri, 1 Mar 2024 13:27:06 +0100
| Newsgroups | dev.linux.lists.cryptsetup |
|---|---|
| Message-ID | <[email protected]> |
Hi, I'm trying to debug a mysterious data corruption apparently happening during cryptsetup reencryption step that follows partition resize. I can deterministically reproduce the following behavior from the initramfs boot stage when reencrypting root partition: ``` BusyBox v1.35.0 (Debian 1:1.35.0-4+b3) built-in shell (ash) Enter 'help' for a list of built-in commands. ~ # while true; do \ > (echo -n "hello" | cryptsetup open /dev/vda4 test ) \ > && dd if=/dev/mapper/test | md5sum \ > && cryptsetup close /dev/mapper/test \ > && (echo -n "hello" | cryptsetup reencrypt /dev/vda4) done d53129dd093a5ac0be99995bd5e46edc - 4958175+0 records in 4958175+0 records out Finished, time 01m28s, 2420 MiB written, speed 27.5 MiB/s c4d10b8b556fe4ea1c2430f866b6aa76 - 4958175+0 records in 4958175+0 records out Finished, time 01m25s, 2420 MiB written, speed 28.4 MiB/s c4d10b8b556fe4ea1c2430f866b6aa76 - 4958175+0 records in 4958175+0 records out Finished, time 01m30s, 2420 MiB written, speed 26.7 MiB/s c4d10b8b556fe4ea1c2430f866b6aa76 - 4958175+0 records in 4958175+0 records out Finished, time 01m34s, 2420 MiB written, speed 25.7 MiB/s c4d10b8b556fe4ea1c2430f866b6aa76 - 4958175+0 records in 4958175+0 records out ``` As you can see, first reencryption changes contents of the LUKS container (as read via the dm-crypt mapper) and all the subsequent ones do not (which I would expect to always be the case). This first call changing the contents is what I'm trying to understand, as to the best of my knowledge, this shouldn't happen in any circumstances, no data shifts take place here and observed behavior of checksum changing is unexpected. Am I right or do I miss something? I can trigger this behavior by resizing /dev/vda4 partition just before booting into initramfs, using any of the tools like systemd-repart or growpart (which extends /dev/vda4 to the end of also just extended /dev/vda). If I don't do such resize, then the effect of first iteration of reencryptions changing the checksum is not present. This is Debian stable and Debian Sid, so on Linux 6.1.0-18-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.76-1 (2024-02-01) x86_64 GNU/Linux) + cryptsetup 2.6.1 flags: UDEV BLKID KEYRING KERNEL_CAPI) and on Linux 6.6.15-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.6.15-2 (2024-02-04) x86_64 GNU/Linux + cryptsetup 2.7.0 flags: UDEV BLKID KEYRING KERNEL_CAPI HW_OPAL. All running in QEMU. So far I've tried: - reviewing output of running with --debug , - reencrypting with and without active mapping open, - resizing partition after full boot or from initramfs phase, using various tools for that, - manipulating partition size change step, - reviewing LUKS header changes between first and subsequent reencryptions, - measuring checksums of partition segments related to previous and new size separately (both do change), - running reencrypt with various --resilience modes, - ensuring cryptsetup-initramfs tooling did not interfere with reencryption, - ... . Could always reproduce the behavior mentioned and did not found any obvious (to my untrained eyes) indications of what could be happening wrong. I've also reviewed https://github.com/mbroz/cryptsetup/blob/main/FAQ.md and saw short mention of corruption risk in context of partition resizing under the section 2.8, but I think it does not apply here? Also, I'm aware of resizing related mention in cryptsetup 2.7 release notes, but I also believe this is not such a case (and I did not observe the warning mentioned there). Also, I've tried to reproduce the issue using disk image file and a loop device, but did not succeed - that is I could not force the first iteration after resize to change checksum - all were always the same. Any tips on what I could be doing wrong or what to look for would be very appreciated as I'm running out of ideas!