Re: Unexpected behavior of reencrypt performed after partition resize

Hi,

I've tried to reproduce it but so far with no luck. More question below:

On 01/03/2024 13:27, Aleksander Łukasz wrote:
> Hi,
> 
> I'm trying to debug a mysterious data corruption apparently happening
> during cryptsetup reencryption step that follows partition resize.
> 
> I can deterministically reproduce the following behavior from the
> initramfs boot stage when reencrypting root partition:
> 
> ```
> BusyBox v1.35.0 (Debian 1:1.35.0-4+b3) built-in shell (ash)
> Enter 'help' for a list of built-in commands.
> 
> ~ # while true; do \
>   >     (echo -n "hello" | cryptsetup open /dev/vda4 test ) \
>   >     && dd if=/dev/mapper/test | md5sum \
>   >     && cryptsetup close /dev/mapper/test \
>   >     && (echo -n "hello" | cryptsetup reencrypt /dev/vda4) done
> d53129dd093a5ac0be99995bd5e46edc  -
> 4958175+0 records in
> 4958175+0 records out
> Finished, time 01m28s, 2420 MiB written, speed  27.5 MiB/s
> c4d10b8b556fe4ea1c2430f866b6aa76  -
> 4958175+0 records in
> 4958175+0 records out
> Finished, time 01m25s, 2420 MiB written, speed  28.4 MiB/s
> c4d10b8b556fe4ea1c2430f866b6aa76  -
> 4958175+0 records in
> 4958175+0 records out
> Finished, time 01m30s, 2420 MiB written, speed  26.7 MiB/s
> c4d10b8b556fe4ea1c2430f866b6aa76  -
> 4958175+0 records in
> 4958175+0 records out
> Finished, time 01m34s, 2420 MiB written, speed  25.7 MiB/s
> c4d10b8b556fe4ea1c2430f866b6aa76  -
> 4958175+0 records in
> 4958175+0 records out
> ```

Would you mind sharing whole vda device partition table _before_ and 
_after_ the aforementioned vda4 resize? The dm-crypt device is unaligned 
to 1MiB (not an issue at the moment), but that's what I plan to focus on 
with my tests.

Also, please share a LUKS2 header dump (luksDump command) output for 
luks device before and after the first reencryption run takes place.

> 
> As you can see, first reencryption changes contents of the LUKS
> container (as read via the dm-crypt mapper) and all the subsequent ones
> do not (which I would expect to always be the case).

Would you mind completely skip the reencryption step and just overwrite 
the device with data read from the same device? But write directly to 
vda4 (that's what reencryption actually does).

First lets try without direct flag (I skip 16MiB, default LUKS2 header size)

dd if=/dev/vda4 of=/dev/vda4 seek=32768 skip=32768

or alternatively with direct flag like (again this assumes default LUKS2 
header size 16 MiB).

dd if=/dev/vda4 of=/dev/vda4 bs=1M seek=16 skip=16 iflag=direct 
oflag=direct,dsync

> 
> I can trigger this behavior by resizing /dev/vda4 partition just before
> booting into initramfs, using any of the tools like systemd-repart or
> growpart
> (which extends /dev/vda4 to the end of also just extended /dev/vda). If

How did you extend vda itself? And how did you pass the change in the 
guest VM?

> 
> 
> Also, I've tried to reproduce the issue using disk image file and a loop
> device, but did not succeed - that is I could not force the first
> iteration after resize to change checksum - all were always the same.

This suggests we might deal with something else than a bug in 
reencryption code.

> Any tips on what I could be doing wrong or what to look for would be
> very appreciated as I'm running out of idea
"lsblk -t /dev/vda" output might also help (again before and after the 
resize)

Could you show me also last 2 cycles of reencryption debug output?
Everything in between

# Progress ????????????, device_size 2538585600
# Next reencryption offset will be X sectors.
# Next reencryption chunk size will be Y sectors).
(...)
# Progress ????????????, device_size 2538585600
# Next reencryption offset will be XX sectors.
# Next reencryption chunk size will be YY sectors).
(...)
# Next reencryption offset will be 2538585600 sectors.
# Next reencryption chunk size will be 0 sectors).
# Destroying keyslot ??.
# Acquiring write lock for device /dev/vda4.

But it's all blind guesses on my side.

Kind regards
Ondrej