Re: Creating detached dm-integrity hashes for existing disk

On 11/21/22 15:50, Ciprian Craciun wrote:
> Hello all!  (Please keep me in CC, as I'm not subscribed to the mailing list.)
> 
> I've sent the following email two months ago, but I assume that either
> it slipped through the cracks or nobody has any experience with the
> use case I'm mentioning.  However, I'll try to submit this question
> one more time, perhaps with more luck this time.  Thanks for the time.

:-)

Actually I planned to respond, just had not time to test it.

TL;DR: the recalculation flag is persistent, it is ok that it remains set.


It is there because if the data device is resized later (and as it is separate,
it can happen), you need to recalculate the rest of the device (also automatically).

Check "integrity dump <metadata_device>", you should see recalculation offset,
and also the recalc sector pointer e.g.:

# integritysetup dump /tmp/dm-integrity-tests-meta
Info for integrity device /tmp/dm-integrity-tests-meta.
superblock_version 5
log2_interleave_sectors 0
integrity_tag_size 32
journal_sections 7
provided_data_sectors 262144
sector_size 4096
recalc_sector 262144
log2_blocks_per_bitmap 12
flags recalculating fix_hmac

Here you can see, that recalc_sector is the same as data sectors, so the recalculation
is finished. If you need to recalculater everything from the start, use
--integrity-recalculate-reset option.

There still can be some bugs (mainly for bitmap mode that is very rarely used;
better test it in internal hash mode).

Some more notes inline:

> I already have a large disk containing important data (backup data)
> and I would like to use `dm-integrity` (with meta-data on a separate
> device) to make sure that when reading from that disk I actually get
> the original, non-corrupted, data.  (It seems that just the SATA
> provided CRC isn't enough sometime with a problematic USB-to-SATA
> adapter, and I get successful reads with corrupted data.)
> 
> Thus, I've tried using `integritysetup format` with `--no-wipe`,
> `--data-device` and `--integrity-bitmap-mode`, and then
> `integritysetup open` with `--integrity-recalculate`.  However upon
> subsequent `integritysetup open` without the recalculate flag, the
> `dump` command still shows the recalculating flag.  Searching on the
> internet about this I've found only a few issues about this, but no
> actual solution or more insight.
> 
> As such my question:  is there a way to actually force `dm-integrity`
> to compute the hashes without wiping my original data?

I think you used it in this mode already.

> Here are the snippets I've tried (on blank files via loop devices):
> 
> 
> # create empty files
> truncate --size 4GiB -- /tmp/dm-integrity-tests-data
> truncate --size 512MiB -- /tmp/dm-integrity-tests-meta
> 
> # fill the data with something to make sure dm-integrity doesn't touch
> the original data
> dd if=/dev/urandom of=/tmp/dm-integrity-tests-dat
> a bs=1M count=128
> iflag=fullblock status=progress

here you decreased the data size (you perhaps want add conv=notrunc)

> 
> # compute a hash of the original data
> md5sum -b -- /tmp/dm-integrity-tests-data /tmp/dm-integrity-tests-meta
> 
> 
> # the data is loop0, and the meta-data is loop1
> losetup --show --sector-size 4096 --find /tmp/dm-integrity-tests-data
> losetup --show --sector-size 4096 --find /tmp/dm-integrity-tests-meta

Do not setup loop devices, integritysetup will do it for you automatically
(and removes them on deactivation). Just use images file paths.

m.