Dar iterating between "random" and last slice in diff mode

Hi


Backup setup: ~11 TiB of data, 99.5 GiB per slice, 72 slices per disk, no
encryption, no compression, split onto two 8 TB disks

dar 2.6.13, static + infinitint version, based on Debian Unstable (sorry, I
forgot to update dar before starting the backup, so I'm now using an old version
from three years ago :/)


Creating (-c) and testing (-t) the archive worked fine and as expected. It wrote
72 slices to the first disk (B1), I then swapped the disk out and it wrote the
other data to the second disk (B2). When testing, it read the last slice first,
then I switched to B1, it tested up to slice 72, then I switched to B2 and it
tested all the rest.


Currently, I'm diffing (-d) the archive and strange things happen. It first
started by reading the last slice on B2 (fine), then I swapped to B1 and it
processed the data up until it needed slice 73, at which point I swapped to B2
and diffing continued. It looks like it tested *almost* all data, but now I'm in
a disk swapping loop:

[...]
OK
/mnt/rootfs/root/.debug/lib64/modules/5.5.0-OSS4.2/extra/mISDN_core.ko.xz/a3c824ac20ad8df2a801a945675fbbc612cfb1f2/probes
valery_root_full_2024_01_21.67.dar is required for further operation, please
provide the file. [return = YES | Esc = NO]
Continuing...
OK
/mnt/rootfs/root/.debug/lib64/modules/5.5.0-OSS4.2/extra/mISDN_core.ko.xz/a3c824ac20ad8df2a801a945675fbbc612cfb1f2/elf
valery_root_full_2024_01_21.117.dar is required for further operation, please
provide the file. [return = YES | Esc = NO]
Continuing...
[...]
OK
/mnt/rootfs/root/.debug/lib64/modules/5.5.0-OSS4.2/extra/mISDN_dsp_oslec.ko.xz/c84fcb4e2d1dfbe3c49ac0beeeb2007127196210/probes
valery_root_full_2024_01_21.67.dar is required for further operation, please
provide the file. [return = YES | Esc = NO]


Slice 117 is the last slice on B2, but 67 is a pretty random slice on B1. So
far, I needed to swap between B1 and B2 three times and I fear that it will
continue to iterate between the two slices for quite some time. Unfortunately, I
can only attach one backup disk at a time and swapping disks is complicated (I
have to physically unscrew the enclosure, unscrew the disk, swap disks, screw
the disk back into place and rescrew the whole enclosure).

Unfortunately, the last slice is 65 GiB in size and I can't easily copy it to B1
(naturally won't fit into the 4 GiB rescue system RAM disk), so I have to keep
swapping disks around like crazy.

According to iotop, dar already read 21.44 TB, which means that it successfully
processed *most* of the archive.


Luckily, I keep a log of all of dar's output, so I checked the creation process
logs and it looks like the files it's currently processing have been added at
the end of the creation process. Unfortunately, it also looks like I understand
what's going on:

[...]
Recording hard link into the archive:
/mnt/rootfs/root/.debug/lib64/modules/5.5.0-OSS4.2/extra/mISDN_core.ko.xz/a3c824ac20ad8df2a801a945675fbbc612cfb1f2/elf
[...]
Recording hard link into the archive:
/mnt/rootfs/root/.debug/lib64/modules/5.5.0-OSS4.2/extra/mISDN_dsp_oslec.ko.xz/c84fcb4e2d1dfbe3c49ac0beeeb2007127196210/elf


The affected files are hard links to other places! This means that dar's
behavior is actually intended and not a bug - good.

The man page says:

"Doing a difference in sequential read mode is possible but hard linked inodes
can only be compared to the filesystem the first time they are met, next hard
links to this same inode cannot obtain the corresponding data because skipping
backward in sequential read mode is forbidden. In that situation, the hard links
are reported as skipped, meaning that data comparison could not be performed."


This makes sense, in general.

I could work around my issue by executing the comparison (-d) mode in
--sequential-read mode, but that would be less efficient and is unnecessary,
because dar is free to actually switch between slices as long as it's only using
data of one -p segment (i.e., in my case slices {1..72} and {73..117}).

However, wouldn't it be useful to also provide the hard link skipping feature as
an option even in non-sequential read mode? The code is there already (more or
less, not sure about point 3), so all that is needed is another option. Here's
my reasoning:

  1.) The actual data has already been tested once (the first time the file has
      been tested).
  2.) Subsequent hard links only point to already tested data, so they can't be
      corrupt if the originally tested data tested successfully (i.e., if it
      wasn't corrupt).
  3.) The only thing that dar needs to make sure is that additional hard links
      point to the correct inode, and that's easily tested by comparing the
      current hard link file's inode against the inode it should point to. Dar
      only needs to know where the link points to (i.e., file path) on the file
      system you test against and that information should already be part of the
      archive at the very point where the file was saved to, so no previous data
      is necessary.


Or, in pseudo code:

file_a: regular;
file_b: hardlink (file_a);

Example for file_b:

if (file_b.is_hardlink ()) {
  /*
   * Only check if the metadata is correct, file content must be the same as
   * for file_a.
   */
  if (inode (file_b) == inode (file_b.hardlink_pathto ())) {
    /*
     * We know that file_b points to file_a in this example, but that's
     * not generally true of course.
     */
    assert (file_a.path () == file_b.hardlink_pathto ());

    return (success);
  }
  else {
    /*
     * inodes of file_b and the file it points to are different - metadata
     * corruption!
     */
    return (error);
  }
}
else {
  return (check_data (file_b));
}


Does that make sense to you?



Mihai