Re: MAYBE BUG in dar 2.7.2 with hardlinks handling

On 03.12.2021 23:25, Denis Corbin wrote:
> 2/ Now, you have guessed, when the first subdirectory returned by the
> operating system is the new one ...

> The order of entries in a directory is no linked to the lexicographical
> order of their name. It is completely filesystem dependent.

Gosh, silly me! These were "the missing pieces" I was failing to think 
about when been trying to get a grasp on what's going on here. I was 
even thinking about creating "-O0" build of dar tomorrow evening to give 
it a gdb treatment accompanied by a pint of beer :-). You literally 
saved me from doing that so Saturday's evening would be more of a 
traditional "reading Phoronix" style.

> Improving this behavior is quite tricky...

TBH now when I understand what's going on I don't think that this 
behaviour is really "in needs" to be improved. There's another possible 
approach here though. If it would be possible to pass the explicit list 
of files to be backed up (with names relative to the specified -R) to 
dar then it would be somewhat easier to have it optimized like this:

1. Read the file list from the reference archive and match it with the 
passed list of files "to be archived". Include all matched files into 
archive in a usual manner, i.e. save or delta-diff-save files that had 
their content changed, do not "save" data for all unchanged. For files 
that are in the archive of reference but are not in the list to archive 
- either mark them as "deleted" or don't do anything about them at all 
depending on some control flag passed by the user to control this.

2. Do a second pass over the list of files to backup but this time only 
taking into account those which were not found in the archive of 
reference. At this time we would have all the required information on 
hands to be able to find and properly (i.e _not_store_) handle all 
hardlinks.

I'm not that familiar with the implementation details of the dar so 
can't judge if:
a) the approach above aligns well with the archive file format (i.e. if 
it is mandatory or not to have all files stored in the specific 
subdirectory to be places inside the archive right after the data 
structure/entry for that directory);
b) whether it is easy to bypass filesystem traversing code and replace 
it with a feeder coming from the user-specified list;

But in general such approach feels like something that might be possible 
and not _that_hard_ to implement.

> But note that this behavior is not new, I've tested 2.7.2 and 2.6.15:
> they behave the same.
Yep, can confirm it too. I had started my experiments with "new 
approach" using 2.6.8 and hitting "unexpected behaviour" was yet another 
reason for me to build fresh libthreadar and dar RPMs for CentOS 7.


> This is my turn to be sorry for this long explanation! :)

No need, it was a really great explanation done in a form that made it 
clear to me what is happening and why. What a great teaching skills 
you've got, congrats!
-- 
Regards, Alexey Loukianov