Re: Suggestion: delta-exclusion = compress-exclusion

On 12/05/2021 21:29, imschmeg wrote:
> On Wed, 12 May 2021 20:33:53 +0200
> Denis Corbin <[email protected]> wrote:
> 
>> On 11/05/2021 23:14, imschmeg wrote:
[...]
>>>
>>> It might also be useful to exclude read-only files from deltas, but
>>> I don't see how to do that easily.
>>>   
>>
>> I don't understand the point: Dar does not modify files under backup,
>> thus read-only files can benefit from binary delta as well as
>> read-write files.
> 
> In my experience, a file with read-only permission will either never be
> altered, so that the initial calculation of delta sigs is wasted.

OK, it makes sense,

>  Or
> it will only very rarely be altered, so that the amortized savings from
> delta compression will be small across many backups.  Of course, there
> are many other reasons to give a file read-only permission, including
> merely making it hard to delete or accidentally modify.
> 
> However, in most cases, the largest of these files have extensions that
> would fall under the delta-exclusion masks (for instance, .iso files).

well, if .iso may be compressible, it depends on what is stored in it.
Most of the time they contain compressed data (compressed packages) but
making an iso file of text documentation should be strongly compressible
(OK, that's to be tested).

> 
> In my particular case, I follow a full backup with a sequence of
> incrementals, all with extracted catalogs. This means that the delta
> signatures get duplicated many times.  Meaning that the cost of delta
> signatures becomes significant for very large files that either never
> change or change only very rarely.
> 
>>
>> But anyway, if you want to do some fancy things while dar does not
>> provide the exact way to exclude/include what you want, there is a
>> simple solution:
>>
>> use the find command to select files and feed its output to dar by
>> mean of --include-from-file or --exclude-from-file
>>
>> In your case:
>>
>> find / -perm -002 -o -perm -020 -o -perm -0200 \
>> 	-o -print > listing.txt
>> dar --exclude-from-file listing.txt ... other options...
> 
> Thanks for that idea!  I like this better than the EA-tagging idea we
> discussed with sockets.  There is a TOCTOU race possibility with using
> find prior to dar, but not very likely in my case because I'm doing the
> backups at logout.

You mentioned that read-only files where either never or very rarely
altered... running find before dar should not miss much of those files
in the gap, right? :)

Anyway, I add the possibility to exclude file from binary delta when it
is read-only to the list of feature requests. I understand read-only
files are those that have no write bit for any of user, group and others:
If a file has an ACL that allows a particular user or group to write to
that file, this will be effective only if the group has write permission
too, thus only testing permission metadata should be sufficient to
validate the read-only nature of a file.

> 
> 
> 
>