Re: Going deeper on merge feature

Denis Corbin <[email protected]> Sat, 16 Dec 2023 18:49:43 +0100
Newsgroups	gmane.comp.sysutils.backup.dar.support
Message-ID	<[email protected]>
On 16/12/2023 15:51, vash-dar.ztzvp--- via Dar-support wrote:
> Hello,

Hello,

> I've been using DAR for many years and helped me very much. Thank
> you.
> 
> I’m not sure to have correctly understood the merge feature or it may
> not work as expected. I’m going to tell about what’s happening.
> 
> This is my case scenario. I'm dealing with two old archives:
> 
> -2020_full-backup -2021_diff-backup
> 
> My goal: save storage space through merging those two old archives.
> 
> The merged archives should contains all the files in their most
> updated status, except those I had deleted during the time lapse
> between those two backups. I don't need 2020 superseded files.
> 
> This is the command line I launched:
> 
> dar -+ merged -A 2020_full -@ 2021_diff
> 
> The merged archive contains 2020_full backup files and 2021 new
> files. Changed files are stuck in 2020 version. This is the opposite
> of what documentation tells. Indeed, the default overwrite policy
> reported by docs is -/ Oo, but it seems dar doesn’t act that way
> considering it’s keeping the oldest version of changed files.
> 
> So i tried to give the following:
> 
> dar -+ merged -A 2020_full -@ 2021_diff -/Oo
> 
> The merged archive is equal to 2021_diff backup archive, that’s not
> what I expected. dar -l output correctly tells that some files were
> previously deleted and all listed files are at most updated versions,
> but not deleted files are only mentioned and not in “Saved” status.
> That means I cannot restore them from the merged archive, I still
> need 2020_full.
> 
> Is there something going wrong? Didn't I correctly understood the
> merge feature? Thank you.


first I understand that 2021_diff-backup is a differential backup
(according the name and the fact it contains 'deleted'
files, as you mentioned).

My question is: what was the backup of reference for 2021_diff? Was it
2020_full-backup or another backup?

Let's assuming the reference backup was 2020_full-backup and consider 
this command-line:

	dar -+ merged -A 2020_full -@ 2021_diff -/Oo

This will effectively take all entries of the 2020_full backup, and for 
those that are present in both full and differential backup will take 
the entry of the differential backup (because of the "-/ Oo' policy)

what you are not taking into account here, is:
1/ that an entry that has not changed since the backup of reference is 
stored as en empty shell in the diff backup: there is only the metadata. 
This get the flag "unsaved".

2/ When a file has been removed between the full and the diff backup, 
the diff contains a special entry telling that the file has been 
removed. (to be able to remove the file from filesystem when restoring 
the differential backup).

For those two previous case, the overwriting policy keep in the
resulting archive the "Unsaved" version of files and the "deleted"
entries. Both cannot be used to restore anything from them.

To solve this, your overwriting policy must be more precise/complex, 
something like:
  - if the entry in both full and diff is saved and the one from diff,
keep the data from diff
  - else if the entry in full is saved while "unsaved" in diff, keep the
one from the full backup
  - else if the entry in full is saved and is "deleted" in diff, just
ignore this entry (or keep the "deleted" entry from diff)
  - and so on...

Now, if you plan is to make a full backup from the diff backup as if a
full backup was done at the time of the diff backup, this is as easy as
adding "full-from-diff" in your merge pattern.

I let you first examin the /etc/darrc provided with dar, where the
"full-from-diff" definition contains the adhoc overwriting policy and a
quick documentation on how to use it.


> 
> -- Vash
> 
> 


Cheers,
Denis
OpenPGP_signature.asc (application/pgp-signature, 840 B)