Re: Restoring everything with dar_manager

On 18/06/2023 06:35, John Goerzen wrote:
> Hi Denis,
> 

[...]

.
> 
> I guess the question I have left is: is there presently a way for a user
> to do consistent restore from a backup set consisting of both
> incremental and differentials, without having the manually track the
> order to do so outside of dar?

No, you need to restore your backup in the order they have been made on 
the branch of backup you want to get to. Dar cannot guess for you which 
backup state you want to restore to:

for example assuming you have this tree of backup:

+ full
|
+----- + diff1
|      + incr1_A
|      + incr1_B
|      + incr1_C
|
+------+ diff2
        + incr2_A
        + incr2_B

dar cannot know which branch you want to restore (diff1 or diff2 ?) and 
at which status (up to diff1 ? incr1_A ? incr1_B ? incr1_C?)

Worse, it may be possible that the backup to be made in that order:
full
diff1
incr1_A
diff2
incr2_A
incr2_B
incr1_B
incr1_C

obviously restoring incr2_B after incr1_C will lead you to restore older 
data...

That's not the role of dar to guess that.

> 
>>> Further let's imagine that before running backup #3, you created 10GB of
>>> data, which was deleted before running backup #4.  Backup #4 isn't going
>>> to record that deletion, since that data wasn't present in #1.  What
>>> will the result be?
>> for each particular entry, dar_manager has a list of status per backup stored in
>> the database. Adding a new backup which does not contain a given entry leads to
>> the same result in the database as adding a differential backup which records
>> this file as removed. As such, these 10G of data would not be restored.
> 
> Right, but let's say we can't use dar_manager - perhaps because we want
> to restore it all.  Here's a test I did:
> 
> jgoerzen@hephaestus:/tmp/dartest$ mkdir data
> jgoerzen@hephaestus:/tmp/dartest$ echo one > data/file1
> jgoerzen@hephaestus:/tmp/dartest$ dar -c bak1 -R data
> jgoerzen@hephaestus:/tmp/dartest$ dar -c bak2 --ref bak1 -R data
> jgoerzen@hephaestus:/tmp/dartest$ rm data/file2

where file2 comes from? When has it been created?

> jgoerzen@hephaestus:/tmp/dartest$ dar -c bak3 --ref bak1 -R data
> 
> jgoerzen@hephaestus:/tmp/dartest$ dar -x bak1 -R restore
> jgoerzen@hephaestus:/tmp/dartest$ dar -x bak2 -R restore
> jgoerzen@hephaestus:/tmp/dartest$ dar -x bak3 -R restore

after bak1, you should restore either bak2 or bak3, it does not make 
sens to restore bak3 after bak2 because they are on different branches 
(= they don't depend on one another, or if you prefer, the first is not 
the reference of the second)

This is advantage of differential backups compared to incremental ones:
you don't need to restore intermediate backup (here bak2) between a 
differential and the full backup it has been based on... But if you 
absolutely want to restore all three of them, then rather use 
incremental backups, making bak2 being the reference backup of bak3.

> jgoerzen@hephaestus:/tmp/dartest$ ls -l data restore
> data:
> total 1
> -rw-r--r-- 1 jgoerzen jgoerzen 4 Jun 17 23:36 file1
> 
> restore:
> total 1
> -rw-r--r-- 1 jgoerzen jgoerzen 4 Jun 17 23:36 file1
> -rw-r--r-- 1 jgoerzen jgoerzen 4 Jun 17 23:37 file2
> 
> As I thought.  I wonder if there's a way for dar to clean up files on
> the filesystem that weren't present at backup time?  I'm not immediately
> seeing one.

Anyway, if you don't want to bother with incremental / differential you 
can go one step above dar: there are, for example, SaraB and Baras 
projects that can do this job for you:

https://sarab.sourceforge.net/


> 
> Thanks!
> 
> - John
> 

Cheers,
Denis