Re: Archive Management

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Joseph,

Joseph Watson wrote:
> Hello,
> 
> I have an idea for managing backup archives in a very space efficient and 
> power-full way.   I have looked through the features of dar and it looks to 
> me like the development version should support what I envision.  I will try 
> to explain with an example.
> 
> The archives would be stored and managed on a dedicated server with redundant 
> storage to protect against loss.
> 
> Day 1: Full backup
> Day 2: Incremental referencing day 1
> Day 3: Incremental referencing day 2
> Day N: Incremental referencing day n-1
> 
> For each one of these archives I will define how long I want to keep it (by 
> some means of my own).  I may keep the weekly backup longer then the daily 
> for instance.
> 
> When a backup has expired and I no longer need it, I want to merge it with the 
> upstream incremental.  This way I can reclaim the space taken by files in the 
> expired archive that are no longer needed by upstream incrementals. 
> 
> shouldn't I be able to do this with the -+ and -/ options in the development 
> version?  Ie if I want to merge day2 into day3
> 
> dar -+ merge_day2_day3 -A day3 -@ day2 -/ A
>

Well, in fact no:
Suppose you have two files TOTO and TITI. while TOTO has not changed
since Day1, TITI changed at the time of day2. Thus day2 contains TITI's
full data while it only contain TOTO's inode reference (ownership,
permission, file's size, etc.). We will say TOTO is "partially saved" to
 define this state, by opposition to "fully saved" that correspond to
the state of TITI here.

Now at the time of Day3 TITI has not changed since day2 but TOTO does.
Thus day3 archive will contain TOTO's full data while it will only
contain TOTO's inode references.

Now, using the 'A' policy (Always overwrite), day3's data will be
overwritten by day2's ones, whatever is the file's status fully or
partially saved. Thus here you will replace TITI's partially saved data
found in day3 by TITI's full saved data found in day2, which is good,
but you will also replace TOTO's full data found in day3 by TOTO's
partially saved found in day2 which is probably not what you expect.

The solution is to have an new atomic policy that makes the selection
based on the the status (full or partially saved)  of the files,
eventually been completed by most recent version information.

With this policy, you should be able to address you need.

> 
> In this way you could have what I would call a rolling backup that is space 
> efficient and very flexible.  It also eliminates the need to do a full backup 
> on a periodic basis.  Every day you only need to do a incremental from the 
> previous day.  This will also help when you have a large about of data that 
> you need to keep a backup of off site.  The smaller backup sizes can be 
> transferred over a low bandwidth connection to a server that would store and 
> manage all the backup archives.  I would also keep a copy on a local server 
> for faster restores.
> 
> What are your thoughts on this?
> 

Yep, this is an interesting use of dar, I must document this also
because due to the many dar's options I guess only few people will
understand that it is possible to do this way. :-)


Regards,
Denis.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHR/FspC5CI8gYGlIRAj3VAJwOgft8eeL6oB9/Wf03bwHqh7UWLQCeNfnc
pcGa8SCLK6lvBye/tC/meIc=
=m0q8
-----END PGP SIGNATURE-----

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2005.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/