RSync like diff for incremental stuff
Cyril Russo <[email protected]> Sat, 13 Jun 2009 12:38:14 +0200
| Newsgroups | gmane.comp.sysutils.backup.dar.general |
|---|---|
| Message-ID | <[email protected]> |
Hi, I'm wondering if DAR could use rsync like system for incremental backup. Currently, if I want to backup my mail file (3GB) that changes hourly, it back it up everytime it's run (and the whole 3GB). With a rsync like algorithm, it would have saved only the difference (which is not that much, something like 10MB). Usually rsync requires 2 copies of the file (the "previous file" and the "new file"), in order to find out the difference. This is an annoying requirement for an archive format, as the "previous file" is usually compressed / encrypted, and as such, would require too much CPU to uncompress / decrypt for comparing. However, librsync has a mechanism to compute a "signature" of a file, and this signature is used in place of the "previous file". The signature itself is very small (compared to the file), so it might be possible to store such signature in the catalog. The signature is compared to the "new file", and the diff can be made from this (so in my previous example, only 10MB of data would be stored in the backup). Upon restoring however, the whole chain of backup must be read (as the final file is made of original_file + diff(s)). As restoring happen very rarely, I don't think it's a problem, as the space gained by using diff worth the extra CPU time. That way, the DAR format would really, really fit all the possible requirement for a backup tool, as it would be optimal in size (rsync like algorithm), optimal in speed (binary code), optimal in security (encryption). What do you think about this ? Cyril ------------------------------------------------------------------------------ Crystal Reports - New Free Runtime and 30 Day Trial Check out the new simplified licensing option that enables unlimited royalty-free distribution of the report engine for externally facing server and web deployment. http://p.sf.net/sfu/businessobjects