n00b questions about using dar for archiving wikis
griffin tucker <darsuporpt340-DSTKDfZ7/[email protected]>
| Newsgroups | gmane.comp.sysutils.backup.dar.support |
|---|---|
| Message-ID | <CAKJX5A60x4V4DfdL+wwu8Ogpc=VwW9voRmey=um1K0Nes=UKmg@mail.gmail.com> |
i'll try to be as comprehensive as possible so that i don't miss anything. i want to make decremental backups of 7tb worth of wikis (text only) provided by wikimedia, each wiki dump in a separate dar archive wikidumps are updated each month. and from what i've read in the dar tutorial, if i understood it correctly, i'll have to write a script to do decremental backups, which shouldn't be a problem. the 7tb is compressed, and uncompressed, being text, is much, much larger, and using btrfs transparent compression leaves a lot to be desired when it comes to stability (i'm starting again after losing data after a very brief power-loss). i've tried using other backup tools such as rdiff-backup, however it seems i would have to have all the files available uncompressed, as opposed to adding each file one-by-one - does dar support adding one file to the archive at a time, or did i misunderstand the tutorial? some of the xml files are split into groups of pages, such as the english wiki, because they would otherwise be very large files. sometimes a copy of the recombined file is provided along with the split pages files, but for the large english wiki meta-history file, for example, a recombined copy is not provided, so writing a script to recombine it shouldn't be too difficult (using cat and grep, etc.). i want the decremental backup each month to only store the changes to the large files, saving a large amount of space. does dar support only backing up the difference of the file, as compared to simply storing a complete copy of the same file when they're different? aside from multiple backups, is there any data redundancy built into dar, yet? (i'm worried about bitrot)