n00b questions about using dar for archiving wikis

griffin tucker <darsuporpt340-DSTKDfZ7/[email protected]>
Newsgroups gmane.comp.sysutils.backup.dar.support
Message-ID <CAKJX5A60x4V4DfdL+wwu8Ogpc=VwW9voRmey=um1K0Nes=UKmg@mail.gmail.com>
i'll try to be as comprehensive as possible so that i don't miss anything.

i want to make decremental backups of 7tb worth of wikis (text only)
provided by wikimedia, each wiki dump in a separate dar archive

wikidumps are updated each month. and from what i've read in the dar
tutorial, if i understood it correctly, i'll have to write a script to do
decremental backups, which shouldn't be a problem.

the 7tb is compressed, and uncompressed, being text, is much, much larger,
and using btrfs transparent compression leaves a lot to be desired when it
comes to stability (i'm starting again after losing data after a very brief
power-loss).

i've tried using other backup tools such as rdiff-backup, however it seems
i would have to have all the files available uncompressed, as opposed to
adding each file one-by-one - does dar support adding one file to the
archive at a time, or did i misunderstand the tutorial?

some of the xml files are split into groups of pages, such as the english
wiki, because they would otherwise be very large files. sometimes a copy of
the recombined file is provided along with the split pages files, but for
the large english wiki meta-history file, for example, a recombined copy is
not provided, so writing a script to recombine it shouldn't be too
difficult (using cat and grep, etc.).

i want the decremental backup each month to only store the changes to the
large files, saving a large amount of space. does dar support only backing
up the difference of the file, as compared to simply storing a complete
copy of the same file when they're different?

aside from multiple backups, is there any data redundancy built into dar,
yet? (i'm worried about bitrot)
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.