Re: Backup strategy

On Wed, 2005-01-19 at 17:08 +0100, Flavio Curti wrote:
> Most servers, I'd like to keep 2 weeks of information with the lowest
> possible disk-usage. So I'm thinking of doing a weekly every two
> weeks, and a daily in between. Will this work? Will the weekly backup
> include everything, as there is no monthly?

So it sounds like your requirement is to be able to restore any file to
what it was on any of the preceding 14 days.  You have control over two
things: how often "monthly" and "weekly" dumps are done.  The goal is to
come up with the timing that minimizes disk usage.

The ideal scheme for you will depend on how many files are getting
modified each day (if all files are touched daily, then you might as
well always do full dumps!), so you'll have to experiment a bit, but
there are some general guidelines.

Note that hdup uses the term 'monthly' synonymously with level-0 or full
dump of the filesystem.  'weekly' means level-1 dump, including all
files modified since the most recent monthly.  'daily' means level-2
dump, including all files modified since the most recent weekly.  There
is no requirement that monthly dumps be done once a month, or that
weeklies be done once a week.  Dailies can not be more often than once a
day, but could be less.

Basically you need enough space on your backup disk to hold two full
dumps, plus a bit more for the incrementals.  You can't make do with
just one full dump since you need the history (think of the day after
you've done a full dump -- you still need the previous one so that older
files are covered).  Luckily the backup files compress well, so the
backup disk need not be bigger than the primary disk, even when holding
two full dumps plus extras.

If you do full dumps more often than once every 14 days, you might need
3 full dumps simultaneously, so let's eliminate that possibility and do
them 15 or more days apart.  On the other hand doing full dumps, say,
once a month, when you only care about 14 days of history, is wasteful
of disk space, since the incremental dumps against a 30-day-old snapshot
are going to be larger than those against a 14-day-old one.  So I think
we can say that doing full backups every 15 days is optimum in your
scenario.  The older full backup should get deleted the day before
you're due to make a new one.

Within the 14-day window, you'll need to keep all the weekly and daily
backups for those days.  How often the weeklies should be done depends
on your pattern of file modification, but you might try something like 3
or 4 days between weeklies, so you're kind of splitting the difference
(3-4 weeklies between each monthly; 3-4 dailies between each weekly).
It is probably not that critical: the bulk of your disk space is going
to be taken up by the full dumps, since on the typical system the great
majority of files don't get modified and only show up in monthlies.

The 'backup.pl' and 'cleanup.pl' scripts in the contrib dir can automate
the scheduling of backups, and deletion of older backups, for you, given
the parameters above.

The other thing that will help disk space usage a lot is making sure you
exclude everything that doesn't need backing up -- temp dirs, cache
dirs, etc.  Also, use bz2 compression.

Bng

-- 
Boris Goldowsky <boris-FrUbXkNCsVf2fBVCVOL8/[email protected]>