Re: NFR: store backuppc attributes in a database

Hi,

On Tue, Jun 2, 2009 at 17:03, Peter Walter <[email protected]> wrote:
> This is just a summary of the proposals. Les Mikesell has been
> eloquent in his defense of the way backuppc operates now.

+1 on keeping things as they are.

I am currently migrating my backups from Bacula to BackupPC, my main
motivation for doing it is the fact that BackupPC does not use a
database backend for metadata.

> To clarify, what we are talking about here is not storing the backed-up
> files themselves in the database, but only the metadata, such as the
> hardlink information and other attributes, in the database. In addition,
> I see this functionality as an extension and addition to the current approach.

Storing metadata on a database is a problem when you have millions of
small files. This is my current situation. On Bacula, even to restore
one file, I have to wait for it to scan all the backup images and load
them from the database, only for it to be able to show me a directory
tree... It's silly, considering that you could "just" use a directory
tree to show the user a directory tree.

I used to use Symantec NetBackup in another company, and it had the
same problem, it took ages to browse backup images containing millions
of small files. Not to mention that more than one time I had problems
with corruption of the metadata database and it was a pain to fix
them...

And, in a disaster scenario, where your BackupPC is no longer working
(doubtful that it will happen if you don't have a database), if you
have all the files in a filesystem, you can always use BackupPC_zcat
and script it so that you can recover the contents of files.

> I would ask anybody interested to review the discussion thread
> for "Backing up a BackupPC server" on the backuppc-users
> list that was started today.

I didn't receive any e-mails on that thread... I will review my
subscription options to see if something is wrong with them.

In any case, I am currently using BackupPC to back files up directly
to USB attached external drives that I send off-site. Every week I
rotate the disks.

I just bought some big disks to a spare server I have, I plan to
create a local RAID5 volume to use as the BackupPC main repository and
leave BackupPC working there on the local disks. To do my off-sites, I
plan to use rsync -aH to duplicate that data into the external USB
disks. If I need to do disaster recovery, I can just install BackupPC
on a machine, then point the repository to the USB disks and restore
directly from there. Simple & easy, failproof.

If you really want to go on with this idea of changing things to add a
database, I suggest a fork of the project that would explore this idea
while the original product evolves in its original path. That way, if
you can really prove that you can come up with a database design that
will not kill the performance of restores, and in a way that I don't
need to worry about database backups, and if I lose the database I can
still restore everything, I might consider switching. Good luck with
that, though...

Cheers,
Filipe

------------------------------------------------------------------------------
OpenSolaris 2009.06 is a cutting edge operating system for enterprises 
looking to deploy the next generation of Solaris that includes the latest 
innovations from Sun and the OpenSource community. Download a copy and 
enjoy capabilities such as Networking, Storage and Virtualization. 
Go to: http://p.sf.net/sfu/opensolaris-get
_______________________________________________
BackupPC-devel mailing list
[email protected]
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-devel
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/