Re: Update on BackupPC Development / Is backuppc dead?

Hi Les,

On 2012‒08‒06 13:05:53-0500, Les Mikesell wrote:
> On Mon, Aug 6, 2012 at 9:46 AM, Wessel Dankers
> <[email protected]> wrote:
> >
> > The ideas overlap to a limited extent with the ideas[0] that Craig posted
> > to this list. For instance, no more hardlinks, and garbage collection is
> > done using flat-file databases. Some things are quite different. I'll try
> > to explain my ideas here.
> 
> Personally I think the hardlink scheme works pretty well up to about
> the scale that I'd want on a single machine and you get a badly needed
> atomic operation with links more or less for free.

Adding a chunk can be done atomically using a simple rename(). Removing a
chunk can be done atomically using unlink(). The only danger lies in
removing a chunk that is still being used (because there's a backup still
in progress whose chunks aren't being counted yet by the gc procedure). The
simplest way to prevent that is to grant exclusive access to the gc
process. Note that hardlinks do not prevent this race either. It's a
problem we need to solve anyway.

Craig lists a couple of reasons to abandon the hardlink scheme in
http://sourceforge.net/mailarchive/message.php?msg_id=27140176

> If you are going to do things differently, wouldn't it make sense to use
> one of the naturally distributed scalable databases (bigcouch, riak,
> etc.) for storage from the start since anything you do is going to
> involve re-inventing the atomic operation of updating a link or replacing
> it and the big win would be making this permit concurrent writes from
> multiple servers?

Using something like ceph/rados for storing the chunks could be interesting
at some point but I'd like to keep things simple for now.

cheers,

-- 
Wessel Dankers <[email protected]>

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

_______________________________________________
BackupPC-devel mailing list
[email protected]
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-devel
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iQIcBAEBCgAGBQJQINN6AAoJEHUPgVbpEVyLo3IP/R3T6ZufsstrCVwKvSVuVMRG
LXvLUY+7uSThhVBZkeJ+WieMhi9kSf50WdwPojOXUk92W32yMJ7KBEZxqNSv8K/K
JxRBHsgOa6QsAUCmHrmRNT4NL6gVHl7v5EaLdwPW7R78/TcGUnY76HH1OHCOxNrK
Zc0SUoNw8855z6JRFowtWZagE+odhT4llXNwx4Vi6tJ7dYSy5HkQDbGy5ZPXk4Of
J4XomGy5pey5tNxYqiKwRc4KTF8xXGVVx/9iNjx3AmvQuHTgGgMLPj8iOw4C/9ne
5thi63mSvTRtpeADWWjStPhuhum0IDb7HY6cst1/ftI/jBnL8RZyNCuJB50ZbROb
66IMtAonpaOVyp6fLjg/YBzvanEmmp9xJdntWVZVYtuaPYva1YI/0Dy1gUZKY4BC
LETfUFz0A2vEk6JL09rW6D9Bsx0kLH9CI20fiIu6ylP4K514AlcM198fMps4e0Ll
jQw6eX0AWpWhX5G0j+UMNDF+Lr6hLrtNJ+JH9cTHEN6QPAEJgfqZJGuJntggwZwF
5weG0YriNNVS8A8PBlasWt0eXAZMszcm/CkS1KTPp/VGOYZ1wOkWXcYHUVs0htpt
wqi6hhrpThsBvFNun3Nzrk58zZp5A+9U/yb6UKhPNHXVp5Z8sYZ6VnaUEb4jLmik
fU+YAOTWaHTUPkBpyoDn
=cdmm
-----END PGP SIGNATURE-----