BackupPC 4.0.0alpha0 released
Craig Barratt <[email protected]> Sun, 23 Jun 2013 18:10:37 -0700
| Newsgroups | gmane.comp.sysutils.backup.backuppc.announce |
|---|---|
| Message-ID | <CADSzEFi_N3jtC9XF5_507AeZnvbkYyxQwsvgBHGxLX8_71q=vg__35625.1823519871$1372036412$gmane$org@mail.gmail.com> |
--===============4170079396409528642== Content-Type: multipart/alternative; boundary=047d7b6d9868c3f13204dfdc1561 --047d7b6d9868c3f13204dfdc1561 Content-Type: text/plain; charset=ISO-8859-1 BackupPC community, I'm pleased to announce that BackupPC 4.0.0alpha0 has been released on SourceForge at: https://sourceforge.net/projects/backuppc/files/backuppc-beta/4.0.0alpha0/ 4.0.0 is a significant upgrade and rewrite. It should be backward compatible with existing installations. The release comprises three packages: - BackupPC-4.0.0alpha0.tar.gz: the usual BackupPC release tar ball. - BackupPC-XS-0.01.tar.gz: a perl XS module with C code that replaces several BackupPC perl libraries for improved performance. - rsync-bpc-3.0.9.0.tar.gz: a modified rsync that runs on the server that has a shim layer that interfaces directly to the BackupPC file system. I'll also push BackupPC-XS-0.01.tar.gz to CPAN, and at some point I'll push the source to GitHub (maybe next weekend; I'm starting a new job tomorrow). Of course, the tar balls above contain all the source too. As you can guess from the release name, I do not recommend using this release in any production environment. Having some people try it out in a sandbox in their environment would be very helpful. One important area to test is upgrading an existing V3 installation, so it would be great to test not just a virgin V4 installation, but also to also first install V3, do some backups, then upgrade to V4. The documentation is only partially updated. I'm happy to answer questions - for example, there are some importact changes to the configuration settings. If you reply to this thread please only copy the backuppc-devel list. I've pasted a short summary of features and changes. More is in the documentation. Craig - No use of hardlinks (except temporarily to do atomic renames). Reference counting is handled at the application level. - Backups are stored as "reverse deltas" - the most recent backup is always filled and older backups are reconstitued by merging all the deltas starting with the nearest future filled backup and working backwards. This is the opposite of V3 where incrementals are stored as "forward deltas" to a prior backup (typically the last full backup or prior lower-level incremental backup, or the last full in the case of rsync). - Since the most recent backup is filled, viewing/restoring that backup (which is the most common backup used) doesn't require mergning any deltas from other backups. - The concepts of incr/full backups and unfilled/filled storage are decoupled. - Uses full-file MD5 digests, which are stored in the directory attrib files. Each backup directory only contains an attrib file. - The Pool layout still support chains to handle md5 collisions. While collisions can be constructed and are now well-known, they are highly unlikely in the wild. Pool files are never renamed or moved, unlike V3. - Any backup can be deleted (deltas are merged into next older backup if it is not filled). - The reverse deltas allow "infinte incrementals" - no need for a full backup if you are willing to trade speed for the risk that a file change will not be detected if the mtime or size don't change. - An rsync "full" backup now uses --checksum (instead of --ignore-times), which is much more efficient on the server side - the server just needs to check the full-file checksum computed by the client, together with the mtime, nlinks, size attributes, to see if the file has changed. If you want a more conservative approach, you can change it back to --ignore-times, which requires the server to send block checksums to the client. - The use of rsync --checksum allows BackupPC to guess a potential match anywhere in the pool, even on a first-time backup. In that case, the usual rsync block checksums are still exchanged to make sure the complete file is identical. - Uses rsync-3.0.9 on the server side (in place of File::RsyncP), with a C code layer to interface to the BackupPC storage. So the whole data path for rsync is now in compiled C code, which is generally faster than perl (other than compression, which is the same code in either case, and I/O). - Due to the use of rsync-3.X, acls and xattrs are supported, and other useful options (but not all) are supported. Rsync protocol 30 supports the efficient incremental file list, which significantly improves memory usage and startup time. It also supports MD5 full-file checksums, which match BackupPC's new digest. That allows a full-file digest to be checked as easily as an mtime on the server side. - Significant portions of the BackupPC code are now compiled C code in a new module called BackupPC::XS that is dynamically linked to perl. - FTP Xfer method isn't supported yet. --047d7b6d9868c3f13204dfdc1561 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr">BackupPC community,<div><br></div><div>I'm pleased to = announce that BackupPC 4.0.0alpha0 has been released on SourceForge at:</di= v><div><br></div><div><blockquote style=3D"margin:0px 0px 0px 40px;border:n= one;padding:0px"> <div style><a href=3D"https://sourceforge.net/projects/backuppc/files/backu= ppc-beta/4.0.0alpha0/">https://sourceforge.net/projects/backuppc/files/back= uppc-beta/4.0.0alpha0/</a><br></div></blockquote><div><br><div>4.0.0 is a s= ignificant upgrade and rewrite. =A0It should be backward compatible with ex= isting installations.</div> <div><br></div><div>The release comprises three packages:<br></div><div><br= ></div><div style>=A0-=A0BackupPC-4.0.0alpha0.tar.gz: the usual BackupPC re= lease tar ball.</div><div style>=A0-=A0BackupPC-XS-0.01.tar.gz: a perl XS m= odule with C code that replaces several BackupPC perl libraries for improve= d performance.</div> <div style>=A0-=A0rsync-bpc-3.0.9.0.tar.gz: a modified rsync that runs on t= he server that has a shim layer that interfaces directly to the BackupPC fi= le system.</div><div><br></div><div style>I'll also push BackupPC-XS-0.= 01.tar.gz to CPAN, and at some point I'll push the source to GitHub =A0= (maybe next weekend; I'm starting a new job tomorrow). =A0Of course, th= e tar balls above contain all the source too.</div> <div style><br></div><div style>As you can guess from the release name, I d= o not recommend using this release in any production environment. =A0Having= some people try it out in a sandbox in their environment would be very hel= pful. =A0One important area to test is upgrading an existing V3 installatio= n, so it would be great to test not just a virgin V4 installation, but also= to also first install V3, do some backups, then upgrade to V4.</div> <div><br></div><div>The documentation is only partially updated. =A0I'm= happy to answer questions - for example, there are some importact changes = to the configuration settings. If you reply to this thread please only copy= the backuppc-devel list.</div> <div><br></div><div style>I've pasted a short summary of features and c= hanges. =A0More is in the documentation.</div><div><br></div><div>Craig<br>= <div><br></div></div></div><div><div style=3D"font-family:arial,sans-serif;= font-size:13px"> =A0- No use of hardlinks (except temporarily to do atomic renames). =A0Refe= rence counting</div><div style=3D"font-family:arial,sans-serif;font-size:13= px">=A0 =A0is handled at the application level.</div><div style=3D"font-fam= ily:arial,sans-serif;font-size:13px"> <br></div><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0- B= ackups are stored as "reverse deltas" - the most recent=A0<span c= lass=3D"" style=3D"background-color:rgb(255,255,204)">backup</span>=A0is al= ways filled</div> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0and older= backups are reconstitued by merging all the deltas starting with the</div>= <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0nearest f= uture filled=A0<span class=3D"" style=3D"background-color:rgb(255,255,204)"= >backup</span>=A0and working backwards.</div> <div style=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div s= tyle=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0This is the opp= osite of V3 where incrementals are stored as "forward deltas"</di= v><div style=3D"font-family:arial,sans-serif;font-size:13px"> =A0 =A0to a prior=A0<span class=3D"" style=3D"background-color:rgb(255,255,= 204)">backup</span>=A0(typically the last full=A0<span class=3D"" style=3D"= background-color:rgb(255,255,204)">backup</span>=A0or prior lower-level</di= v><div style=3D"font-family:arial,sans-serif;font-size:13px"> =A0 =A0incremental=A0<span class=3D"" style=3D"background-color:rgb(255,255= ,204)">backup</span>, or the last full in the case of rsync).</div><div sty= le=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div style=3D"= font-family:arial,sans-serif;font-size:13px"> =A0- Since the most recent=A0<span class=3D"" style=3D"background-color:rgb= (255,255,204)">backup</span>=A0is filled, viewing/restoring that=A0<span cl= ass=3D"" style=3D"background-color:rgb(255,255,204)">backup</span>=A0(which= is</div><div style=3D"font-family:arial,sans-serif;font-size:13px"> =A0 =A0the most common=A0<span class=3D"" style=3D"background-color:rgb(255= ,255,204)">backup</span>=A0used) doesn't require mergning any deltas fr= om other backups.</div><div style=3D"font-family:arial,sans-serif;font-size= :13px"><br> </div><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0- The c= oncepts of incr/full backups and unfilled/filled storage are decoupled.</di= v><div style=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div= style=3D"font-family:arial,sans-serif;font-size:13px"> =A0- Uses full-file MD5 digests, which are stored in the directory attrib f= iles.</div><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 = =A0Each=A0<span class=3D"" style=3D"background-color:rgb(255,255,204)">back= up</span>=A0directory only contains an attrib file.</div> <div style=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div s= tyle=3D"font-family:arial,sans-serif;font-size:13px">=A0- The Pool layout s= till support chains to handle md5 collisions. =A0While collisions</div><div= style=3D"font-family:arial,sans-serif;font-size:13px"> =A0 =A0can be constructed and are now well-known, they are highly unlikely = in the wild.</div><div style=3D"font-family:arial,sans-serif;font-size:13px= ">=A0 =A0Pool files are never renamed or moved, unlike V3.</div><div style= =3D"font-family:arial,sans-serif;font-size:13px"> <br></div><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0- A= ny=A0<span class=3D"" style=3D"background-color:rgb(255,255,204)">backup</s= pan>=A0can be deleted (deltas are merged into next older=A0<span class=3D""= style=3D"background-color:rgb(255,255,204)">backup</span>=A0if it is</div> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0not fille= d).</div><div style=3D"font-family:arial,sans-serif;font-size:13px"><br></d= iv><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0- The reve= rse deltas allow "infinte incrementals" - no need for a full=A0<s= pan class=3D"" style=3D"background-color:rgb(255,255,204)">backup</span></d= iv> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0if you ar= e willing to trade speed for the risk that a file change will</div><div sty= le=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0not be detected i= f the mtime or size don't change.</div> <div style=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div s= tyle=3D"font-family:arial,sans-serif;font-size:13px">=A0- An rsync "fu= ll"=A0<span class=3D"" style=3D"background-color:rgb(255,255,204)">bac= kup</span>=A0now uses --checksum (instead of --ignore-times),</div> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0which is = much more efficient on the server side - the server just needs to</div><div= style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0check the ful= l-file checksum computed by the client, together with the mtime,</div> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0nlinks, s= ize attributes, to see if the file has changed. =A0If you want a more</div>= <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0conservat= ive approach, you can change it back to --ignore-times, which</div> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0requires = the server to send block checksums to the client.</div><div style=3D"font-f= amily:arial,sans-serif;font-size:13px"><br></div><div style=3D"font-family:= arial,sans-serif;font-size:13px"> =A0- The use of rsync --checksum allows=A0<span class=3D"" style=3D"backgro= und-color:rgb(255,255,204)">BackupPC</span>=A0to guess a potential match</d= iv><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0anywhe= re in the pool, even on a first-time=A0<span class=3D"" style=3D"background= -color:rgb(255,255,204)">backup</span>. =A0In that case, the usual</div> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0rsync blo= ck checksums are still exchanged to make sure the complete file</div><div s= tyle=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0is identical.</= div><div style=3D"font-family:arial,sans-serif;font-size:13px"> <br></div><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0- U= ses rsync-3.0.9 on the server side (in place of File::RsyncP), with</div><d= iv style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0a C code la= yer to interface to the=A0<span class=3D"" style=3D"background-color:rgb(25= 5,255,204)">BackupPC</span>=A0storage. =A0So the whole data</div> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0path for = rsync is now in compiled C code, which is generally faster than perl</div><= div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0(other tha= n compression, which is the same code in either case, and I/O).</div> <div style=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div s= tyle=3D"font-family:arial,sans-serif;font-size:13px">=A0- Due to the use of= rsync-3.X, acls and xattrs are supported, and other</div><div style=3D"fon= t-family:arial,sans-serif;font-size:13px"> =A0 =A0useful options (but not all) are supported. =A0Rsync protocol 30 sup= ports</div><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 = =A0the efficient incremental file list, which significantly improves memory= </div><div style=3D"font-family:arial,sans-serif;font-size:13px"> =A0 =A0usage and startup time. =A0It also supports MD5 full-file checksums,= which</div><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 = =A0match=A0<span class=3D"" style=3D"background-color:rgb(255,255,204)">Bac= kupPC</span>'s new digest. =A0That allows a full-file digest to be chec= ked</div> <div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0as easily= as an mtime on the server side.</div><div style=3D"font-family:arial,sans-= serif;font-size:13px"><br></div><div style=3D"font-family:arial,sans-serif;= font-size:13px"> =A0- Significant portions of the=A0<span class=3D"" style=3D"background-col= or:rgb(255,255,204)">BackupPC</span>=A0code are now compiled C code in a</d= iv><div style=3D"font-family:arial,sans-serif;font-size:13px">=A0 =A0new mo= dule called=A0<span class=3D"" style=3D"background-color:rgb(255,255,204)">= BackupPC</span>::XS that is dynamically linked to perl.</div> <div style=3D"font-family:arial,sans-serif;font-size:13px"><br></div><div s= tyle=3D"font-family:arial,sans-serif;font-size:13px">=A0- FTP Xfer method i= sn't supported yet.</div></div></div></div> --047d7b6d9868c3f13204dfdc1561-- --===============4170079396409528642== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ This SF.net email is sponsored by Windows: Build for Windows Store. http://p.sf.net/sfu/windows-dev2dev --===============4170079396409528642== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ BackupPC-announce mailing list [email protected] List: https://lists.sourceforge.net/lists/listinfo/backuppc-announce Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ --===============4170079396409528642==--