Re: more info on 25gig files

On Thu, 2005-06-30 at 07:37, [email protected] wrote:
> Hi,
> 
> Donovan Baarda wrote:
> > There was a long discussion on the rsync lists about the best heuristic
> > for blocksize and blocksum size (rsync also trims the strongsum size to
> > make the signature smaller, and it was getting blocksum collisions on
> > large files). I helped figure out the formula's for blocksize and
> > blocksum size that rsync now uses.
> 
> There's a related issue we discussed a while ago:
> One thing that's different between rsync and naive use of librsync (as
> done by rdiff and, I think, rdiff-backup) is that the latter doesn't
> have a strong full-size integrity check. With librsync's default 4-byte
> checksums, using a small block size on a large file makes the
> probability of silent corruption disturbingly non-negligible.

Actually, librsync uses 8 byte strong checksums, which combined with the
4 byte rollsum, gives you 12 bytes of checksum for each block (but you
can only really count on 8 bytes for maliciously crafted data). This
makes it comfortingly negligable unless you have huge files and a
stupidly small blocksize.

The disturbing part is that any corruption that does occur is silently
ignored.

-- 
Donovan Baarda <[email protected]>



-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click