Re: Proposal for variable length extents

On Tue, May 30, 2006 at 05:23:44PM -0600, Andreas Dilger wrote:
> Interesting.  Do you have any good algorithms for computing
> non-obviously-weak checksums efficiently?  I agree that we likely
> don't care about crypto strong checksums, but rather blatant
> corruption.  We need some form of feedback or we can lose out on
> some kinds of corruption if we just do a simple sum.

Well, the obvious choice if we really think CRC's are too expensive is
the ones-complement checksum as used by TCP/IP.  Why it is better than
a simple additive checksum is described fairly well here:

http://www.netrino.com/Connecting/1999-11/index.html

I'd like to reserve judgement on whether or not using a CRC really
results in a measurable increase of CPU load given a reasonable
workload, though.  Especially given that the main case where we this
tends to be a major issue is for databases, and for many databases at
steady state the number of metadata operations that would result in
checksums is probably small enough that it might not be measurable.

						- Ted