16 bit checksums limit dar's potential

With all the effort that went into making dar as limitless as possible,
I'm stuck wondering why 16 bit checksums were chosen.    16 checksum
bits only covers 65536 possible checksums.  And because of dar's almost
limitless potential, it's very easy to create archives that expose the
weakness of 16 bit checksums.

 

Some examples:

 

Once an archive holds 65537 files, then you are guaranteed that 2 files
have the same checksum.    If the contents of those files were flipped
somehow in the archive, testing would not detect this.

 

16 bit checksums make it impossible to implement an effective
approximate "diff" operation that uses only checksum information from
the catalog and the filesystem.    An approximate diff operation like
this would be very useful in my opinion because I could keep all the
catalog files online and use the quick diff based on checksum from the
catalog to determine whether or not I had to fetch the physical media
holding the full dar archive to address a possible file corruption.

 

The same approximate diff operation could be used to implement an
incremental archive option that relied on checksums instead of
timestamps.  At 16 bits I wouldn't trust that check.  At 128 bits I
would.

 

Also long checksums allow dar catalogs to be used as a tool to manage
filesystem redundancy.  If strong, cryptographic checksums (128bits or
longer) are used then checksum collusions can be checked to detect cases
where the same data is being stored over and over again.

 

 

Is there any interest in getting larger checksums in dar?  

 

Thanks,

Ryan Thomas

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

_______________________________________________
Dar-discussions mailing list
Dar-discussions-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/dar-discussions