Re: Binary comparison of files

The code now reads in both complete files before starting a compare.
Its also possible to read the files in chuncks of 4KB, and then doing a 
compare of those 4KB.

This will lead to a (much) lower memory consumption and a much faster result 
for large files that have differences in the first part.

/P

----- Original Message ----- 
From: "Sébastien Lorion" <[email protected]>
To: <[email protected]>
Sent: Wednesday, January 28, 2009 9:17 PM
Subject: Re: [ADVANCED-DOTNET] Binary comparison of files


Unrolling the loop in your method would make the code much faster. Also, if
unsafe code is not an issue, casting the byte[] to int* and comparing ints
instead of bytes would further increase the speed.

Attached is code I had laying around and that I adapted for your case (not
fully tested).

Sébastien
On Wed, Jan 28, 2009 at 1:26 PM, Geoff Taylor 
<[email protected]>wrote:

> > If all you need to know is that the files are different; I would use a
> > hash.  CRC32 is fairly fast.  For more reliability you can use a
> > cryptographic hash.
> >
> > If you want to know what the differences are; the performance would
> > depend on how you plan on processing the differences.
>
> Surely a hash will require processing all the file before returning its
> result?  So if the first byte is different, it's still going to read in 
> all
> of both files and calculate both hashes.
>
> Why not just go for a simpler approach using buffered IO.  You read two
> chunks (one chunk from each file) into byte buffers then do a byte-by-byte
> comparison, breaking the loop at the first byte that doesn't match.
>
> Since they're small files, you could probably just do a ReadAllBytes
> instead
> of reading into buffers.  That would give you the length of both, so you
> could check that first, giving you another shortcut to a fast negative.
>
> Here's a similar method that uses assertions instead of failure conditions
> that should show you what I mean:
>
>        public static void Compare (string referenceFilename, string
> testFilename)
>        {
>            byte[] referenceFile = File.ReadAllBytes (referenceFilename);
>            byte[] testFile = File.ReadAllBytes (testFilename);
>            Assert.AreEqual (testFile.Length, referenceFile.Length, "Files
> are of different lengths.  Reference file is {0} bytes, test file is {1}
> bytes.", referenceFile.Length, testFile.Length);
>            for (int counter = 0; counter < referenceFile.Length; 
> counter++)
>            {
>                if (referenceFile [counter] != testFile [counter])
>                {
>                    Assert.Fail ("Files do not match (at position " +
> counter + " - [before '" + indicatorString + "']).");
>                }
>            }
>
>            return;
>        }
>
> I'm pretty sure this'll be faster than using a hash (although I'd love to
> see a comparison of timings).
>
> You might be able to make the method faster by looking at how the data is
> read in.
>
> Good luck,
>
>                        Geoff
>
> ===================================
> View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
>

===================================
View archives and manage your subscription(s) at 
http://peach.ease.lsoft.com/archives

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives