Re: Binary comparison of files

The functions take a buffer with offset/length as input exactly for that
reason. I just conformed to code posted before.

Sébastien

On Wed, Jan 28, 2009 at 3:36 PM, Peter vd Weerd <[email protected]> wrote:

> The code now reads in both complete files before starting a compare.
> Its also possible to read the files in chuncks of 4KB, and then doing a
> compare of those 4KB.
>
> This will lead to a (much) lower memory consumption and a much faster
> result for large files that have differences in the first part.
>
> /P
>
> ----- Original Message ----- From: "Sébastien Lorion" <
> [email protected]>
> To: <[email protected]>
> Sent: Wednesday, January 28, 2009 9:17 PM
> Subject: Re: [ADVANCED-DOTNET] Binary comparison of files
>
>
> Unrolling the loop in your method would make the code much faster. Also, if
> unsafe code is not an issue, casting the byte[] to int* and comparing ints
> instead of bytes would further increase the speed.
>
> Attached is code I had laying around and that I adapted for your case (not
> fully tested).
>
> Sébastien
> On Wed, Jan 28, 2009 at 1:26 PM, Geoff Taylor <[email protected]
> >wrote:
>
>  > If all you need to know is that the files are different; I would use a
>> > hash.  CRC32 is fairly fast.  For more reliability you can use a
>> > cryptographic hash.
>> >
>> > If you want to know what the differences are; the performance would
>> > depend on how you plan on processing the differences.
>>
>> Surely a hash will require processing all the file before returning its
>> result?  So if the first byte is different, it's still going to read in
>> all
>> of both files and calculate both hashes.
>>
>> Why not just go for a simpler approach using buffered IO.  You read two
>> chunks (one chunk from each file) into byte buffers then do a byte-by-byte
>> comparison, breaking the loop at the first byte that doesn't match.
>>
>> Since they're small files, you could probably just do a ReadAllBytes
>> instead
>> of reading into buffers.  That would give you the length of both, so you
>> could check that first, giving you another shortcut to a fast negative.
>>
>> Here's a similar method that uses assertions instead of failure conditions
>> that should show you what I mean:
>>
>>       public static void Compare (string referenceFilename, string
>> testFilename)
>>       {
>>           byte[] referenceFile = File.ReadAllBytes (referenceFilename);
>>           byte[] testFile = File.ReadAllBytes (testFilename);
>>           Assert.AreEqual (testFile.Length, referenceFile.Length, "Files
>> are of different lengths.  Reference file is {0} bytes, test file is {1}
>> bytes.", referenceFile.Length, testFile.Length);
>>           for (int counter = 0; counter < referenceFile.Length; counter++)
>>           {
>>               if (referenceFile [counter] != testFile [counter])
>>               {
>>                   Assert.Fail ("Files do not match (at position " +
>> counter + " - [before '" + indicatorString + "']).");
>>               }
>>           }
>>
>>           return;
>>       }
>>
>> I'm pretty sure this'll be faster than using a hash (although I'd love to
>> see a comparison of timings).
>>
>> You might be able to make the method faster by looking at how the data is
>> read in.
>>
>> Good luck,
>>
>>                       Geoff
>>
>> ===================================
>> View archives and manage your subscription(s) at
>> http://peach.ease.lsoft.com/archives
>>
>>
> ===================================
> View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
>
> ===================================
> View archives and manage your subscription(s) at
> http://peach.ease.lsoft.com/archives
>

===================================
View archives and manage your subscription(s) at http://peach.ease.lsoft.com/archives