Re: hash question

Brian Burton <[email protected]>
Newsgroups gmane.mail.spam.spamprobe.general
Message-ID <[email protected]>
Thomas Schürger wrote:
>>If I used a digest (128 bit MD5 hash) the record size would double to 24 
>>bytes but losses would be virtually eliminated.  Computing an MD5 for 
>>each term would probably be significantly slower than either 32 or 64 
>>hash as well.
> 
> Using a 64 bit hash key sounds reasonable (in relation to the numer of
> unhashed keys to store). More important would be the use of a good
> hash function which minimizes collisions. Something like taking the
> first 64 bits from the MD5, SHA-1 or RIPEMD160 hashs could be a good
> choice.

I'll have to write a program to experiment with hash collision rates on 
my 2.24 million term database.  I have some free hash code to use 
evahash, FNV, and MD5.  I'll be very interested to see how many 
collisions I get vs. CPU time to compute the hash.

I'm inclined to rework the hash format to use a 64 bit key just as a 
precaution.  Maybe I could offer a choice of either with a little code 
tweaking.

All the best,
++Brian


-------------------------------------------------------
This SF.net email is sponsored by Demarc:
A global provider of Threat Management Solutions.
Download our HomeAdmin security software for free today!
http://www.demarc.com/info/Sentarus/hamr30
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.