Re: hash question
Brian Burton <[email protected]>
| Newsgroups | gmane.mail.spam.spamprobe.general |
|---|---|
| Message-ID | <[email protected]> |
Thomas Schürger wrote: >>If I used a digest (128 bit MD5 hash) the record size would double to 24 >>bytes but losses would be virtually eliminated. Computing an MD5 for >>each term would probably be significantly slower than either 32 or 64 >>hash as well. > > Using a 64 bit hash key sounds reasonable (in relation to the numer of > unhashed keys to store). More important would be the use of a good > hash function which minimizes collisions. Something like taking the > first 64 bits from the MD5, SHA-1 or RIPEMD160 hashs could be a good > choice. I'll have to write a program to experiment with hash collision rates on my 2.24 million term database. I have some free hash code to use evahash, FNV, and MD5. I'll be very interested to see how many collisions I get vs. CPU time to compute the hash. I'm inclined to rework the hash format to use a 64 bit key just as a precaution. Maybe I could offer a choice of either with a little code tweaking. All the best, ++Brian ------------------------------------------------------- This SF.net email is sponsored by Demarc: A global provider of Threat Management Solutions. Download our HomeAdmin security software for free today! http://www.demarc.com/info/Sentarus/hamr30