Re: SpamProbe 1.2 released

Anthony Campbell wrote:
> On 22 Jun 2005, Jem wrote:
>>>The 32 bit hash database is now "official" so I can't really change it
>>>anymore.  I'd like to add a 64 bit hash file option later but I've been
>>
>>I've been using v1.1x7 with your new Hashed-array-cached database for 3 
>>months and have not seen any problems. I am using a fixed 16 MB database 
>>and the accuracy is just as good as before, with a much larger PBL db.
>>
> 
> So am I, except that I used a 32MB database. But I see that Brian's
> advice is to use pbl in general so I'm thinking of changing. I'm a bit
> worried about running out of space eventually.

The primary reason I recommend PBL over hash if you have enough disk 
space is that with PBL there is no chance for ambiguity in the database. 
  The hash stores 32-bit hash code not the terms themselves so you can 
get collisions.  This is most dangerous if it happens that a message 
digest (used by SP to remember how it counted a particular message's 
terms) collides with another word and you decide to retrain that message.

Is that really going to be a problem for people?  So far I haven't seen 
any problem with this in my own database.  In my recommendations in the 
readme I try to be conservative so that average users don't get in over 
their heads and yell at me. :-)

The best of both worlds is probably the split database type.  That uses 
a hash file for the message terms and a tiny PBL (or BDB) database for 
the message digests.  I might switch to that for my own email.

All the best,
++Brian


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click