Re: SpamProbe 1.2 released

On 22 Jun 2005, Brian Burton wrote:
> Anthony Campbell wrote:
> >On 22 Jun 2005, Jem wrote:
> >>>The 32 bit hash database is now "official" so I can't really change it
> >>>anymore.  I'd like to add a 64 bit hash file option later but I've been
> >>
> >>I've been using v1.1x7 with your new Hashed-array-cached database for 3 
> >>months and have not seen any problems. I am using a fixed 16 MB database 
> >>and the accuracy is just as good as before, with a much larger PBL db.
> >>
> >
> >So am I, except that I used a 32MB database. But I see that Brian's
> >advice is to use pbl in general so I'm thinking of changing. I'm a bit
> >worried about running out of space eventually.
> 
> The primary reason I recommend PBL over hash if you have enough disk 
> space is that with PBL there is no chance for ambiguity in the database. 
>  The hash stores 32-bit hash code not the terms themselves so you can 
> get collisions.  This is most dangerous if it happens that a message 
> digest (used by SP to remember how it counted a particular message's 
> terms) collides with another word and you decide to retrain that message.
> 
> Is that really going to be a problem for people?  So far I haven't seen 
> any problem with this in my own database.  In my recommendations in the 
> readme I try to be conservative so that average users don't get in over 
> their heads and yell at me. :-)
> 
> The best of both worlds is probably the split database type.  That uses 
> a hash file for the message terms and a tiny PBL (or BDB) database for 
> the message digests.  I might switch to that for my own email.
> 
> All the best,
> ++Brian

Thanks for this clarification. I still feel that the better flexibility
of pbl (i.e. you can see the original terms) is quite an advantage.

On a different note, I've found the -H option to be extremely useful for
catching the huge number of bounced mails that I'm getting now. By
filtering on the "Return-Path" header I get every single one of them!

Anthony

-- 
ac-MeA/[email protected]    ||  http://www.acampbell.org.uk for
using Linux GNU/Debian ||  blog, book reviews, electronic  
Microsoft-free zone    ||  books and skeptical articles


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_id=7477&alloc_id=16492&op=click