Re: Problems with Bayesian filtering

Our bayesian filters are being worked around by spammers using "chaff 
text" which is either in white font on white background, 1 pt font, or 
in some cases, just normal text a few CR's below the spammer's VERY 
SHORT message.

What is the solution to this little trick?

-Carl

[email protected] wrote:

>----- Original Message ----- 
>From: "David Nicol" 
>Sent: Thursday, February 19, 2004 2:05 PM
>Subject: Re: Roster of members ... John Stanley
>
>
>  
>
>>John Stanley wrote:
>>
>>    
>>
>>>discovering (and then discarding) "Bayesian filtering" 
>>>
>>>      
>>>
>>Let's hear some discussion of your arguments against Bayesian Filtering
>>    
>>
>
>Sorry it's taken me so long to respond.  I've been swamped these last 7-days...
>
>From what I've seen, Bayesian tends to max-out somewhere between 92% and 99.8% depending on the specific implementation and the phase of the moon.  To regularly get performance above 90%, the user needs to "understand" (at least at some rudimentary level) how the filter works and needs do more initial up-front training and periodic maintenance than most "Joe/Jane User" types have the patience for...  While this is a dam-sight better than not having a filter at all, it's not the stop-spammers-in-their-tracks kind of performance I'd like to see.
>
>Professionally maintained (ISP-based) Bayesian filters can frequently get higher levels of accuracy, but in the process, they sacrifice one of Bayesian-filtering's largest benefits - the ability for an individual to make their own decision about what is, or is not, spam.
>
>IMHO, for a content filtering system to get real groundswell adoption, it needs to be something that you could give to your grandmother and have it work "out of the box" with minimal maintenance.  None of the Bayesian filters I've seen on the market fit this basic requirement.  Some take weeks/months to train and others require more understanding of Bayesian-filtering and/or basic-IP communication technology and/or other computer details than most users want to know.  (And, no, I don't claim to have looked at "all" of them...  This is just an admittedly blatant over-generalization based on the 10+ filters I've examined.)
>
>Bayesian also has a number of basic problems with what it's actually modeling that start to become a problem when your spam/ham message ratios start to get too far from 50/50.  There are ways to deal with this, but those "ways" again tend to either require more user involvement -and/or- will themselves sometimes cause more false positives/negatives at the "edges" of the model.
>
>There are other reasons why I don't view current-day Bayesian as the be-all and end-all of personal spam filtering, but I think that's enough to start.
>
>Opinions anyone else?  Agree?  Disagree?  Why?
>
>      ... JLS
>  
>

-- 
Carl Hutzler
Director, AntiSpam Operations
America Online Mail Operations
[email protected]
703.265.5521 work
703.915.6862 cell