Re: zero count tokens in decision matrix

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

k b wrote:
> I guess the other approach would be to just not consider any token
> that hasn't reached maturity (combined count of 5 or whatever).
> Then if you don't have enough tokens to fill the decision matrix, so be it.
> 
> Again, this should be a very rare occurrence, but interesting to note.
> Your thoughts?

The general idea is to introduce yet another bias towards scoring a
message non-spam but to do so without compromising accuracy for real
spam.  The significance of this is mostly limited to small databases
when few words have been seen.  If all unknown words were ignored then
there are only a few messages in the database and most words are unknown
then one or two spammy words in a ham could lead to a false positive.

You can tweak this slightly if you'd like.  One way is with the -C
command line option.  For example -C1 would allow all tokens to be used
in scoring even if they have never been seen before.  I haven't tried
that option myself so I don't know how it would work out.

Currently there isn't any command line option to change the default
probability by itself.  You can, however, create a config file and edit
that to set the new word score to whatever you like.  For example
setting it to 0.5 would ensure that new words have no impact on the
score of emails at all while 0.9 would cause all new words to be
considered spam indicators.

Personally I think the default works pretty well.  Obviously, or it
wouldn't be the default. :-)

All the best,
++Brian
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iQEVAwUBRBAq+DxRyEoJfXIFAQK4eQf/YpxRXd4vaNChBge8YE05BjtwEOKq0MBL
vx7J18zHJT3y0k0ld+CBroT6mJGQ7s/1iGi4VxsekSJctbq5X0GwVbwx6q6PkfIX
LbArmEMe1FaHmwABRplpMiHvvbmCsx2ZsG/FWwbSpTbTpCpVo571XGFxjCFAFtWN
Urzk0prbiYiw31lHj9gYuezrXK1OBVWge4xep2jNZgpiwSnblB2Sg2asFL6H0n6+
6dMBbjyA4ZFoiY76KvLblRqXbkcTL+IisCXadssTV2Nw8NXNoawrNuPvdSep3JYe
4ZPK9FgQlv8kSF3tMlhlBXiasPyBPkMxmO2eL3gkcL3/k8zXye1nTA==
=sugF
-----END PGP SIGNATURE-----


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642