Re: Re: Error (signal 11) during auto-train

> It's also very difficult to have a completely accurate corpus of any
> significant size.  I keep one which I feed through as a check every
> few months, and even though it has been as closely checked as I can
> manage, among the false positives that SP finds, there's always one
> or two genuine spams that slipped through the first 3 or four times
> round the loop!

OK, these mails date back to 2003, and I've refined the accuracy
of the corpus roughly every 3 months.  Look what it found today
in one corpus of roughly 7500 supposedly spam mails:

>O 1 [email protected] Thu Oct 17 07:03 57/2629 Online FedEx Tracking -
xxxxxxxxxxxxxxxx
 O 2 [email protected] Thu Jan 16 23:00 298/12474 Congratulations!  You won
"ATHEARN 3-BAY COVERED HOPP
 O 3 MAILER-DAEMON@aol.  Thu Dec 5 17:19 106/3788 Returned mail: User unknown
 O 4 [email protected] Fri Jan 3 09:04 208/6727 Your Alert for "terry
pratchett"
 O  5 Continental_Airlin  Wed Feb 19 17:49  68/4202  OnePass Partner Offers
 O 6 alerts-auctions@ya Sun Feb 16 06:30 61/2835 Search terms: bali card game
- SPLASH DOWN ( PLAYSTAT
 O  7 Service-XsV/[email protected]  Thu Apr 24 23:51  91/2914  Domain VT.com
 O 8 customerservice@ap Sun May 4 17:33 37/1300 Customer Service at Apex
digital Inc.

One of those was an offer of $8K for a domain I hold - in the spam
bucket for three years and never caught in dozens of rechecks.  Yikes.

I almost despair of ever creating a completely clean test suite
of any significant size.  Note that for a good testing regimen
you need more than one corpus, so that you don't find yourself
'training the corpus'.  I.e. you may use one set for
training, but you use a different set to measure the effectiveness
of the training.  Anyone who has ever written a compiler will
be intimately familiar with the problem here.


Graham


-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642