(no subject)

To: Jason White <[email protected]>
CC: [email protected]
In-reply-to: <[email protected]> (message from Jason
	White on Wed, 6 May 2009 19:42:24 +1000)
Subject: Re: [Crm114-general] Effects of retaining original encoding/headers in
	mail	classification
References:  <[email protected]>


   From: Jason White <[email protected]>

   The accuracy hasn't been particularly high. I just discovered a
   possible cause: at some point in the long forgotten past, for
   reasons that I can't remember, I set "pipe_decode=yes" in my
   ~/.muttrc configuration file. The effect of this, due to other
   configuration settings, is to decode the message and remove all of
   the headers (except From, To, Subject and Date) before piping it to
   Mailreaver for training. Of course, the original headers and
   encoding are intact when messages are processed for classification,
   since Mailreaver is called by Procmail in that case.

   If this is likely to have significantly degraded accuracy, should I retrain
   now from a fresh set of empty CSS files?

Yes.  And empty the reavercache (or just delete that whole directory;
it will be re-created if needed). 

The real problem here is that the "sfid" (Spam Filter IDentification)
field has been destroyed; this is how mailtrainer and mailreaver "look
up" the original form of your email in the reavercache.  

   Obviously, the cache won't be of much use, since the messages are
   stored there in the stripped and decoded form generated by Mutt
   when pipe_decode was incorrectly set.

Yep.  It's useless now.  Nuke it.

   I note that the CRM114 documentation advises users to ensure that
   the messages are trained, as far as possible, in the format in
   which they are received; thus I assume that the headers and
   encoding are known to have a significant impact on accuracy, as is
   to be expected.

Yes.  There's a whole lot of data in the incoming fields that
is very useful to your spamfilter.

   - Bill Yerazunis (back online!)

------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image 
processing features enabled. http://p.sf.net/sfu/kodak-com