BlameThorstenAndJenny: doesn't appear to classify some mails correctly after learning

Hi!

I reported this as Debian bug, bug number still pending.

Hmmm, maybe its not a bug at all. And crm114 needs some more of similar 
mails to classify correctly. At least it classified some already learned 
and afterwards incorrectly classified mails correctly after I learned some 
more similar mails. What do you think? Is this expected behaviour? I do 
not remember this happening this way. I think before a learning of a 
certain mail had an immediate effect on its spam status after 
reclassifying.
 

I updated to crm114, 20090807, BlameThorstenAndJenny today. Since cssutil 
complained:

 Minor Caution - this file has the learncount slot in use.
 This is not a problem for Markovian classification, but it will have some
 issues with an OSB classfier.

and I use a OSB classifier and also due to the debconf warning, I recreated 
the files from scratch.

It appears to learn:

martin@shambhala:~/.crm114> ./stats.sh
Als SPAM markiert: 3

 Sparse spectra file spam.css statistics:

 Total available buckets          :      1048577
 Total buckets in use             :         1481
 Total in-use zero-count buckets  :            0
 Total buckets with value >= max  :            0
 Total hashed datums in file      :         1565
 Documents learned                :           40
 Features learned                 :         1566
 Average datums per bucket        :         1.06
 Maximum length of overflow chain :            2
 Average length of overflow chain :         1.00
 Average packing density          :         0.00

Als HAM markiert: 20

 Sparse spectra file nonspam.css statistics:

 Total available buckets          :      1048577
 Total buckets in use             :         8187
 Total in-use zero-count buckets  :            0
 Total buckets with value >= max  :            0
 Total hashed datums in file      :         8389
 Documents learned                :           41
 Features learned                 :         8390
 Average datums per bucket        :         1.02
 Maximum length of overflow chain :            2
 Average length of overflow chain :         1.01
 Average packing density          :         0.01


Script is:

martin@shambhala:~/.crm114> cat ./stats.sh
#!/bin/sh

echo "Als SPAM markiert: $(find reaver_cache/known_spam/ | wc -l)"
cssutil -rb spam.css

echo "Als HAM markiert: $(find reaver_cache/known_good/ | wc -l)"
cssutil -rb nonspam.css


Yet CRM114 doesn't appear to recognize some mails correctly after 
learning.

I learn it:

martin@shambhala:~/Zeit> cat Aufräumpolicy\ _tmp.mbox| crm -u ~/.crm114 
mailreaver.crm --good | grep CRM
/bin/ln: Erzeuge harte Verknüpfung 
„reaver_cache/known_good/20091103_160046_476304_CDA7397D“: Die Datei 
existiert bereits
X-CRM114-Version: 20090807-BlameThorstenAndJenny ( TRE 0.7.6 (BSD) ) 
MR-27CA1CFB
X-CRM114-CacheID: sfid-20091103_160046_476304_CDA7397D
X-CRM114-Notice: Please train this message.
X-CRM114-Action: LEARNED AND CACHED GOOD

This is what CRM114 thinks about it afterwards:

martin@shambhala:~/Zeit> cat Aufräumpolicy\ _tmp.mbox| crm -u ~/.crm114 
mailreaver.crm | grep CRM
X-CRM114-Version: 20090807-BlameThorstenAndJenny ( TRE 0.7.6 (BSD) ) 
MR-27CA1CFB
X-CRM114-CacheID: sfid-20091103_160046_476304_CDA7397D
X-CRM114-Status: UNSURE (   8.14  )
X-CRM114-Notice: Please train this message.


These are my differences to the original configuration:

martin@shambhala:~/.crm114> diff -u /usr/share/crm114/mailfilter.cf 
mailfilter.cf
--- /usr/share/crm114/mailfilter.cf     2009-08-07 17:22:37.000000000 +0200      
+++ mailfilter.cf       2009-11-03 16:11:45.779852564 +0100                      
@@ -169,8 +169,8 @@                                                              
 #  ---------  will be inserted at the front of the subject if we think 
the      
 #  ---------  mail is spam.                                                     
 #                                                                               
-# :spam_flag_subject_string: //                                                 
-:spam_flag_subject_string: /ADV:/                                               
+:spam_flag_subject_string: //                                                   
+#:spam_flag_subject_string: /ADV:/                                              
                                                                                 
 #  ---------  Do we want to insert a "flagging" string on the subject line      
 #  ---------  for good email?  Usually we don't.... so we set this to the       
@@ -180,13 +180,13 @@                                                            
 #  ------------Similarly, do we want to insert a "flagging" string on           
 #  -------------the subject line of an "unsure" email?  This way we know        
 #  --------------we need to train it even if "headers" is turned off.           
-# :unsure_flag_subject_string: //                                               
-:unsure_flag_subject_string: /UNS:/
+:unsure_flag_subject_string: //
+# :unsure_flag_subject_string: /UNS:/

 # ------------- Do we want Training ConFirmation flags on the results of
 # ------------- a message to be learned?  Default is "TCF:".
-:confirm_flag_subject_string: /TCF:/
-#:confirm_flag_subject_string: //
+#:confirm_flag_subject_string: /TCF:/
+:confirm_flag_subject_string: //


 # ---------  Do we want to do any "rewrites" to increase generality and
@@ -194,16 +194,16 @@
 #    --------- NOTE: this option is somewhat slow.  If your mailserver is
 #      --------- maxed out on CPU, you might want to turn this off.
 #
-:rewrites_enabled: /yes/
-#:rewrites_enabled: /no/
+#:rewrites_enabled: /yes/
+:rewrites_enabled: /no/


 #  ---------  Do we copy incoming text into allmail.txt ?  default is 
yes, but
 #   ---------  experienced users will probably set this to 'no' after 
testing
 #    ---------  their configuration for functionality.
 #
-:log_to_allmail.txt:  /yes/
-# :log_to_allmail.txt: /no/
+# :log_to_allmail.txt:  /yes/
+:log_to_allmail.txt: /no/

 #   -------  Another logging option - log all mail to somewhere else
 #    -------  entirely.  Whatever pathname is given here will be prefixed

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference

_______________________________________________
Crm114-general mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/crm114-general

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)

iEYEABECAAYFAkrwUEcACgkQmRvqrKWZhMdclQCfeKG5GnRIG4BCfOxdxzYy/ypI
akIAoJZcIK79BvHBPOuvqcg30JKZhb0Z
=scJa
-----END PGP SIGNATURE-----