Re: New "register if needed" feature?

Randall Nortman <[email protected]>
Newsgroups gmane.mail.bogofilter.devel
Message-ID <[email protected]> 
I've created a *preliminary* patch implementing the feature I
proposed.  The patch against 0.95.2 is attached; I haven't tried
applying it to the CVS code yet.  The code is minimally tested, and I
wasn't very careful about matching the existing indentation style,
which I can fix.  As David Relson said, most of the logic works in
much the same way the thresh_update logic works.  I added two options:

  --train-on-error                  Train only if message is misclassified
  --output-only-misclassified       Output only misclassified messages

Any other opinions on option names are welcome.  I hate naming
things.

The --train-on-error option enables the train on error feature.  It
must be combined with either -n or -s (and cannot be combined with -N,
-S, or -u), and it works in single-message mode as well as with -M,
-B, or -b.  If -v is specified, a line is printed to stderr (actually,
dbgout) saying how many messages were misclassified.

The --output-only-misclassified option is so that scripts can
determine which messages in the set were misclassified (and
subsequently registered in the wordlist).  If combined with -p on an
mbox, the output will be an mbox with only the misclassified messages
(I think; this is untested).  If combined with -v on a maildir or in
bulk mode, you'll get the filenames of only the misclassified
messages, plus the X-Bogosity information (just like -v normally
does).

While thinking of how I'm going to integrate this new feature into my
training scripts, another new feature occurred to me: I'd like to add
a REG_MIXED mode, which would be combined with bulk mode (-b or -B),
to allow registration of both ham and spam in a single run.  Instead
of just providing filenames of objects to be classified, you would
provide "spam:<filename>" or "ham:<filename>".  Then, each time
bogofilter moves on to the next object, it would switch into REG_SPAM
or REG_GOOD mode as appropriate.  This could be combined with
train-on-error or used independently.  This would primarily be useful
for initial training, so that the training script can alternate
between ham and spam messages.  From what I can tell, you need to
register some of each before bogofilter will start classifying
anything, right?  So if train-on-error is going to be useful for
initial training, you need to be able to switch between ham and spam.
(This is how bogominitrain.pl works.)  I intend to use this on
maildirs, and give bogofilter one filename at a time on stdin using
-b, picking files alternately from the ham and spam maildirs.

I think this is rather easy to implement by having the _next_mailstore
functions set the right bit (REG_SPAM or REG_GOOD) in the global
run_type variable each time a mailstore is opened (after parsing out
the "spam:" or "ham:" prefix).  You could also allow it to work in
mbox mode (-M) rather than bulk mode if the messages had a header to
tell bogofilter whether to register them as ham or spam, but that
might be a bit trickier to implement.

But before I do any more on this, I figured I'd share my work so far
in case anybody thinks I'm taking the wrong approach.  Ultimately, I'd
like to see my patch integrated into the mainstream code, if others
consider it useful.  Comments welcome.

Randall Nortman

_______________________________________________
Bogofilter-dev mailing list
[email protected]
http://www.bogofilter.org/mailman/listinfo/bogofilter-dev
patch-train-on-error (text/plain, 7.9 KB) 
diff -ru bogofilter-0.95.2-dist/src/bogoconfig.c bogofilter-0.95.2/src/bogoconfig.c
--- bogofilter-0.95.2-dist/src/bogoconfig.c	2005-06-29 07:12:09.000000000 -0400
+++ bogofilter-0.95.2/src/bogoconfig.c	2005-07-31 19:58:55.000000000 -0400
@@ -142,6 +142,8 @@
     { "timestamp",			R, 0, O_TIMESTAMP },
     { "unsure-subject-tag",		R, 0, O_UNSURE_SUBJECT_TAG },
     { "wordlist",			R, 0, O_WORDLIST },
+    { "train-on-error",			N, 0, O_TRAIN_ON_ERROR },
+    { "output-only-misclassified",	N, 0, O_OUTPUT_ONLY_MISCLASSIFIED },
     /* end of list */
     { NULL,				0, 0, 0 }
 };
@@ -253,7 +255,8 @@
 {
 /*  flags '-s', '-n', '-S', and '-N' are mutually exclusive with
     flags '-p', '-u', '-e', and '-R'. */
-    run_classify = (run_type & (RUN_NORMAL | RUN_UPDATE)) != 0;
+    run_classify =
+	(run_type & (RUN_NORMAL | RUN_UPDATE | RUN_TRAINERROR)) != 0;
     run_register = (run_type & (REG_SPAM | REG_GOOD | UNREG_SPAM | UNREG_GOOD)) != 0;
 
     if (*outfname && !passthrough)
@@ -263,7 +266,7 @@
 		      outfname);
     }
     
-    if (run_register && (run_classify || Rtable))
+    if (run_register && ((run_type & RUN_UPDATE) || Rtable))
     {
 	(void)fprintf(stderr,
 		      "Error:  Option '-u' may not be used with options '-s', '-n', '-S', or '-N'.\n"
@@ -271,6 +274,11 @@
 	return EX_ERROR;
     }
 
+    if ((run_type & RUN_TRAINERROR) && !run_register)
+    {
+	(void)fprintf(stderr,
+		      "Warning: Option --train-on-error has no effect without -s or -n\n");
+    }
     return EX_OK;
 }
 
@@ -350,6 +358,8 @@
     "  --unsure-subject-tag              like spam-subject-tag\n",
     "  --user-config-file                configuration file\n",
     "  --wordlist                        specify wordlist parameters\n",
+    "  --train-on-error                  Train only if message is misclassified\n",
+    "  --output-only-misclassified       Output only misclassified messages\n",
     "\n",
     "bogofilter is a tool for classifying email as spam or non-spam.\n",
     "\n",
@@ -537,7 +547,7 @@
 	break;
 
     case 'N':
-	run_type = check_run_type(UNREG_GOOD, REG_GOOD | UNREG_SPAM);
+	run_type = check_run_type(UNREG_GOOD, REG_GOOD | UNREG_SPAM | RUN_TRAINERROR);
 	break;
 
     case 'O':
@@ -567,11 +577,11 @@
 	break;
 
     case 'S':
-	run_type = check_run_type(UNREG_SPAM, REG_SPAM | UNREG_GOOD);
+	run_type = check_run_type(UNREG_SPAM, REG_SPAM | UNREG_GOOD | RUN_TRAINERROR);
 	break;
 
     case 'u':
-	run_type |= RUN_UPDATE;
+	run_type = check_run_type(RUN_UPDATE, RUN_TRAINERROR);
 	break;
 	
     case 'U':
@@ -708,6 +718,12 @@
     case O_UNSURE_SUBJECT_TAG:		unsure_subject_tag = get_string(name, val);		break;
     case O_UNICODE:			encoding = get_bool(name, val) ? E_UNICODE : E_RAW;	break;
     case O_WORDLIST:			configure_wordlist(val);				break;
+    case O_TRAIN_ON_ERROR:
+	run_type = check_run_type(RUN_TRAINERROR, RUN_UPDATE | UNREG_SPAM | UNREG_GOOD);
+	break;
+    case O_OUTPUT_ONLY_MISCLASSIFIED:
+	output_only_misclassified = true;
+	break;
 
     case O_DB_TRANSACTION:		eTransaction = get_txn(name, val);			break;
 
diff -ru bogofilter-0.95.2-dist/src/bogofilter.c bogofilter-0.95.2/src/bogofilter.c
--- bogofilter-0.95.2-dist/src/bogofilter.c	2005-05-25 19:32:19.000000000 -0400
+++ bogofilter-0.95.2/src/bogofilter.c	2005-07-31 20:20:30.000000000 -0400
@@ -71,10 +71,12 @@
     uint msgcount = 0;
     rc_t status = RC_OK;
     bool register_opt = (run_type & (REG_SPAM | UNREG_SPAM | REG_GOOD | UNREG_GOOD)) != 0;
-    bool register_bef = register_opt && passthrough;
-    bool register_aft = ((register_opt && !passthrough) || (run_type & RUN_UPDATE)) != 0;
+    bool register_bef = register_opt && passthrough && !(run_type & RUN_TRAINERROR);
+    bool register_aft = ((register_opt && !passthrough && !(run_type & RUN_TRAINERROR)) || (run_type & RUN_UPDATE)) != 0;
     bool write_msg    = passthrough || Rtable;
-    bool classify_msg = write_msg || ((run_type & (RUN_NORMAL | RUN_UPDATE))) != 0;
+    bool classify_msg = write_msg || ((run_type & (RUN_NORMAL | RUN_UPDATE | RUN_TRAINERROR))) != 0;
+    uint num_misclassified = 0;
+    bool last_msg_misclassified;
 
     wordhash_t *words;
 
@@ -90,6 +92,7 @@
     while ((*reader_more)()) {
 	double spamicity;
 	wordhash_t *w = wordhash_new();
+	last_msg_misclassified = false;
 
 	rstats_init();
 	passthrough_setup();
@@ -119,14 +122,33 @@
 		if (status == RC_HAM && spamicity >= thresh_update)
 		    register_words(REG_GOOD, w, msgcount);
 	    }
+            else if (run_type & RUN_TRAINERROR)
+            {
+                if (((run_type & REG_SPAM) && (status != RC_SPAM))
+                    || ((run_type & REG_GOOD) && (status != RC_HAM)))
+                {
+                    /* We miscategorized, so first register the message
+                       appropriately */
+                    register_words(run_type & (REG_SPAM|REG_GOOD), w, 1);
+                    ++num_misclassified;
+                    last_msg_misclassified = true;
+                    /* Now re-classify to get the updated score and status */
+                    lookup_words(w);
+                    spamicity = msg_compute_spamicity(w, NULL);
+                    status = msg_status();
+                }
+            }
 
-	    if (verbose && !passthrough && !quiet) {
-		const char *filename = (*reader_filename)();
-		if (filename)
-		    fprintf(fpo, "%s ", filename); 
-	    }
+	    if (!output_only_misclassified || last_msg_misclassified)
+	    {
+		if (verbose && !passthrough && !quiet) {
+		    const char *filename = (*reader_filename)();
+                    if (filename)
+                      fprintf(fpo, "%s ", filename); 
+                }
 
-	    write_message(status);		/* passthrough */
+                write_message(status);		/* passthrough */
+            }
 	    if (logflag && !register_opt) {
 		write_log_message(status);
 		msgcount = 0;
@@ -162,6 +184,12 @@
     if (DEBUG_MEMORY(1))
 	MEMDISPLAY;
 
+    if ((run_type & RUN_TRAINERROR) && verbose) {
+      (void)fprintf(dbgout,
+                    "# %d messages were misclassified and subsequently registered as %s.\n",
+                    num_misclassified, (run_type & REG_SPAM) ? "spam" : "ham");
+    }
+
     return status;
 }
 
diff -ru bogofilter-0.95.2-dist/src/common.h bogofilter-0.95.2/src/common.h
--- bogofilter-0.95.2-dist/src/common.h	2005-06-18 09:37:00.000000000 -0400
+++ bogofilter-0.95.2/src/common.h	2005-07-31 11:55:48.000000000 -0400
@@ -104,7 +104,8 @@
     REG_SPAM   = BIT(2),
     REG_GOOD   = BIT(3),
     UNREG_SPAM = BIT(4),
-    UNREG_GOOD = BIT(5)
+    UNREG_GOOD = BIT(5),
+    RUN_TRAINERROR = BIT(6)
 } run_t;
 extern run_t run_type;
 extern bool  run_classify;
diff -ru bogofilter-0.95.2-dist/src/globals.c bogofilter-0.95.2/src/globals.c
--- bogofilter-0.95.2-dist/src/globals.c	2005-06-28 07:18:52.000000000 -0400
+++ bogofilter-0.95.2/src/globals.c	2005-07-31 19:48:54.000000000 -0400
@@ -36,6 +36,7 @@
 bool	terse;				/* '-t' */
 int	bogotest;			/* '-X', env("BOGOTEST") */
 int	verbose;			/* '-v' */
+bool	output_only_misclassified;	/* '--output-only-misclassified' */
 
 /* config file options */
 double	min_dev;
diff -ru bogofilter-0.95.2-dist/src/globals.h bogofilter-0.95.2/src/globals.h
--- bogofilter-0.95.2-dist/src/globals.h	2005-06-07 19:06:20.000000000 -0400
+++ bogofilter-0.95.2/src/globals.h	2005-07-31 19:49:13.000000000 -0400
@@ -37,6 +37,8 @@
 extern	int	bogotest;		/* '-X', env("BOGOTEST") */
 extern	int	verbose;		/* '-v' */
 extern	bool	replace_nonascii_characters;	/* '-n' */
+extern	bool	output_only_misclassified;	/* '--output-only-misclassified' */
+
 
 /* config file options */
 extern	double	min_dev;
diff -ru bogofilter-0.95.2-dist/src/longoptions.h bogofilter-0.95.2/src/longoptions.h
--- bogofilter-0.95.2-dist/src/longoptions.h	2005-06-20 19:30:56.000000000 -0400
+++ bogofilter-0.95.2/src/longoptions.h	2005-07-31 19:50:51.000000000 -0400
@@ -65,7 +65,9 @@
     O_UNICODE,
     O_UNSURE_SUBJECT_TAG,
     O_USER_CONFIG_FILE,
-    O_WORDLIST
+    O_WORDLIST,
+    O_TRAIN_ON_ERROR,
+    O_OUTPUT_ONLY_MISCLASSIFIED
 } longopts_t;
 
 #ifndef	DISABLE_UNICODE
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.