Re: UNS: Re: crm filter cleanup on productive system

On Sun, Feb 22, 2009 at 7:27 PM, Frank Doege <fdoege-zZ82ZiX1S8lUvkYWv5dGcgC/[email protected]> wrote:
> Hi Ger & all,
>
> thanks for your answer.
>
> What exactly do you mean by double sided training ?
>
> Currently i train always the unsure messages and tell the filter if they
> are spam or ham. If a message is wrongly classified into junk or good i
> train that too. Is that already the double sided training you meant ?

Nope. 'double sided training' is something that's happening in the,
err, backend, rather.

It comes down to this: you have your usual training scheme, such as
the one you describe (which is known as Training On Error, or, more
correctly, given the (default) settings of mailtrainer.crm, as Thick
Threshold Training. You see a mistake or an unsure, you train it. So
far, nothing different for 'single sided' or 'double sided'.

The difference is at a lower layer: OSB is trained 'single sided',
i.e. you tell the spam CSS database there's another spam to be trained
OR you tell the ham CSS database there's another ham to be trained. In
fact, that's what mailtrainer.crm does for you, under the hood: you
say 'ham' or 'spam' and it will issue the proper training comment with
the proper CSS database. (database: 'singular' ;-) )

Now 'double sided' doesn't change a thing about this, EXCEPT, under
the hood, you train a spam by training it /into/ the spam CSS (like
with the 'single sided' approach described above) AND you train it
/out of/ the ham CSS database. Thus, it's training into/out of
databases, plural. Since we only have two of those, it makes sense to
call that one 'double sided'.

For completeness, here's how crm114 trains 'out of' instead of 'into':
you can specify the training option <refute> (see QUICKREF.txt) with a
classifier 'train' command: that's the signal crm114 needs to know you
want it to train in a way which is saying 'this does /not/ belong in
here', while regular training is a way to teach the classifier that
'this does belong in this category'.

In GerH's mailfilter.cf, here's the relevant snippet:

#
# [i_a]
# Set to empty // if you do want mailtrainer to use 'regular'
one-sided training.
# 'double sided' training means a message is not only trained as X (spam or ham)
# but _also_ <refute>-trained as !X (not spam or not-ham) at the same time.
#
# Normally, you wouldn't SET this option, unless you're using, for example, the
# winnow classifier, which fares quite a bit better when trained doublesided.
#
# NOTE: of course, you can only SET this option when using classifiers which
#       support the <refute> attribute!
#
# Default when not specified: NOT SET.
#
#:do_doublesided_training: /SET/
:do_doublesided_training: //

In case of a winnow config like this:

  :clf: /alt.winnow/

you must ensure you've commented out that doublesided stuff so this
becomes the active line (and the other is removed or '#' commented
out):

  :do_doublesided_training: /SET/

to get optimum performance for winnow.
For all the others, e.g.

  :clf: /alt.osbf unique microgroom/

you should force mailtrainer.crm to use single sided training, as that
is what all the other classifiers like best:

  :do_doublesided_training: //


(Yeah, now it's a manual option, which risks setting it up
incorrectly, resulting in seriously degraded performance
filtering-wise, so the next release will probably see an 'autosense'
bit of code which sets this up for you. Right now, it was done as a
manual parameter since I've been using that for my tests.)





> I understand that the neural network option is not yet for mail but iam
> just to much interested in using it for other purposes, i got the email
> from Bill this morning and i have currently some problems in
> understanding how this exactly works, but ill proceed in this discussion
> in a seperate mail.
>
> May i ask if you and Bill are the math gurus, which implemented the
> algorithms itself ?

Bill's the guru.
I came in back in '07 as I wanted this bugger to run smoothly on my
various systems (Windows and UNIX); as I employ crm114 in non-email
environments I had to grok classifiers way beyond the 'use this
setting and we're probably doing all right' level. That includes both
algorithms and implementations.

I think I 'get' the Bayesian/Markovians quite well by now, Hyperspace
is the runner-up grok-wise and SVM and the others are at a level where
I 'can see', but still have a hard time with the intricacies of the
implementations. (Where the mark of 'really knowing it' is: "complete
these exercises: (1) implement the buggers yourself - to show you got
it, indeed, (2) list the [implicit] assumptions and shortcuts used in
both algorithms /and/ their crm114 implementations and discuss their
benefits and drawbacks, (3) suggest different ways to implement them
to improve (a) run-time classification speed, (b) accuracy (to be
defined), (c) circumvent selected assumptions / shortcuts." When you
have to tune this sort of stuff for your own purposes, nothing less
will do. [Can't tune a voodoo box, unless you count [semi]random
trial&error as a viable tuning system. ;-)) ])



-- 
Met vriendelijke groeten / Best regards,

Ger Hobbelt

--------------------------------------------------
web:    http://www.hobbelt.com/
        http://www.hebbut.net/
mail:   [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H