Re: CRM114 antispam score display (was: Re: branches/KDE/3.5/kdepim/kmail)

Am Sonntag 08 Juli 2007 schrieb Ingo Klöcker:

Hi Ingo!

> > Well yes. Maybe something general where you can specify the complete
> > score range and the necessary thresholds would be suitable.
> >
> > ScoreRange=-400,400
> > ScoreUnsureThreshold=-10
> > ScoreGoodTreshold=10
> >
> > Or just one range for each of those?
>
> Actually, I would prefer a function mapping the scores to the interval
> [0, 1] where 0 or below means "most likely no spam" and 1 or above
> means "most likely spam". Multiplying the score with a factor and then
> adding a value (i.e. an affine transformation) will hopefully suffice
> in all cases. So we would calculate
>
>   double normalizedScore( double score )
>   {
>     return ScoreScalingFactor * score + ScoreTranslation;
>   }
>
> Assuming that for CRM114 anything below -10 means "spam" and anything
> above 10 means "ham" we would use
>
>   ScoreScalingFactor=-0.05 // == - 1/20
>   ScoreTranslation=0.5
>
> and we would get
>
>   normalizedScore(-10) == -0.05 * (-10) + 0.5 == 1
>   normalizedScore(0) == 0.5
>   normalizedScore(10) == -0.05 * 10 + 0.5 == 0
>
> So, as desired, -10 (spam) would map to 1 (spam) and 10 (ham) would map
> to 0 (ham).

So anything between 0 and 1 would be unsure? What about spam filter that 
do not support unsure? Will they only have one threshold spam or not spam 
with anything above it (spam) mapped to above 1 and anything below it 
mapped below 0 (ham) mapped to below 0? But then that would merely be a 
boolean display for those filters. So this normalized score would be 
specific to spam filters that support unsure?

> > > > From what I understand I need to know the exact treshold on that
> > > > CRM114 classifies a mail as SPAM at least?
> > >
> > > Yes.
> >
> > I will ask on the crm114-general mailinglist for that one. CRM114
> > does not seem to specify the treshold in its headers and depending on
> > the classifier one uses the tresholds may vary. Maybe it would be
> > good if CRM114 puts thresholds for SPAM and UNSURE into the headers
> > somehow.
> >
> > > > Ingo, Andreas what about mails classified as UNSURE? Does spam
> > > > score display in KMail support those?
> > >
> > > Well, I guess for scores corresponding to UNSURE the color bar
> > > should be partially filled. For ham the color bar should be empty
> > > and for spam it should be completely filled.
> >
> > Actually I do not quite understand the spam score display
> > completely...
>
> Assuming normalized scores, for ham (0 or below) the color bar is
> completely empty, for spam (1 or above) the color bar is completely
> filled and for anything between 0 and 1 the color bar is partially
> filled.

But then the spam score display does not give any clue about how much a 
spam is spam and how much a ham is ham. It merely gives an boolean 
indication if a mail is spam or ham and a floating indicication of how 
much a unsure mails tends to be spam or ham. This way it would only make 
sense to spam filter that support unsure and only show a limited part of 
their full spam score range!

> > I wonder why there is green in there after all when its a spam. When
> > a mail is unsure or ham I get a blank box. I would have expected
> > something green here ;-).
>
> See above. For unsure you should get a partial gradient, e.g. from
> green to yellow.

Ok, thats not yet implemented, in the current spam wizard entry 
improvement.

> > For some mails that were flagged by
> > SpamAssassin I got a partially filled box with a partial color
> > gradient, for example the gradient up to yellow. I would have
> > interepreted this as UNSURE.
>
> A partially filled box is supposed to signifiy UNSURE.

But does it work that way for spam filters that do not support unsure? A 
filter without the concept of unsureness just decides on how much a mail 
is spam or how much it is spam. Thus the bar would either be completely 
empty or completely filled for those filters.

> > So how does this actually work? Maybe it should be rethought a bit, I
> > do not think its very intuitive. I would use the following:
> >
> > - a red (SPAM) / yellow (UNSURE) / green (HAM) box for a boolean /
> > triplean ;-)
> >
> > - a red (SPAM) / yellow (UNSURE) / green (HAM) bar that displays the
> > amount of spamicity, unsurecity or hamicity. Hmmm, but this might be
> > confusing as well. Need to think about this a bit more.
>
> I think at least for continuous scores (i.e. floating
> point/non-boolean) it's okay as it is. A bit of green means "more
> likely ham than spam". If the gradient extents to yellow it means
> "might be ham or spam". And if the gradient extents to red it means
> "more likely spam than ham".

It still does not make sense for me, cause then a empty spam score display 
(i.e. field is white) means "this is ham for sure" while a partially 
filled bar with a bit of green in it only means "this is more likely ham 
that spam". This just does not seem intuitive to me: A white field 
meaning more ham than one with a bit of green.

Then I at least suggest the following change to the display:

- if the score is below 0 (ham), then the spam score field is completely 
green. That mail is ham, no doubt about it, at least the spam filter has 
none.

- if the score is above 1 (spam), then the spam score field is completely 
red. That mail for sure is spam, at least in the oppinion of the spam 
filter.

If it is between 0 and 1 it is shown as you stated. That would at least 
make *more* sense to me.

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7