Re: Fwd: Re: ifile vs. Graham modified naive bayes?

> [email protected]:

>    What would be nice would be to stuff the stats into a DBM file on a
>    word by word basis.

is it worth the complications in building?  now ifile does a prertty good
job and runs on about every platform.  if we wanted speed, we might use
mmap(2).  it is feasable with current database sizes and might perform
better than a DB without beeing as complicated.  difficult to draw the
line, though.  more features could lead to a situation where a DB might
help much more.  the pure text-format of the current database made it
easier in case someone comes up with tools to improve the classifier
"afterwards", i mean some sort of an optimizer.  does that make sense?

>    It is always possible with email to detect where the headers are, and
>    is usually possible to detect a .signature.

i agree re. headers and MIME, but everything else mutates ifile into
something bigger.  something bigger than ifile should be well designed.  i
find particular appealing the small size of it, the robustness and the
simplicity.  everybody can understand it, even the way it (tries to) adapt
to stuff appearing in headers versus possibly the same looking stuff in the
body.

so, what are the features ifile has that need improving the most?

and what features does ifile not have, but should?

>    I'd think it helpful to be able to tell that certain words are found
>    in those two areas of the message.

i think it would be better if ifiles ability to classify border-cases were
improved, thus making it more "precise" with the risk of more false
classifications.  and i say yes to headers and MIME distinguishing, but no
to the DB before mmap(2) is tested.

> As a pointed example, it has the result that some messages with
> .signatures containing the word "sexy" would not lead to any confusion
> of this with messages containing the word "sexy" in the body.  They will
> be regarded as totally distinct.  Headers will correlate with headers;
> body with body; .sig with .sig.  That's not turning it into being
> _totally_ "contexted," but it adds a _little_ more context.

i have to admit that i'm against .sig distinguishing, because there are so
many people out there who either don't know how sigs are formatted or whose
mailers have a propriatary idea about it.  you know how we mean :)

clemens