Re: Fwd: general document classification using crm114

On Fri, Jul 18, 2008 at 1:18 PM, Hoon-Young Cho <[email protected]> wrote:
>  Bit Entropy sounds promising..but, if documents can be tokenized (or word
>  -delimited), then can BE beat other approaches? anyway, I think I have to
>  try your first or second approach first before I can test the BE method.

I don't have numbers on this, so I don't know if BE will beat other
classifiers given that you are able to tokenize the input.
Besides, since your documents are rather short, you'll probably
experience different behaviour for any classifier as most published
material is about (longer) emails.

Anyway, a good thing would be to create your own particular test set
for testing the various classifiers (and learn methods! Learn methods
can have a huge impact on the results) and see which one works best
for your data feed.

Oh, and since you have such small documents, DO NOT take my advice of
'making them look like emails': the added data, though identical for
all data files then, will noticably influence your pR numbers. Think
of it like this: say you add a 'header' line to make it look like a
real email; that line is 5 words. Your /actual/ data is only 10 words,
so that makes a document of 15 words. Now IF a classifier X finds one
of those words to be 'remarkable' (either good or bad), generally
speaking its influence on the final pR number will be somewhat reduced
when it's only one shouting in a sea of 15 instead of a lake of 10. I
know, not all classifiers work like this, and it also depends on the
options/flags you feed your classifier, but when you don't know
/exactly/ what your classifier does to your data & numbers, it's
better not to introduce the extra risk of 'watering' your results this
way.
For larger documents it's no matter, because the added amount is
insignificant, but for your small sized documents, the impact /can/ be
huge.

So that would mean you'ld better go the route of checking out the
mail*.crm scripts -- they can handle your data as is, btw -- and see
what you want to do with that.

If you wish to improve your numbers, and given your data
specifications now, I'd say: start with testing various learn methods.
mailtrainer.crm supports them all, in a way, it's just that you'll
need to 'translate' the various published methods (e.g. TOE, TONE,
etc.) into the proper valaues for the various parameters in
mailconfig.cf to configure mailtrainer.crm act in different ways.

-- 
Met vriendelijke groeten / Best regards,

Ger Hobbelt

--------------------------------------------------
web: http://www.hobbelt.com/
 http://www.hebbut.net/
mail: [email protected]
mobile: +31-6-11 120 978
--------------------------------------------------

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/