Re: Fwd: general document classification using crm114

   From: "Ger Hobbelt" <[email protected]>

   On Fri, Jul 18, 2008 at 1:18 PM, Hoon-Young Cho <[email protected]> wrote:
   >  Bit Entropy sounds promising..but, if documents can be tokenized (or word
   >  -delimited), then can BE beat other approaches? anyway, I think I have to
   >  try your first or second approach first before I can test the BE method.

   I don't have numbers on this, so I don't know if BE will beat other
   classifiers given that you are able to tokenize the input.
   Besides, since your documents are rather short, you'll probably
   experience different behaviour for any classifier as most published
   material is about (longer) emails.

   Anyway, a good thing would be to create your own particular test set
   for testing the various classifiers (and learn methods! Learn methods
   can have a huge impact on the results) and see which one works best
   for your data feed.

Yes, very much so.  

   Oh, and since you have such small documents, DO NOT take my advice of
   'making them look like emails': the added data, though identical for
   all data files then, will noticably influence your pR numbers. Think
   of it like this: say you add a 'header' line to make it look like a
   real email; that line is 5 words. Your /actual/ data is only 10 words,
   so that makes a document of 15 words. Now IF a classifier X finds one
   of those words to be 'remarkable' (either good or bad), generally
   speaking its influence on the final pR number will be somewhat reduced
   when it's only one shouting in a sea of 15 instead of a lake of 10. I
   know, not all classifiers work like this, and it also depends on the
   options/flags you feed your classifier, but when you don't know
   /exactly/ what your classifier does to your data & numbers, it's
   better not to introduce the extra risk of 'watering' your results this
   way.
   For larger documents it's no matter, because the added amount is
   insignificant, but for your small sized documents, the impact /can/ be
   huge.

Also true.

   So that would mean you'ld better go the route of checking out the
   mail*.crm scripts -- they can handle your data as is, btw -- and see
   what you want to do with that.

   If you wish to improve your numbers, and given your data
   specifications now, I'd say: start with testing various learn methods.
   mailtrainer.crm supports them all, in a way, it's just that you'll
   need to 'translate' the various published methods (e.g. TOE, TONE,
   etc.) into the proper valaues for the various parameters in
   mailconfig.cf to configure mailtrainer.crm act in different ways.

Maybe.... but if you have a big bash script on top anyway, you
might be better off using the short little scripts from the
manual to learn and to classify.

   - Bill Yerazunis

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/