Re: empty texts

   From: Thomas Michael Hagen <[email protected]>

   is there a technique to classify empty instances (mails or articles
   without any text in them at all)?

   i'm having a bit of trouble getting my categorizer to recognize the
   empty newspaper articles in my corpus as 'out'. it keeps classifying
   them as 'con'(sumer), which is the first category in my list (and also
   the first alphabetically).

   will learning (training) these empty articles as 'out' help at all? so
   far, it doesn't seem so.

No, there isn't any way to train an empty text.

What I'd recommend is to use an EVAL on the length of the input, and
if it's below some threshold (like 20 characters), insert a text like:

   "The original text of this article is too small.  This is probably
    an error in the original encoding.  Sorry, but we can't recover it
    for you."

and classify *that* into "out".  

I think that will do it for you.

  - Bill Yerazunis

------------------------------------------------------------------------------