Re: Re: Dangerous locale patch to 1.1.3

Jens Peter Secher <[email protected]>:

> Camillo Särs <[email protected]> writes:
>
>> I think it's getting increasingly obvious that ifile as a whole needs several 
>> different modules.  Right away, the following spring to mind:
>
> I don't think so.

same here.

>> - MIME-decoder.  "Text" parts, for your favorite definition of "text", can be 
>> tokenized.  Other parts can generate features for filtering, but should not be 
>> tokenized as such.
>> - MIME interpreter that can flag things like character sets, languages.
>
> ... should be left to metamail and the like, which are continuously
> developed and updated.

yes.

> For instance, a possible future feature is to also use proximity of
> word.  Such an extension would affect all of the above modules in that
> they would have to agree.

with proximity, do you mean things like integrating suffixes like -ing
-ed or more geneneral, and still only valid for english, flattening
{has, have, had, ...}, {am, be, was, ...} or even wordnet like
capabilities?  as much as i'd like to see proximity, this would have
to be done for several languages and is thus somewhat bigger than a
simple patch.  if i understand the current making of the lexer, this
would even mean redesign.

otoh, multi-nationalization would be a good thing for ifile.  it
should be a well thought out addition, though.

> No, the point of using ifile in server mode should be that the
> database would not have to be reparsed for each message.  Therefore,
> IMHO, the tokeniser and database interacteraction should be in server
> mode /a priori/, and using ifile as a one-message-only classifier
> should just be a special case (maybe a special option to tell ifile
> "don't hang around, I won't give you anymore").

this sounds reasonable.

clemens

------------------------ Yahoo! Groups Sponsor ---------------------~-->
4 DVDs Free +s&p Join Now
http://us.click.yahoo.com/pt6YBB/NXiEAA/jd3IAA/CefplB/TM
---------------------------------------------------------------------~->

To UNSUBSCRIBE, send an empty message to
[email protected]

ifile web site: http://www.ai.mit.edu/~jrennie/ifile/ 

Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/