Re: Problem with -8 option

> > One question remains: as can be seen from my database dump, 8 bit
> > characters are not converted to lowercase. For example, the German
> > uppercase Umlaut characters ÄÖÜ remain like that instead of using the
> > lowercase variants äöü. There are other 8 bit uppercase characters in
> > the latin1 character set that could require similar lowercasing (e.g.,
> > some French characters with accents).
> > 
> > Would it be a big change to support such non-ASCII lowercasing? Or is
> > the current implementation a design philosophy to stay more or less
> > independent of the character encoding?
> 
> I think this must have something to do with the LANG environment 
> variable.  I ran a test tokenizing your email with and without the LANG 
> environment variable and seemed to get the correct results.
> 
> $ ./spamprobe -8 tokenize ~/sampleemail.txt
>   ... characters were upper case
> $ env LANG=de_DE.iso88591 ./spamprobe -8 tokenize ~/sampleemail.txt
>   ... characters were lower case
> 
> Dmitry Koteroff posted to sourceforge that procmail does not pass the 
> value of the LANG environment variable to SP.  Maybe that's the cause of 
> your problem?
> 
> <http://sourceforge.net/tracker/index.php?func=detail&aid=905363&group_id=61201&atid=496457>

Yes, setting the LANG variable inside my procmailrc solved the problem.

Thanks for the hint!

> Really SP should determing the right character encoding based on the 
> headers in each email so that it can properly handle multilingual email. 
>   That's something I'd like to add but for now it's best just to ensure 
> that LANG is set appropriately for your own primary language.

This might be something for the future. For now, setting the LANG
variable is fine (at least for me). :)


Greetings,
Thomas.


-------------------------------------------------------
SF.Net email is sponsored by: Discover Easy Linux Migration Strategies
from IBM. Find simple to follow Roadmaps, straightforward articles,
informative Webcasts and more! Get everything you need to get up to
speed, fast. http://ads.osdn.com/?ad_idt77&alloc_id492&op=click