Dangerous locale patch to 1.1.3

Hi,

There has previously (before I started using ifile) been discussion about the 
tokenization of non-ascii messages.  Currently, ifile uses the "C" locale for 
tokenization, which means that tokens are basically ascii alpha characters.  But 
they are that *consistently*.

I took the liberty of studying the lexer for a few moments, and concluded that 
"isalpha" is used to determine whether a character belongs in a token or not.  I 
decided to open this particular can of worms by adding locale support.  To 
1.1.3, as I've not bothered to upgrade yet. :)

Patch follows:
--------------------------------------------------------------
*** ifile.c     2002-10-09 14:42:13+03  1.1.3
--- ifile.c     2002-10-09 14:44:54+03
***************
*** 22,27 ****
--- 22,28 ----
   #include <sys/sem.h>

   #include <time.h>
+ #include <locale.h>
   #include <ifile.h>        /* standard ifile library */

   #define SEMKEY  10439838
***************
*** 94,99 ****
--- 95,104 ----
     for (i=0; i < args.num_files; i++)
       ifile_verbosify(ifile_verbose, "file #%d: %s\n", i,
                     EXT_ARRAY_GET(args.file, char *, i));
+
+   /* Set locale for comparisons */
+   setlocale(LC_CTYPE, "");
+   setlocale(LC_COLLATE, "");

     /* Get home directory */
     home_dir = getenv("HOME");
--------------------------------------------------------------
(LC_COLLATE is probably unnecessary above)

Applying this patch does exactly what I inteded it to do, in my environment. 
However, after this patch ifile probably no longer behaves as *you* would intend 
it to.

Caveats:
- Now "LC_CTYPE" is used to determine what is "isalpha()". This depends on your 
system configuration, so you better be right.  I'd recommend setting it 
explicitly.  Every time.  All the time.
- Using your system locale to determine how an incoming email should be 
tokenized is not really the right way(tm) to do stuff.  OTOH, what is? ascii :)?
- Using different LC_CTYPE values for different runs of ifile will do strange 
stuff to your .idata.
- I didn't check if ifile uses collation for comparison.  If it does, setting 
LC_COLLATE may do strange, but sometimes smart, stuff to matching.
- Consider "hard-coding" the LC_CTYPE value into .idata instead?

As you can see, this patch should not be taken lightly.  I am considering trying 
it out for my own email (which contains quite a lot of 8bit-encoded characters 
in non-spam email) using the locale "sv".  If the email uses a 7-bit encoding, 
ifile reverts to its ordinary behavior [under the "sv" locale].

I send this patch more as something to discuss than as a serious submission.  An 
equally valid discussion would be "is isalpha() the right function?"

Cheers,
Camillo