Re: display undeclared iso-charset in UTF environment

 On Friday, September 8, 2006 at 15:46:58 +0200, Christian Ebert wrote:

> But I want to see Euros, loads of 'em!

    Quick and easy: Just send 5€ to each one on the list, subscribe
yourself on top, and wait some days for loads of return. ;-)

    It's a false friend: Assuming unlabelled means Latin-9 would, in
most cases, wrongly show €uros where there are not intended, and show
question marks where €uros really are. Of course if an article happens
to be L9, then you get rich. But it's a minority case. The balance is
completely for 1252.


> * Alain Bench on Friday, September 08, 2006 at 02:55:28 +0200:
>> some auto-sensing between UTF-8 and CP-1252

    Pseudo code:

| guess_list="utf-8 cp1252"
| for candidate in $guess_list; do
|   iconv -s -f $candidate -t utf-8  article > /dev/null
|   if [ $? == 0 ]; then
|     break
|   fi
| done
| iconv -c -f $candidate  article

    If all fails, fallbacks to last in list. And Japanese guys would use
something as "iso-2022-jp euc-jp shift_jis utf-8" guess_list. The nice
thing with Japanese is that those charsets are very different, thus
easely distinguishable; Our 8bits charsets are not... No way to so guess
L1 or L2 or 125x or L9 or...


>> grep for any char above 0x80?
> how do I do that in s-lang?

    string_match (rawart, "[\x80-\xFF]", 1)


>> better way to get current CODESET
> I don't know it.

    Me neither. Hum... in the absence of "nl_langinfo(CODESET)" and
"locale charmap", I would probably call locale_charset.c, attached. It
has the advantage to always give a canonicalized charset name, while
nl_langinfo on some platforms may give st*pid names. Like HP-UX giving
"utf8", and by default refusing the standard "utf-8" with dash.


Bye!	Alain.
-- 
When you want to reply to a mailing list, please avoid doing so from a
digest. This often builds incorrect references and breaks threads.

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642

/* Prints the portable name for the current locale's charset.
 * Build with gcc -o locale_charset locale_charset.c -lcharset
 * to catch /usr/local/lib/libcharset.so.1
 * Or -liconv to catch /usr/local/lib/libiconv.so.2
 * Or -lintl  to catch /usr/local/lib/libintl.so.2
 */

#include <stdio.h>
#include <stdlib.h>
#include <locale.h>
#include <localcharset.h>

int main ()
{
  setlocale(LC_ALL, "");
  printf("%s\n", locale_charset());
  exit(0);
}