Re: display undeclared iso-charset in UTF environment
Alain Bench <[email protected]>
| Newsgroups | gmane.network.slrn.user |
|---|---|
| Message-ID | <[email protected]> |
Hi Christian,
On Thursday, September 7, 2006 at 16:59:43 +0200, Christian Ebert wrote:
> %% charset that will be fixed if not declared
> private variable fix_charset = "ISO-8859-15";
Assuming that unlabelled articles are in Latin-9 seems not a good
choice. Experience shows that, for westerners, the vast majority of such
articles really are in CP-1252 or Latin-1. A small minority is in UTF-8.
And peanuts for all other charsets, including Latin-9.
So, for westerners, the best choice is probably either a fixed
CP-1252, or some auto-sensing between UTF-8 and CP-1252 (I mean a
Vim-like fileencodings="ucs-bom:utf-8:latin1").
Then there are other populations, and a Chinese or Polish guy
probably prefers some other default charset (or auto-sensing list).
Should be configurable.
> private variable hibits = "\\C[ßäöüáéíàèçôœëñåøµ©®¢£€]";
I don't understand the interest of grepping specifically those
chars. Why not just convert all unlabelled articles? Or grep for any
char above 0x80?
> % check whether we are in an UTF environment
> private variable utf_env = getenv ("LANG");
> if (NULL == utf_env)
> utf_env = getenv ("LC_CTYPE");
> if (NULL != utf_env and 0 == string_match (utf_env, "\\Cutf-8", 1))
> utf_env = NULL;
Is there no better way to get current CODESET then parsing env vars?
It seems to me you get parsing reversed: The active value is LC_ALL if
defined, otherwise LC_CTYPE, otherwise LANG. Finally, once you have the
current CODESET, why limit the fixing to UTF-8? It could be also usefull
in other locales.
> iconv = sprintf ("/sw/bin/iconv -c -f %s -t UTF-8", fix_charset);
| $ /sw/bin/iconv
| bash: /sw/bin/iconv: No such file or directory
BTW you could simply remove "-t UTF-8" so iconv itself converts to
the current locale's charset.
Bye! Alain.
--
And now, ladies and gentlemen, Silly Footer announces:
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642