Re: display undeclared iso-charset in UTF environment

 On Sunday, September 10, 2006 at 17:08:01 +0200, Christian Ebert wrote:

    [auto-sensing]
> Incorporated in a layman's way in attached version.

    Problem: The goal of the candidate loop is to guess the origin
charset of the article. This is independant from reader's locale, hence
the "-t utf-8" of my pseudo code (I could have choosen -t ucs-4, or any
*whole* Unicode charset). Removing it broke the goal: The guessing loop
perturbated by current locale may well miss the right candidate.

    Example: In a Latin-1 locale, an UTF-8 article containing an €uro
will fail the UTF-8 (to L1) candidate test, and will fallback wrongly to
CP-1252. Visual result: The 3 chars "â?¬".

    Solution: Readd the "-t utf-8" in the loop. Or "-t UCS-4" if it's
faster (probably is).


> The calls to iconv are a bit expensive and noisy

    Noisy? With the --silent > /dev/null? What do you mean?

    Expensive: Yes. But I just thought about a possible shortcut: If
guessing loop fails, we fallback to last candidate in list, right? So
why test -f last -t UTF-8 at all? Needless. Let's just loop all but last
candidates, and if all fail convert -f last (implicitly -t locale) and
display. Economy: 1 iconv call. And if the list contains 1 charset only,
skip the guessing loop completely.


    [\x80-\xFF]
> is the above range the same as in decimal "[\d128-\d255]" ?

    I suppose yes, though I must admit my S-Lang knoweledge is limited
and rusty: Didn't touch it since 2 years, and haven't yet installed v2.


> Is there any advantage of hex over decimal?

    No: Use the base you're comfortable with.


> though I /should/ be doing supposedly more important stuff

    If you mean a romance, OK: Just why are you still reading this mail?
But if you mean daywork, you'll get more €uros here, trust me. ;-)


Bye!	Alain.
-- 
Erratum: Sorry, I always get this phrase upside down. It should read:
« ...and wait loads of days for some return ».


-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642