clearing the utf8 flag

[email protected] (Paul Bijnens)
Newsgroups perl.unicode
Message-ID <[email protected]>
I have a program that reads and writes (among others) strings that
should be utf8 encoded.  I say "should", because somewhere deep
inside the dark corners of that program, sometimes, the utf8 flag on
a string is lost. (I'm still investigating where, tips to attack
such a problem are welcome.)

When writing the string, the program clears the utf8 flag
and writes a simple string of octets using:

     $s = encode("utf8", $s) if $s =~ /[^\x00-\x7f]/;
     $n = length($s);   # yes, we need length in bytes
     ...
     print $s;

Why would someone test for pure 7-bit strings instead of:

     $s = encode("utf8", $s) if Encode::is_utf8($s);

which seems superior to avoid double utf8 encodings,
should the utf8-flag be lost.  And it's faster.

Or even simply:     Encode::_utf8_off($s)

The problem is that I'm usually wrong.  Am I this time?
Am I missing something?  Or do I need more coffee?


-- 
Paul Bijnens, Xplanation                            Tel  +32 16 397.511
Technologielaan 21 bus 2, B-3001 Leuven, BELGIUM    Fax  +32 16 397.512
http://www.xplanation.com/          email:  [email protected]
***********************************************************************
* I think I've got the hang of it now:  exit, ^D, ^C, ^\, ^Z, ^Q, F6, *
* quit,  ZZ, :q, :q!,  M-Z, ^X^C,  logoff, logout, close, bye,  /bye, *
* stop, end, F3, ~., ^]c, +++ ATH, disconnect, halt,  abort,  hangup, *
* PF4, F20, ^X^X, :D::D, KJOB, F14-f-e, F8-e,  kill -1 $$,  shutdown, *
* kill -9 1,  Alt-F4,  Ctrl-Alt-Del,  AltGr-NumLock,  Stop-A,  ...    *
* ...  "Are you sure?"  ...   YES   ...   Phew ...   I'm out          *
***********************************************************************
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.