Re: About HTML unicode

At 12:31 am +0800 3/12/04, He Zhiqiang wrote:

>Now i encountered another problem,  there are a few files contains 
>not only one charset but also two or more, for example, file1 
>contains japanese and chinese, if i use open() to  load the data 
>into memory, ord and length etc.. can't correctly work! Perhasp i 
>miss something to encode or decode the data ?
>code:
>#!/usr/bin/perl -w
>use utf8;
>open(FD, "< file1");
>while(<FD>) {
>chomp;
>print "length = ".length($_);
>}
>close FD;
>----------
>length() can not count the correct non-ASCII characters. :(

If the file is in UTF-8, then it may be in any number of _languages_ 
but it uses only one character set -- Unicode.  So far as I know "use 
utf8" is now redundant and ineffectual in Perl.  You will get the 
correct character count (6 characters rather than 18 bytes) by 
opening the file handle as utf-8 as below.

no warnings;
my $f = "/tmp/cjk.txt";
my $text = "\x{56d8}\x{56d9}\x{56da}\x{56db}\x{56dc}\x{56dd}\n";
open F, ">$f";
print F $text; # writes $text to $f as UTF-8
close F;
open F, "<:utf8",  $f;
for (<F>) {
   chomp;
   print "$_  -  Length = " . length() . $/;
}

JD