Unicode characters

Hi Perl Gurus,

I am using functions decode_entities() & decode_utf8() to decode the html
codes and UTF (latin characters) respectively. (from module use Encode).
The functions which i mentioned above works upto ASCII Decimals 255 and
above that it works differently.
This is the URL i referred to know the list of html codes and latin
characters [http://www.ascii.cl/htmlcodes.htm].

Attached the sample script.
 Where i give the input values which i got from a XML SOAP response for
decoding (The SOAP response doesn't gives the HTML numbers or HTML codes as
in the above said URL list).

The script gives me what i expected for array values from arr_val[0] to
arr_val[4] ((i.e) upto ASCII Decimals range 0-255)
but for arr_val[5] (which have ASCII Decimals greater than 255) the decoded
values are different.

Given the list of array variable values and their expected values. The
decoding fails for array variable arr_val[5].
Similarly i would need to encode also.

$arr_val[0] = '!"#$%&'()*+,-./   0123456789:;<=>?' ;
             expected decoded values -- !"#$%&'()*+,-./ 0123456789:;<=>?

$arr_val[1] =
'@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~' ;
             expected decoded values --
@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~

$arr_val[2] =
'Ã~@Ã~AÃ~BÃ~CÃ~DÃ~EÃ~FÃ~GÃ~HÃ~IÃ~JÃ~KÃ~LÃ~MÃ~NÃ~OÃ~PÃ~QÃ~RÃ~SÃ~TÃ~UÃ~VÃ~WÃ~XÃ~YÃ~ZÃ~[Ã~\Ã~]Ã~^Ã~_'
;
             expected decoded values -- ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞß

$arr_val[3] = 'Ã|
áâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ' ;
             expected decoded values -- àáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿ

$arr_val[4] =
'¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿' ;
             expected decoded values -- ¡¢£¤¥¦§¨©ª«¬­®¯°±²³´µ¶·¸¹º»¼½¾¿

$arr_val[5] = 'others Å~RÅ~SÅ|
šŸÆ~Râ~@~Sâ~@~Tâ~@~Xâ~@~Yâ~@~Zâ~@~\â~@~]â~@~^â~@| â~@¡â~@¢â~@¦' ;
             expected decoded values -- others ŒœŠšŸƒ–—‘’‚“”„†‡•…‰€™

  Could you please help to know what i am missing or doing wrong.
I'll greatly appreciate the help.

Thanks
Saravanan Balaji.

#!/ms/dist/perl5/bin/perl5.8 -I ../   
 
use MSDW::Version   
'HTML-Parser'        => '3.56', # HTML::Entities may be used by HTTP::Response   
;   
 
use Encode;
use strict;   
use Data::Dumper;   
use HTML::Entities;
use HTML::Entities qw(encode_entities_numeric);

 
my @arr_val = ();
$arr_val[0] = '!"#$%&'()*+,-./   0123456789:;<=>?' ;
$arr_val[1] = '@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~' ;
$arr_val[2] = '懒旅呐魄壬仕掏蜗醒矣哉肿刭谯茌捱' ;
$arr_val[3] = '噌忏溴骁栝觌祉铒瘃蝮趱鲼' ; 
$arr_val[4] = '、￥ウЖ┆辈炒刀犯购患骄' ; 
$arr_val[5] = 'others ' ;

my $bcp_in_file = "/tmp/testbcp.in" ;
my $out_str = "" ;
if (!(open ( TEMP_OUT, ">$bcp_in_file" ) ))  ##REVISIT##  
{
	print "Error: cannot open the file \n";
}

foreach my $temp_var (@arr_val)
{
    print "\nProcessing value 		[$temp_var] \n";

    decode_entities($temp_var) ;
    print "After HTML decode [$temp_var] \n";
    my $temp_var2  = decode_utf8($temp_var);
    print "After UTF8 decode [$temp_var2] \n\n";
    print TEMP_OUT $temp_var2 ;   

    #my $temp_var3 = encode_utf8($temp_var2);
    #print "After UTF8 encode [$temp_var3] \n";
    #my $temp_var4 = encode_entities($temp_var3, '"&<>' );
    #print "After HTML encode [$temp_var4] \n";
}
1;   
############ End of Script #################