Re: Problem with Text::Template 1.43 and Frontier::RPC2 and Japanese text

Hi Kevin,

Thanks for your response.  I tried to reply to your address [email protected]
but there were problems reaching your address.  Anyways, here is an update
to the problem. 

At the time I reported the problem I did not use either utf or bytes pragma.
Actually I didn't know what they were since I am new to 5.6.1 and Unicode,
have been working with 5.005 in the past.

I tried each of "use utf8", "no utf8", "use bytes" and "no bytes", but each
one still resulted in corrupted text.  I am not really clear on what exactly
these do. I assume "use utf8" tells Perl to specifically handle all characters
as UTF-8 internally.  But i would think that "no utf8" would be the same as
"use bytes".  

In any case I had no luck getting Text::Template with type FILE to work with
any of these pragma settings.  To simplify things a bit I tried to use type
STRING and for the source string a Shift-JIS encoded japanese string.  Then
I tried the various pragmas again and found that there was no corruption when
I had "use utf8" enabled.  I am a bit baffled as to why it would work with
the STRING but not the FILE type in Text::Template.

Since the STRING type doesn't really help me and I couldn't get it to work
with the FILE type I decided to try it on a different version of Perl.  I
installed Perl 5.005 and then re-tested, there is no corruption!  So the
problem has to do with how Perl 5.6.1 handles my Shift-JIS characters.  I would
think that I would want to turn off UTF-8 handling in Perl 5.6.1 so that it
doesn't try to handle the Shift-JIS as UTF-8.  

Using Perl 5.005 is a workaround, but eventually I would like my code to work
in Perl 5.6.1 and later.  So I would still be interested in any help anybody
can offer to solve this problem.

>Hi Peter,
>
>Are you using the utf or bytes pragma(s)? I couldn't reproduce this,
>but would be happy to try if you wanted to send me your template. When
>I transmit data, I try to use UTF-8.. but that may not suit your needs.
>
>Cheers,
>Kevin
>
>On Mon, Jun 10, 2002 at 08:48:37PM +0900, Peter Chow ([email protected]) said something similar to:
>> I am using Perl 5.6.1 on Debian Linux with Text::Template 1.43
>> and Frontier::RPC2 0.06. 
>>
>> My template file is in Japanese Shift-JIS encoding.  I am writing
>> a XML-RPC server using Frontier::RPC2 as a CGI-based type running
>> under Apache.  The method I am calling is supposed to take a few
>> parameters, populate the values into the template with Text::Template
>> and then send out an email. 
>>
>> What I have found is that the Japanese characters in my template
>> gets corrupted in the output of the email when Text::Template runs
>> with the data received from the XML-RPC client.  Initially I thought
>> the problem was with Text::Template, but I tested calling my
>> Text::Template code directly and there was no character corruption. 
>> The problem only seems to occur when it is called with the data
>> received from the XML-RPC client.
>>
>> I thought the problem was with the XML-RPC module, so I tried using
>> RPC::XML instead of Frontier::RPC2, but the same corruption occurs.
>> I also tried using the Template module instead of Text::Template,
>> but still the same corruption. 
>>
>> The parameters values I am sending between the XML-RPC client and
>> server are currently just ASCII text.  The terminal I am using to
>> view the results is in Shift-JIS. 
>>
>> Is there some Unicode conversion I have to do somewhere?
>> Any suggestions would be greatly appreciated.