Re: Template Toolkit and UTF-8 template files

On 21/10/14 09:07, David Hickman wrote:
>
> *Hi Tom,*
>
> **
>
> *Thanks for your quick reply.  The output is being generated by Perl, 
> the strange thing being that when I use the ENCODING => 'UTF-8' 
> configuration parameter this breaks the output.  If I don’t include it 
> then the £ sign is returned in the correct UTF-8 format.*
>
> **
>
> *I’ll have a look at the debugging options that you suggest and see if 
> I can find anything else out.*
>
>

In that case it sounds like you might be passing around UTF-8 in the 
code, rather than Unicode strings.

I'd suggest starting from the part of the code that sends the output to 
the browser, work backwards from there - usually there's either an 
output layer on the filehandle, or something like Encode::encode('UTF-8' 
=> $output).

If you don't have either, that means you're not really dealing with 
characters, just bytes. One way to test that: with the version of the 
code where the £ sign is being rendered correctly in the browser, try 
removing everything else from the template apart from the £ sign. Then 
see what length($output) returns, where $output is the result of 
rendering the template ($tt->process('template', {...}, \$output) for 
example). If length() returns 1, then it's probably a Unicode string. If 
it's >1, you have something else - presumably UTF-8 bytes.

I'd generally recommend using Unicode inside the app rather than UTF-8, 
otherwise even basic things like substr() are likely to result in 
corrupted output. Decode UTF-8 to Unicode string as soon as you read the 
data, and encode Unicode to UTF-8 just before you write to a 
socket/filehandle.

cheers,

Tom

_______________________________________________
templates mailing list
[email protected]
http://mail.template-toolkit.org/mailman/listinfo/templates