Output filters, data encoding

André Warnier (tomcat/perl) <[email protected]>
Newsgroups gmane.comp.apache.mod-perl
Message-ID <[email protected]>
Hi.
I'm writing a new PerlOutputFilter, stream version.
I have written several working ones before, so I know the general scheme.
But in this latest filter, I have a problem with the data encoding, which I did not 
encounter previously.
I did not find an answer in the on-line mod_perl documentation, which seems silent about 
any encoding of the data that one might read in a filter.
(I also tried the WWW, without any more success).

To read the response data coming from upstream, I use the standard $f->read(), as in

	while (my $sz = $f->read(my $buffer, BUFF_LEN)) {
..

and then I need to pass this data to another module for processing (Template::Toolkit).
To make a long story short, Template::Toolkit misinterprets the data I'm sending to it, 
because this data /is/ actually UTF-8, but apparently not marked so internally by the 
$f->read(). So TT2 re-encodes it, leading to double UTF-8 encoding.

My question is : can I - and how -, set the filehandle that corresponds to the $f->read(), 
to a UTF-8 layer ?
I have tried

line 155: binmode($f,'encoding:(UTF-8)');

and that triggers an error :
  Not a GLOB reference at (my filter) line 155.\n
)

Or do I need to read the data 'as is', and separately do an

  $decoded_buffer = decode('UTF-8', $buffer);

?

Note 1 : I can of course do the above decode(), but it seems more elegant to just insert 
the I/O layer.

Note 2 : I know that the pre-filter data is UTF-8 (or at least I choose to believe that it 
is), because I check $f->r->content_type(), which returns "text/html;charset=UTF-8".
Also, the filter otherwise works as expected, and I get the expected output in the 
browser, except that e.g. "München" becomes "München".

Thanks in advance for any tips
André
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.