Re: enabling utf8 encoding for sax parer XML::SAX::Expat

On Wed, Apr 29, 2009 at 11:19 AM, Michael Ludwig <[email protected]> wrote:

> eyal edri schrieb:
>
> > can it be that since i'm using LWP::UserAgent , perl somehow changes
> > the encoding while d/l?
>
> Just ran into this myself. I think LWP does that, not Perl. And it may
> even be the correct thing to do, given that what you send over the wire
> are octets, not strings. See the note in the perldoc for HTTP::Request:
>

So you think that if i'll save the $resp->content in memory and then save it
to file
using >:encoding(utf-8) it will fix the wrong file encoding ? (assuming lwp
does change the file encoding while d/l?)

if it's true, i then encounter a different problem (which is really
hypothetical), when d/l a really big file,
i am not sure it can fit in memory before saving to disk.


>
> | $r->content( $bytes )
> |
> | Note that the content should be a string of bytes. Strings in perl can
> | contain characters outside the range of a byte. The Encode module can
> | be used to turn such strings into a string of bytes.
>
> So it is documented, just rather concisely, not very verbosely. The
> relevant concepts are explained elsewhere. See perldoc:
>
> * perluniintro
> * perlunitut
> * perlunicode
> * perlunifaq
>
> > $userAgent->request($request, $file) ?
> >
> > should i add any sort of HTTP header with encoding like the ones
> > returned in HTTP responses?
>
> You should indicate the encoding in the Content-Type, yes.
>
> The issue here, however, is unrelated to that. Use the Encode module
> to convert your characters to octets. This yields a string of octets,
> rather than a string of characters. In the following sample snipped,
> I convert the output of an XSL transformation, which is a string of
> UTF-8 characters, into a string of octets, which is what needs to go
> over the wire. This string of octets has the UTF-8 flag unset, which
> can be checked by using Encode::is_utf8().
>
>   my $octets = encode( 'utf8', $ss->output_string( $result));
>   my $req = HTTP::Request->new( POST => $url);
>
>   $req->content_type('text/xml; charset=utf-8');
>   $req->content( $cont );
>   my $res = $ua->request( $req);
>   if ( ! $res->is_success ) {
>       print STDERR $res->status_line, "\n";
>    }
>
> Michael Ludwig
> _______________________________________________
> Perl-XML mailing list
> [email protected]
> To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
>



-- 
Eyal Edri

_______________________________________________
Perl-XML mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs