Re: enabling utf8 encoding for sax parer XML::SAX::Expat

here's what i found so far:


   1. after sniffing the request and response to the site with the xml, i
   found that the orginal file does NOT have the magic numbers, so lwp doesnt
   alter it while d/l.
   2. i also think that the file is indeed in UTF8, but missing the magic
   numbers for some reason.
   3. i tried this as well:   $parser->parse_uri($srcFile, *ProtocolEncoding
   => 'UTF-8'*), also didn't help.
   4. Is it true to say that xml parsers in perl can only parse files
   correctly if they have magic numbers?
   5. i come to a conclusion like you guys said from the begining, to
   convert the file encoding to UTF-8, using Encode.
   i guess i'll have to read all the file to string in memory and then
   convert it right?

thanks for all the info, i've learned alot.


On Wed, Apr 29, 2009 at 5:09 PM, Michael Ludwig <[email protected]> wrote:

> eyal edri schrieb:
> > On Wed, Apr 29, 2009 at 11:19 AM, Michael Ludwig <[email protected]>
> > wrote:
> >
> >> eyal edri schrieb:
> >>
> >>> can it be that since i'm using LWP::UserAgent , perl somehow changes
> >>> the encoding while d/l?
> >> Just ran into this myself. I think LWP does that, not Perl. And it
> >> may even be the correct thing to do, given that what you send over
> >> the wire are octets, not strings. See the note in the perldoc for
> >> HTTP::Request:
>
> As Robin has suggested, I mistakenly identified the issue you're having
> with another one I had just ran into myself which had to do with POSTing
> a document, which is probably unrelated to your problem.
>
> > So you think that if i'll save the $resp->content in memory and then
> > save it to file using >:encoding(utf-8) it will fix the wrong file
> > encoding ? (assuming lwp does change the file encoding while d/l?)
>
> No, I don't think that will help. The correct encoding should be
> specified in the Content-Type header of the response, like this:
>
> Content-Type: text/xml; charset=utf-8
>
> You can access the decoded string of bytes using one of:
>
> $r->decoded_content( %options )    # HTTP::Response
> $mess->decoded_content( %options ) # HTTP::Message, base class
>
> But this applies the information in the response headers, and cannot
> magically divine the correct encoding.
>
> If both Content-Type header and XML declaration are either missing or
> incorrect, this is a broken service, and to repair it, you have to
> resort to what Robin suggested, and failing that, apply low-level
> techniques, but I would rather complain to the service provider.
>
> Michael Ludwig
> _______________________________________________
> Perl-XML mailing list
> [email protected]
> To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
>



-- 
Eyal Edri

_______________________________________________
Perl-XML mailing list
[email protected]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs