LWP & UTF-8.
Подбельский В.В. <[email protected]>
| Newsgroups | gmane.comp.lang.perl.modules.lwp |
|---|---|
| Message-ID | <[email protected]> |
Howdy,
While creating a bot for LJ i've encountered with the following error:
"Parsing of undecoded UTF-8 will give garbage when decoding entities at D:/perl.5.8.8/site/lib/LWP/Protocol.pm line 114."
Here's the piece of code, which reproduces the error:
###
my $lj_url = 'http://www.livejournal.com/';
my $cj = HTTP::Cookies->new();
my $ua = LWP::UserAgent->new(agent => 'Howdy?', cookie_jar => $cj);
$ua->default_header('Accept-Language' => 'ru, en',
'Accept-Charset' => 'utf-8;q=1, *;q=0.1',
'Referer' => $lj_url);
print "Getting the login form...\n";
$res = $ua->get($lj_url . 'login.bml?nojs=1');
exit;
It's said in HTML::HeadParser's pod that:
"Note that the HTML::HeadParser might get confused if raw undecoded
UTF-8 is passed to the parse() method. Make sure the strings are
properly decoded before passing them on."
And error seems to be on this line in LWP's Protocol.pm:
114: $parser->parse($$content) or undef($parser);
If i make a change, so the content gets decoded before being parsed:
$parser->parse(decode_utf8($$content)) or undef($parser);
the error message fades away.
Is there's something i'm doing wrong or is it really a bug?
If it helps:
Perl version: 5.8.8
Binary build 817 [257965] provided by ActiveState http://www.ActiveState.com
Built Mar 20 2006 17:54:25
LWP version: $VERSION = "5.805"; # $Id: LWP.pm,v 1.149 2005/12/08 12:06:22 gisle Exp $
Got perl from the activestate, everything is from the pack.
Thanks in advance. (Shalom ebanats!)