Re: LWP ends up with unreadable characters

On Tuesday 25 April 2006 07:17 am, you wrote:
> [email protected] writes:
> > i'm trying to use LWP on:
> >
> > 	http://education.yahoo.com/reference/dict_en_es/spanish/a_1;_ylt=
> > 	AoFfUtrOQo3d1vl10ohvPPb2s8sF
> >
> > when i do :
> >
> >        $ua = LWP::UserAgent->new;
> >        $res1 = $ua->get($url,%header);
> >        my $page = $res1->content;
> >
> > '$page' ends up with unreadable characters. the code works fine for
> > most sites. also, if i fetch the page with 'lynx' i get readable stuff,
> > and a browser's 'view source' function on the page gets a normal
> > result.
> >
> > ideas?
>
> Try to provide a complete program that we can run to reproduce your
> problem.  I certainly get text out when I try to access your URL with
> this program:
>
> #!/usr/bin/perl -w
>
> use strict;
> use LWP::UserAgent;
>
> my $ua = LWP::UserAgent->new;
> my $res =
> $ua->get('http://education.yahoo.com/reference/dict_en_es/spanish/a_1;_ylt=
>AoFfUtrOQo3d1vl10ohvPPb2s8sF'); my $page = $res->content;
>
> print $page;
> __END__
>
> Perhaps you have something interesting in %header that you don't tell us
> about?
>

thanks everyone for your responses. and indeed it has to do with the '%header' 
statement, which runs:

	my %header = (
	    'Keep-Alive' => '300',
	    'Connection' => 'keep-alive',
	    'User-Agent' => 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10)
	        Gecko/20050925 Firefox/1.0.4 (Debian package 1.0.4-2sarge5)',
	    'Pragma' => 'no-cache',
	    'Cache-control' => 'no-cache',
	    'Accept' => 'image/png,*/*;q=0.5',
	    'Accept-Encoding' => 'gzip,deflate',
	    'Accept-Charset' => 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
	    'Accept-Language' => 'en-us,en;q=0.5',
	    'Host' => $host,
	    );
	
if   " 'Accept-Encoding' => 'gzip,deflate' "   is eliminated, the subsequent 
fetch on the website is normal ascii.

tom arnall
north spit, ca