Re: using cookie in google scholar from perl

On Nov 9, 5:16 pm, [email protected] (Francois) wrote:
> I would like to use bibTex data in a perl script from google scholar.
> These data are controlled in google scholar with the preference panel
> wich use cookies to remember the user's settings. bibTex are not
> displayed with the default settings.
>
> Using the perl script I receive an empty page with the following urlhttp://scholar.google.com/scholar.bib?num=100&hl=en&lr=&q=info:9Y4Rq3...
> despite I try to set the same cookie as the cookie sets in my browser
> when "display bibTex links" is on.
> The same url give what I want if I send it in my browser with the
> correct preferences.
>
> My code goes like this:
>  my $url1="http://scholar.google.com/scholar.bib?
> num=100&hl=en&lr=&q=info:9Y4Rq3zllPUJ:scholar.google.com/
> &output=citation&oe=ASCII&oi=citation";
>  my $ua = LWP::UserAgent->new;
>  $ua->agent('Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.8.1.9)
> Gecko/20071025 Firefox/2.0.0.9');
>  my $c = HTTP::Cookies::Netscape->new(file=>"cookies.txt",
> autosave=>"1");
>  my $h = HTTP::Headers->new(
>             Accept => "text/xml,application/xml,application/xhtml+xml,text/
> html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5",
>             Host => "www.unifr.ch",
>    );
> my $req =HTTP::Request->new(GET=>$url1,$h);
>  $c->add_cookie_header($req);
>  $ua->cookie_jar($c);
>  my $res =$ua->request($req);
>  die ($res->status_line) unless ($res->is_success);
> print $res->as_string;
>
> The output is
> HTTP/1.1 200 OK
> Cache-Control: private
> Connection: Close
> Date: Fri, 09 Nov 2007 16:04:42 GMT
> Server: GWS/2.1
> Content-Length: 0
> Content-Type: text/plain; charset=ISO-8859-1
> Client-Date: Fri, 09 Nov 2007 16:04:44 GMT
> Client-Peer: 66.102.1.99:80
> Client-Response-Num: 1
> Set-Cookie:
> PREF=ID=8d4ee7c002f1dc72:TM=1194624282:LM=1194624282:S=vBeATmmCJPdAHCQy;
> expires=Sun, 17-Jan-2038 19:14:07 GMT; path=/; domain=.google.com
> Set-Cookie: GSP=ID=8d4ee7c002f1dc72; expires=Sun, 17-Jan-2038 19:14:07
> GMT; path=/; domain=.scholar.unifr.ch
>
> The relevant line in my cookies file use by the script is
> .scholar.google.com     TRUE    /       FALSE   2147368448      GSP
> ID=c77566a14ed11a16:IN=7e6cc990821af63+b8acc395c41ea61f:CF=4
>
> The cookie data from my browser are
> name: GSP
> Value: ID=c77566a14ed11a16:IN=7e6cc990821af63+b8acc395c41ea61f:CF=4
> Domain: .scholar.google.com
> Path: /
> ...
>
> Well the cookie from the script output seems to be different form the
> cookie I want to send. This is maybe the reason of my empty page but I
> can't understand why that cookie is changed ...
>
> Thanks for any help !
>
> Francois

Well, one needs luck some time: if I replace the domain
scholar.google.com by the domain my pc is in (.unifr.ch) the script
works ... I have read some more on cookies but can't understand how
the server at google scholar can find the right cookie when the domain
is changed...
Francois