Re: Crypt::SSLeay HTTPS_PROXY support

Also, while newing the UserAgent, try the

$ua = LWP::UserAgent->new(env_proxy => 0) ;

This must be it.. damn.. I should have gotten this one earlier...

Regards,
Atul.

On 9/20/06, Atul Saroop <[email protected]> wrote:
>
> Hi Jason:
>
> Can you try another thing then...
>
> try setting
>
> $ua->proxy(['https'], '');
>
> as the null string.. that ensures that libwww necessarily goes through
> Crypt::SSLeay only. I know for sure that this has to be done for
> WWW::Mechanize.. but not sure about LWP.
>
> Also, can you send the whole response content also?
>
> Another pointer .. does your access with simple http take place at all to
> websites like www.google.com?
>
> Regards,
> Atul.
>
>
> On 9/20/06, jason <[email protected]> wrote:
> >
> >  Hi, Atul
> > My proxy is an apache server. I did not set up authentication for proxy.
> >
> >
> > To test whether it is the username/password that causes the issues, I
> > set up proxy authentication. I tested the proxy authentication with firefox
> > browser. It is ok for firefox. I got the https://www.nodework.com web
> > page and here is the server access log.
> > - test [27/Sep/2006:09:48:14 -0400] "CONNECT www.nodeworks.com:443HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT
> > 5.1; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7"
> > - test [27/Sep/2006:09:48:17 -0400] "CONNECT www.nodeworks.com:443HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT
> > 5.1; en-US; rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7"
> >
> > Then I run the code with proxy username/password,
> >   $ENV{HTTPS_PROXY_USERNAME} = "test";
> >   $ENV{HTTPS_PROXY_PASSWORD} = "test";
> > I got the same result.
> > - test [27/Sep/2006:09:48:57 -0400] "CONNECT www.nodeworks.com:443HTTP/1.0" 200 - "-" "libwww-perl/5.805"
> >  - test [27/Sep/2006:09:48:58 -0400] "CONNECT 207.154.17.141:80HTTP/1.0" 403 298 "-" "libwww-perl/5.805"
> >
> > I think the root cause is that the libwww or Crypt::SSLeay sends an
> > extra CONNECT to the proxy server (here 207.154.17.141) itself. This
> > request gets an 403 response from server.
> >
> > Thanks
> >
> > -jason
> >
> >
> >
> > Atul Saroop wrote:
> >
> > Jason:
> >
> > The
> > $ENV{HTTPS_PROXY_USERNAME} = 'jason_username' ;
> > $ENV{HTTPS_PROXY_PASSWORD} = 'jason_password' ;
> > lines are supposed to contain your proxy username and password. Note
> > that you may not explicitly know that your windows network login (in case
> > you work on an NT network) authentication is used by proxy servers under NT
> > authentication.
> >
> > So, try filling in those authentication details in the above mentioned
> > lines.
> >
> > Also, in my version of your code that was working all this while, I
> > emptied out the username and it gave me a 500 also, but a print of
> > $res->content shows that it is a proxy authentication failure.
> >
> > I hope this should help you solve your problem.
> >
> > Regards,
> > Atul.
> >
> >
> > On 9/19/06, jason <[email protected]> wrote:
> > >
> > >  Hi, atul
> > >
> > > I added "use LWP::Debug qw(+);"
> > > Here is output from client
> > > ----------------------
> > > LWP::UserAgent::new: ()
> > > LWP::UserAgent::request: ()
> > > LWP::UserAgent::send_request: GET https://www.nodeworks.com
> > > LWP::UserAgent::_need_proxy: Not proxied
> > > LWP::Protocol::http::request: ()
> > > SSL_connect:before/connect initialization
> > > SSL_connect:SSLv3 write client hello A
> > > SSL3 alert read:fatal:handshake failure
> > > SSL_connect:failed in SSLv3 read server hello A
> > > LWP::UserAgent::request: Simple response: Internal Server Error
> > > 500
> > > --------------------------------------
> > > This is the apache access log
> > > --------------------------------
> > >  - - [26/Sep/2006:08:56:59 -0400] "CONNECT www.nodeworks.com:443HTTP/1.0" 200 - "-" "libwww-perl/5.805"
> > >  - - [26/Sep/2006:08:57:01 -0400] "CONNECT 207.154.17.141:80<http://207.154.17.141/>HTTP/1.0" 403 298 "-" "libwww-perl/5.805"
> > > --------------------------------------------------------
> > >
> > > I added other parameters and used the certificate from Crypt-SSLeay
> > > distribution.
> > > ----------------------
> > >  $ENV{HTTPS_DEBUG} = 2;
> > >   # PROXY_BASIC_AUTH
> > >   $ENV{HTTPS_PROXY_USERNAME} = '';
> > >   $ENV{HTTPS_PROXY_PASSWORD} = '';
> > >
> > >   # DEFAULT SSL VERSION
> > >   $ENV{HTTPS_VERSION} = '3';
> > >
> > >   # CLIENT CERT SUPPORT
> > >   $ENV{HTTPS_CERT_FILE} = 'certs/notacacert.pem';
> > >   $ENV{HTTPS_KEY_FILE}  = 'certs/notacakeynopass.pem';
> > >
> > >   # CA CERT PEER VERIFICATION
> > >   $ENV{HTTPS_CA_FILE}   = 'certs/ca-bundle.crt ';
> > >   $ENV{HTTPS_CA_DIR}    = 'certs/';
> > >
> > >   # DEFAULT SSL VERSION
> > >   $ENV{HTTPS_VERSION} = '3';
> > >   ----------------------------------------
> > > I get the same result.
> > >
> > > You can run the script against my proxy server 207.154.17.141. Maybe
> > > my proxy server is not configured properly. I checked my proxy configuration
> > > using firefox browser.
> > >
> > > Thanks
> > >
> > >
> > > -jason
> > >
> > >
> > > [email protected] wrote:
> > >
> > > Jason:
> > >
> > > Try adding a
> > > use LWP::Debug qw(+);
> > > at the top of the code and see what the perl program reports.
> > >
> > > Btw, your code runs fine on my proxy configuration. I just have to add
> > > a $ENV{HTTPS_PROXY_USERNAME} and a $ENV{HTTPS_PROXY_PASSWORD} with the
> > >
> > > other specifications also.
> > >
> > > Regards,
> > > Atul.
> > >
> > > Jason wrote:
> > >
> > >  Hi,
> > >
> > > I could not get the example script from Crypt::SSLeay run successfully.
> > > Here is the script
> > > ---------------------
> > > #!/usr/bin/perl
> > > # PROXY SUPPORT
> > >  $ENV{HTTPS_PROXY} = '
> > > 207.154.17.141:80 <http://207.154.17.141/>';
> > >  # DEBUGGING SWITCH / LOW LEVEL SSL DIAGNOSTICS
> > >  $ENV{HTTPS_DEBUG} = 2;
> > >
> > >
> > >  use LWP::UserAgent;
> > >  my $ua = new LWP::UserAgent;
> > >  my $req = new HTTP::Request('GET', '
> > > https://www.nodeworks.com');
> > >  my $res = $ua->request($req);
> > >  print $res->code."\n";
> > > ------------------------------------
> > >
> > > I tested the proxy server with Firefox browser. It worked fine.  I
> > >
> > > looked into the access log in the proxy server.  When I run the example
> > > script.
> > >
> > > I got two access log entries
> > > --------------------
> > > - - [25/Sep/2006:20:42:23 -0400] "CONNECT
> > > www.nodeworks.com:443
> > > HTTP/1.0" 200 - "-" "libwww-perl/5.805"
> > > - - [25/Sep/2006:20:42:24 -0400] "CONNECT
> > > 207.154.17.141:80 <http://207.154.17.141/> HTTP/1.0"
> > > 403 298 "-" "libwww-perl/5.805"
> > > -------------------
> > >
> > > Notice that LWP tried to issue an CONNECT method to the proxy server
> > > itself. It got a 403 response for this request.  On the other hand,
> > >
> > > firefox did not send CONNECT method to proxy server itself.
> > > - - [25/Sep/2006:19:33:34 -0400] "CONNECT
> > > www.nodeworks.com:443
> > > HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
> > > rv:
> > > 1.8.0.7) Gecko/20060909 Firefox/1.5.0.7"
> > > - - [25/Sep/2006:19:33:38 -0400] "CONNECT www.nodeworks.com:443
> > >
> > > HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
> > > rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7"
> > >
> > > - - [25/Sep/2006:19:34:32 -0400] "CONNECT www.nodeworks.com:443
> > > HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT
> > > 5.1; en-US;
> > > rv:1.8.0.7) Gecko/20060909 Firefox/1.5.0.7"
> > > - - [25/Sep/2006:19:34:29 -0400] "CONNECT
> > > www.nodeworks.com:443
> > > HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
> > > rv:
> > > 1.8.0.7) Gecko/20060909 Firefox/1.5.0.7"
> > >
> > > Not sure what is trick here.  It seemed that other can get the example
> > > script run properly.  Can anyone give me some help on this?
> > >
> > > Thanks
> > >
> > > -jason
> > >
> > >
> >
>