Re: Crypt::SSLeay HTTPS_PROXY support
jason <[email protected]>
| Newsgroups | gmane.comp.lang.perl.modules.lwp |
|---|---|
| Message-ID | <[email protected]> |
Hi, Atul
Still it does not work.
--------------My code
#!/usr/bin/perl
# PROXY SUPPORT
$ENV{HTTPS_PROXY} = '207.154.17.141:80';
#$ENV{HTTPS_PROXY} = 'localhost:8080';
# DEBUGGING SWITCH / LOW LEVEL SSL DIAGNOSTICS
$ENV{HTTPS_DEBUG} = 2;
# PROXY_BASIC_AUTH
$ENV{HTTPS_PROXY_USERNAME} = "test";
$ENV{HTTPS_PROXY_PASSWORD} = "test";
# DEFAULT SSL VERSION
$ENV{HTTPS_VERSION} = '3';
# CLIENT CERT SUPPORT
$ENV{HTTPS_CERT_FILE} = 'certs/notacacert.pem';
$ENV{HTTPS_KEY_FILE} = 'certs/notacakeynopass.pem';
# CA CERT PEER VERIFICATION
$ENV{HTTPS_CA_FILE} = 'certs/ca-bundle.crt';
$ENV{HTTPS_CA_DIR} = 'certs/';
# DEFAULT SSL VERSION
$ENV{HTTPS_VERSION} = '3';
use LWP::Debug qw(+);
use LWP::UserAgent;
my $ua = LWP::UserAgent->new(env_proxy => 0) ;
$ua->proxy(['https'], '');
my $req = new HTTP::Request('GET', 'http://www.google.com');
my $res = $ua->request($req);
print $res->code."\n";
my $req = new HTTP::Request('GET', 'https://www.nodeworks.com');
$res = $ua->request($req);
print $res->code."\n";
-----------------------------
This is debug output
-----------------------
LWP::UserAgent::new: ()
LWP::UserAgent::proxy: ARRAY(0x818e314)
LWP::UserAgent::proxy: https
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://www.google.com
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 230 bytes
LWP::UserAgent::request: Simple response: Found
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://www.google.com/intl/zh-CN/
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 693 bytes
LWP::Protocol::collect: read 406 bytes
LWP::Protocol::collect: read 1430 bytes
LWP::Protocol::collect: read 50 bytes
LWP::UserAgent::request: Simple response: OK
200
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET https://www.nodeworks.com
LWP::UserAgent::_need_proxy: Not proxied
LWP::Protocol::http::request: ()
SSL_connect:before/connect initialization
SSL_connect:SSLv3 write client hello A
SSL3 alert write:fatal:handshake failure
SSL_connect:error in SSLv3 read server hello A
LWP::UserAgent::request: Simple response: Internal Server Error
500
-----------------------------------
Thanks for the help
-jason
Atul Saroop wrote:
> Also, while newing the UserAgent, try the
>
> $ua = LWP::UserAgent->new(env_proxy => 0) ;
>
> This must be it.. damn.. I should have gotten this one earlier...
>
> Regards,
> Atul.
>
> On 9/20/06, *Atul Saroop* <[email protected]
> <mailto:[email protected]>> wrote:
>
> Hi Jason:
>
> Can you try another thing then...
>
> try setting
>
> $ua->proxy(['https'], '');
>
> as the null string.. that ensures that libwww necessarily goes
> through Crypt::SSLeay only. I know for sure that this has to be
> done for WWW::Mechanize.. but not sure about LWP.
>
> Also, can you send the whole response content also?
>
> Another pointer .. does your access with simple http take place at
> all to websites like www.google.com <http://www.google.com>?
>
> Regards,
> Atul.
>
>
> On 9/20/06, *jason* < [email protected]
> <mailto:[email protected]>> wrote:
>
> Hi, Atul
> My proxy is an apache server. I did not set up authentication
> for proxy.
>
> To test whether it is the username/password that causes the
> issues, I set up proxy authentication. I tested the proxy
> authentication with firefox browser. It is ok for firefox. I
> got the https://www.nodework.com web page and here is the
> server access log.
> - test [27/Sep/2006:09:48:14 -0400] "CONNECT
> www.nodeworks.com:443 <http://www.nodeworks.com:443> HTTP/1.1"
> 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
> rv:1.8.0.7 <http://1.8.0.7>) Gecko/20060909 Firefox/1.5.0.7"
> - test [27/Sep/2006:09:48:17 -0400] "CONNECT
> www.nodeworks.com:443 <http://www.nodeworks.com:443> HTTP/1.1"
> 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
> rv:1.8.0.7 <http://1.8.0.7>) Gecko/20060909 Firefox/1.5.0.7"
>
> Then I run the code with proxy username/password,
> $ENV{HTTPS_PROXY_USERNAME} = "test";
> $ENV{HTTPS_PROXY_PASSWORD} = "test";
> I got the same result.
> - test [27/Sep/2006:09:48:57 -0400] "CONNECT
> www.nodeworks.com:443 <http://www.nodeworks.com:443> HTTP/1.0"
> 200 - "-" "libwww-perl/5.805"
> - test [27/Sep/2006:09:48:58 -0400] "CONNECT
> 207.154.17.141:80 <http://207.154.17.141:80> HTTP/1.0" 403 298
> "-" "libwww-perl/5.805"
>
> I think the root cause is that the libwww or Crypt::SSLeay
> sends an extra CONNECT to the proxy server (here
> 207.154.17.141 <http://207.154.17.141>) itself. This request
> gets an 403 response from server.
>
> Thanks
>
> -jason
>
>
>
> Atul Saroop wrote:
>> Jason:
>>
>> The
>> $ENV{HTTPS_PROXY_USERNAME} = 'jason_username' ;
>> $ENV{HTTPS_PROXY_PASSWORD} = 'jason_password' ;
>> lines are supposed to contain your proxy username and
>> password. Note that you may not explicitly know that your
>> windows network login (in case you work on an NT network)
>> authentication is used by proxy servers under NT authentication.
>>
>> So, try filling in those authentication details in the above
>> mentioned lines.
>>
>> Also, in my version of your code that was working all this
>> while, I emptied out the username and it gave me a 500 also,
>> but a print of $res->content shows that it is a proxy
>> authentication failure.
>>
>> I hope this should help you solve your problem.
>>
>> Regards,
>> Atul.
>>
>>
>> On 9/19/06, *jason* <[email protected]
>> <mailto:[email protected]>> wrote:
>>
>> Hi, atul
>>
>> I added "use LWP::Debug qw(+);"
>> Here is output from client
>> ----------------------
>> LWP::UserAgent::new: ()
>> LWP::UserAgent::request: ()
>> LWP::UserAgent::send_request: GET
>> https://www.nodeworks.com <https://www.nodeworks.com/>
>> LWP::UserAgent::_need_proxy: Not proxied
>> LWP::Protocol::http::request: ()
>> SSL_connect:before/connect initialization
>> SSL_connect:SSLv3 write client hello A
>> SSL3 alert read:fatal:handshake failure
>> SSL_connect:failed in SSLv3 read server hello A
>> LWP::UserAgent::request: Simple response: Internal Server
>> Error
>> 500
>> --------------------------------------
>> This is the apache access log
>> --------------------------------
>> - - [26/Sep/2006:08:56:59 -0400] "CONNECT
>> www.nodeworks.com:443 <http://www.nodeworks.com:443/>
>> HTTP/1.0" 200 - "-" "libwww-perl/5.805"
>> - - [26/Sep/2006:08:57:01 -0400] "CONNECT
>> 207.154.17.141:80 <http://207.154.17.141/> HTTP/1.0" 403
>> 298 "-" "libwww-perl/5.805"
>> --------------------------------------------------------
>>
>> I added other parameters and used the certificate from
>> Crypt-SSLeay distribution.
>> ----------------------
>> $ENV{HTTPS_DEBUG} = 2;
>> # PROXY_BASIC_AUTH
>> $ENV{HTTPS_PROXY_USERNAME} = '';
>> $ENV{HTTPS_PROXY_PASSWORD} = '';
>>
>> # DEFAULT SSL VERSION
>> $ENV{HTTPS_VERSION} = '3';
>>
>> # CLIENT CERT SUPPORT
>> $ENV{HTTPS_CERT_FILE} = 'certs/notacacert.pem';
>> $ENV{HTTPS_KEY_FILE} = 'certs/notacakeynopass.pem';
>>
>> # CA CERT PEER VERIFICATION
>> $ENV{HTTPS_CA_FILE} = 'certs/ca-bundle.crt ';
>> $ENV{HTTPS_CA_DIR} = 'certs/';
>>
>> # DEFAULT SSL VERSION
>> $ENV{HTTPS_VERSION} = '3';
>> ----------------------------------------
>> I get the same result.
>>
>> You can run the script against my proxy server
>> 207.154.17.141 <http://207.154.17.141/>. Maybe my proxy
>> server is not configured properly. I checked my proxy
>> configuration using firefox browser.
>>
>> Thanks
>>
>>
>> -jason
>>
>>
>> [email protected] <mailto:[email protected]> wrote:
>>> Jason:
>>>
>>> Try adding a
>>> use LWP::Debug qw(+);
>>> at the top of the code and see what the perl program reports.
>>>
>>> Btw, your code runs fine on my proxy configuration. I just have to add
>>> a $ENV{HTTPS_PROXY_USERNAME} and a $ENV{HTTPS_PROXY_PASSWORD} with the
>>>
>>>
>>> other specifications also.
>>>
>>> Regards,
>>> Atul.
>>>
>>> Jason wrote:
>>>
>>>> Hi,
>>>>
>>>> I could not get the example script from Crypt::SSLeay run successfully.
>>>> Here is the script
>>>> ---------------------
>>>> #!/usr/bin/perl
>>>> # PROXY SUPPORT
>>>> $ENV{HTTPS_PROXY} = '
>>>>
>>>> 207.154.17.141:80 <http://207.154.17.141/>';
>>>> # DEBUGGING SWITCH / LOW LEVEL SSL DIAGNOSTICS
>>>> $ENV{HTTPS_DEBUG} = 2;
>>>>
>>>>
>>>> use LWP::UserAgent;
>>>> my $ua = new LWP::UserAgent;
>>>> my $req = new HTTP::Request('GET', '
>>>>
>>>> https://www.nodeworks.com <https://www.nodeworks.com/>');
>>>> my $res = $ua->request($req);
>>>> print $res->code."\n";
>>>> ------------------------------------
>>>>
>>>> I tested the proxy server with Firefox browser. It worked fine. I
>>>>
>>>>
>>>> looked into the access log in the proxy server. When I run the example
>>>> script.
>>>>
>>>> I got two access log entries
>>>> --------------------
>>>> - - [25/Sep/2006:20:42:23 -0400] "CONNECT
>>>>
>>>> www.nodeworks.com:443 <http://www.nodeworks.com:443/>
>>>> HTTP/1.0" 200 - "-" "libwww-perl/5.805"
>>>> - - [25/Sep/2006:20:42:24 -0400] "CONNECT
>>>>
>>>> 207.154.17.141:80 <http://207.154.17.141/> HTTP/1.0"
>>>> 403 298 "-" "libwww-perl/5.805"
>>>> -------------------
>>>>
>>>> Notice that LWP tried to issue an CONNECT method to the proxy server
>>>> itself. It got a 403 response for this request. On the other hand,
>>>>
>>>>
>>>> firefox did not send CONNECT method to proxy server itself.
>>>> - - [25/Sep/2006:19:33:34 -0400] "CONNECT
>>>>
>>>> www.nodeworks.com:443 <http://www.nodeworks.com:443/>
>>>> HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
>>>> rv:
>>>>
>>>> 1.8.0.7 <http://1.8.0.7/>) Gecko/20060909 Firefox/1.5.0.7"
>>>> - - [25/Sep/2006:19:33:38 -0400] "CONNECT www.nodeworks.com:443
>>>>
>>>> <http://www.nodeworks.com:443/>
>>>> HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
>>>> rv:1.8.0.7 <http://1.8.0.7/>) Gecko/20060909 Firefox/1.5.0.7"
>>>>
>>>>
>>>> - - [25/Sep/2006:19:34:32 -0400] "CONNECT www.nodeworks.com:443 <http://www.nodeworks.com:443/>
>>>> HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT
>>>>
>>>> 5.1; en-US;
>>>> rv:1.8.0.7 <http://1.8.0.7/>) Gecko/20060909 Firefox/1.5.0.7"
>>>> - - [25/Sep/2006:19:34:29 -0400] "CONNECT
>>>>
>>>> www.nodeworks.com:443 <http://www.nodeworks.com:443/>
>>>> HTTP/1.1" 200 - "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
>>>> rv:
>>>>
>>>> 1.8.0.7 <http://1.8.0.7/>) Gecko/20060909 Firefox/1.5.0.7"
>>>>
>>>> Not sure what is trick here. It seemed that other can get the example
>>>> script run properly. Can anyone give me some help on this?
>>>>
>>>> Thanks
>>>>
>>>> -jason
>>>>
>>>>
>>>>
>>>
>>
>>
>
>