Re: LWP hangs forever with get
Jesper Persson <[email protected]> Mon, 26 May 2014 08:47:35 +0200
| Newsgroups | gmane.comp.lang.perl.modules.lwp |
|---|---|
| Message-ID | <CA+uC3V3+FDSe1zzGvm8_hS=YZ4HX1M-csn-AcwF9bQJ2o_P1Vg@mail.gmail.com> |
--001a11c146d07dfe4704fa47f55d
Content-Type: text/plain; charset=UTF-8
The "hang" happens in c:\strawberry\perl\vendor\lib\Net\HTTP\Methods.pm in
the procedure
sub read_response_headers
where it calls
my($status, $eol) = my_readline($self, 'Status');
Its actually not a hang because perl is looping in my_readline where it
keeps repeating the code:
READ:
{
die "read timeout" unless $self->can_read;
my $n = $self->sysread($_, 1024, length);
unless (defined $n) {
redo READ if $!{EINTR} || $!{EAGAIN};
Should that be possible? Maybe the procedure could have a max number of
"redo READ"'s or a timer to give up after some time?
Best regards
Jesper Persson
On 21 May 2014 21:10, Jesper Persson <[email protected]> wrote:
> Well I do set the $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0; so I guess the
> change shouldn't affect me.
>
> perl code
> -----------------------------------
> use LWP::Simple qw($ua get);
> use IO::Socket::SSL qw(debug3);
> $ua->timeout(5);
> $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 1;
> $contents = get("https://www.butler.edu/boa/default.aspx");
> -----------------------------------
>
> with the perl version that doesnt hang:
>
> strawberry perl 5.18.1.1
> libwww-perl-6.05
>
> DEBUG: .../IO/Socket/SSL.pm:1805: new ctx 39861360
> DEBUG: .../IO/Socket/SSL.pm:449: socket not yet connected
> DEBUG: .../IO/Socket/SSL.pm:451: socket connected
> DEBUG: .../IO/Socket/SSL.pm:469: ssl handshake not started
> DEBUG: .../IO/Socket/SSL.pm:504: using SNI with hostname www.butler.edu
> DEBUG: .../IO/Socket/SSL.pm:527: set socket to non-blocking to enforce
> timeout=5
> DEBUG: .../IO/Socket/SSL.pm:540: Net::SSLeay::connect -> -1
> DEBUG: .../IO/Socket/SSL.pm:550: ssl handshake in progress
> DEBUG: .../IO/Socket/SSL.pm:560: waiting for fd to become ready: SSL wants
> a read first
> DEBUG: .../IO/Socket/SSL.pm:570: handshake failed because socket did not
> became ready
> DEBUG: .../IO/Socket/SSL.pm:1842: free ctx 39861360 open=39861360
> DEBUG: .../IO/Socket/SSL.pm:1847: free ctx 39861360 callback
> DEBUG: .../IO/Socket/SSL.pm:1850: OK free ctx 39861360
>
> NO HANG :-)
>
> with the perl version that hangs:
>
> strawberry 5.18.2.2
> libwww-perl-6.05
> DEBUG: .../IO/Socket/SSL.pm:2137: new ctx 49554496
> DEBUG: .../IO/Socket/SSL.pm:478: socket not yet connected
> DEBUG: .../IO/Socket/SSL.pm:480: socket connected
> DEBUG: .../IO/Socket/SSL.pm:498: ssl handshake not started
> DEBUG: .../IO/Socket/SSL.pm:528: using SNI with hostname www.butler.edu
> DEBUG: .../IO/Socket/SSL.pm:566: set socket to non-blocking to enforce
> timeout=5
> DEBUG: .../IO/Socket/SSL.pm:579: Net::SSLeay::connect -> -1
> DEBUG: .../IO/Socket/SSL.pm:589: ssl handshake in progress
> DEBUG: .../IO/Socket/SSL.pm:599: waiting for fd to become ready: SSL wants
> a read first
> DEBUG: .../IO/Socket/SSL.pm:619: socket ready, retrying connect
> DEBUG: .../IO/Socket/SSL.pm:2098: ok=1 cert=53220176
> DEBUG: .../IO/Socket/SSL.pm:2098: ok=1 cert=60570800
> DEBUG: .../IO/Socket/SSL.pm:2098: ok=1 cert=60570608
> DEBUG: .../IO/Socket/SSL.pm:1339: scheme=www cert=60570608
> DEBUG: .../IO/Socket/SSL.pm:1348: identity=www.butler.edu cn=
> www.butler.edu alt=2 www.butler.edu 2 butler.edu
> DEBUG: .../IO/Socket/SSL.pm:579: Net::SSLeay::connect -> -1
> DEBUG: .../IO/Socket/SSL.pm:589: ssl handshake in progress
> DEBUG: .../IO/Socket/SSL.pm:599: waiting for fd to become ready: SSL wants
> a read first
> DEBUG: .../IO/Socket/SSL.pm:619: socket ready, retrying connect
> DEBUG: .../IO/Socket/SSL.pm:579: Net::SSLeay::connect -> 1
> DEBUG: .../IO/Socket/SSL.pm:634: ssl handshake done
>
> HANGS FOREVER :-(
>
> /Jesper
>
>
>
> On 21 May 2014 19:03, Tom Hukins <[email protected]> wrote:
>
>> On Wed, May 21, 2014 at 06:57:58PM +0200, Jesper Persson wrote:
>> > this perl code
>> > ------------------------------------------
>> > use LWP::Simple qw($ua get);
>> > $ua->timeout(5);
>> > $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;
>> > $contents = get("https://www.butler.edu/boa/default.aspx");
>> > ------------------------------------------
>> >
>> > just tried fresh perl installs.
>> >
>> > strawberry perl 5.16.3.1: doesnt hang.
>> >
>> > strawberry perl 5.18.0.1: doesnt hang.
>> >
>> > strawberry perl 5.18.1.1: doesnt hang.
>> >
>> > strawberry perl 5.18.2.1: hangs.
>> >
>> > strawberry perl 5.18.2.2: hangs.
>>
>> I expect you're using different versions of LWP with these different
>> versions of Perl and you're encountering the change documented at
>> https://metacpan.org/pod/LWP#PERL_LWP_SSL_VERIFY_HOSTNAME
>>
>> Tom
>>
>
>
--001a11c146d07dfe4704fa47f55d
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">The "hang" happens in c:\strawberry\perl\vendor\=
lib\Net\HTTP\Methods.pm in the procedure<div>sub read_response_headers<br><=
/div><div>where it calls=C2=A0</div><div>my($status, $eol) =3D my_readline(=
$self, 'Status');<br>
</div><div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Its a=
ctually not a hang because perl is looping in my_readline where it keeps re=
peating the code:</div><div class=3D"gmail_extra"><br></div><div class=3D"g=
mail_extra">
<div class=3D"gmail_extra">READ:</div><div class=3D"gmail_extra">=C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 {</div><div class=3D"gmail_extra">=C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 die "read timeout&quo=
t; unless $self->can_read;</div><div class=3D"gmail_extra">=C2=A0 =C2=A0=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 my $n =3D $self->sysread($_, =
1024, length);</div>
<div class=3D"gmail_extra">=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0=
=C2=A0 unless (defined $n) {</div><div class=3D"gmail_extra">=C2=A0 =C2=A0=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 redo READ if $!{EI=
NTR} || $!{EAGAIN};</div><div class=3D"gmail_extra"><br></div><div class=3D=
"gmail_extra">Should that be possible? Maybe the procedure could have a max=
number of "redo READ"'s or a timer to give up after some tim=
e?</div>
<div class=3D"gmail_extra"><br></div><div class=3D"gmail_extra">Best regard=
s</div><div class=3D"gmail_extra">Jesper Persson</div></div><div class=3D"g=
mail_extra"><br><br><div class=3D"gmail_quote">On 21 May 2014 21:10, Jesper=
Persson <span dir=3D"ltr"><<a href=3D"mailto:[email protected]=
" target=3D"_blank">[email protected]</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex"><div dir=3D"ltr">Well I do set the=C2=A0<span style=3D"col=
or:rgb(80,0,80);font-family:arial,sans-serif;font-size:13px">$ENV{PERL_LWP_=
SSL_VERIFY_</span><span style=3D"color:rgb(80,0,80);font-family:arial,sans-=
serif;font-size:13px">HOSTNAME} =3D 0;</span>=C2=A0so I guess the change sh=
ouldn't affect me.<div>
<br></div><div>perl code</div><div>-----------------------------------</div=
><div><div class=3D""><div>use LWP::Simple qw($ua get);</div></div><div>use=
IO::Socket::SSL qw(debug3);</div><div class=3D""><div>$ua->timeout(5);<=
/div>
<div>$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} =3D 1;</div>
<div>$contents =3D get("<a href=3D"https://www.butler.edu/boa/default.=
aspx" target=3D"_blank">https://www.butler.edu/boa/default.aspx</a>");=
</div></div></div><div>-----------------------------------<br></div><div><b=
r>
</div><div>with the perl version that doesnt hang:</div>
<div><br></div><div><div>strawberry perl 5.18.1.1</div><div>libwww-perl-6.0=
5</div><div><br></div><div>DEBUG: .../IO/Socket/SSL.pm:1805: new ctx 398613=
60</div><div>DEBUG: .../IO/Socket/SSL.pm:449: socket not yet connected</div=
>
<div>DEBUG: .../IO/Socket/SSL.pm:451: socket connected</div><div>DEBUG: ...=
/IO/Socket/SSL.pm:469: ssl handshake not started</div><div>DEBUG: .../IO/So=
cket/SSL.pm:504: using SNI with hostname <a href=3D"http://www.butler.edu" =
target=3D"_blank">www.butler.edu</a></div>
<div>DEBUG: .../IO/Socket/SSL.pm:527: set socket to non-blocking to enforce=
timeout=3D5</div><div>DEBUG: .../IO/Socket/SSL.pm:540: Net::SSLeay::connec=
t -> -1</div><div>DEBUG: .../IO/Socket/SSL.pm:550: ssl handshake in prog=
ress</div>
<div>DEBUG: .../IO/Socket/SSL.pm:560: waiting for fd to become ready: SSL w=
ants a read first</div><div>DEBUG: .../IO/Socket/SSL.pm:570: handshake fail=
ed because socket did not became ready</div><div>DEBUG: .../IO/Socket/SSL.p=
m:1842: free ctx 39861360 open=3D39861360</div>
<div>DEBUG: .../IO/Socket/SSL.pm:1847: free ctx 39861360 callback</div><div=
>DEBUG: .../IO/Socket/SSL.pm:1850: OK free ctx 39861360</div><div><br></div=
><div>NO HANG :-)</div></div><div><br></div><div>with the perl version that=
hangs:</div>
<div><br></div><div><div>strawberry 5.18.2.2</div><div>libwww-perl-6.05</di=
v><div>DEBUG: .../IO/Socket/SSL.pm:2137: new ctx 49554496</div><div>DEBUG: =
.../IO/Socket/SSL.pm:478: socket not yet connected</div><div>DEBUG: .../IO/=
Socket/SSL.pm:480: socket connected</div>
<div>DEBUG: .../IO/Socket/SSL.pm:498: ssl handshake not started</div><div>D=
EBUG: .../IO/Socket/SSL.pm:528: using SNI with hostname <a href=3D"http://w=
ww.butler.edu" target=3D"_blank">www.butler.edu</a></div><div>DEBUG: .../IO=
/Socket/SSL.pm:566: set socket to non-blocking to enforce timeout=3D5</div>
<div>DEBUG: .../IO/Socket/SSL.pm:579: Net::SSLeay::connect -> -1</div><d=
iv>DEBUG: .../IO/Socket/SSL.pm:589: ssl handshake in progress</div><div>DEB=
UG: .../IO/Socket/SSL.pm:599: waiting for fd to become ready: SSL wants a r=
ead first</div>
<div>DEBUG: .../IO/Socket/SSL.pm:619: socket ready, retrying connect</div><=
div>DEBUG: .../IO/Socket/SSL.pm:2098: ok=3D1 cert=3D53220176</div><div>DEBU=
G: .../IO/Socket/SSL.pm:2098: ok=3D1 cert=3D60570800</div><div>DEBUG: .../I=
O/Socket/SSL.pm:2098: ok=3D1 cert=3D60570608</div>
<div>DEBUG: .../IO/Socket/SSL.pm:1339: scheme=3Dwww cert=3D60570608</div><d=
iv>DEBUG: .../IO/Socket/SSL.pm:1348: identity=3D<a href=3D"http://www.butle=
r.edu" target=3D"_blank">www.butler.edu</a> cn=3D<a href=3D"http://www.butl=
er.edu" target=3D"_blank">www.butler.edu</a> alt=3D2 <a href=3D"http://www.=
butler.edu" target=3D"_blank">www.butler.edu</a> 2 <a href=3D"http://butler=
.edu" target=3D"_blank">butler.edu</a></div>
<div>DEBUG: .../IO/Socket/SSL.pm:579: Net::SSLeay::connect -> -1</div><d=
iv>DEBUG: .../IO/Socket/SSL.pm:589: ssl handshake in progress</div><div>DEB=
UG: .../IO/Socket/SSL.pm:599: waiting for fd to become ready: SSL wants a r=
ead first</div>
<div>DEBUG: .../IO/Socket/SSL.pm:619: socket ready, retrying connect</div><=
div>DEBUG: .../IO/Socket/SSL.pm:579: Net::SSLeay::connect -> 1</div><div=
>DEBUG: .../IO/Socket/SSL.pm:634: ssl handshake done</div><div><br></div>
<div>HANGS FOREVER :-(</div></div><span class=3D""><font color=3D"#888888">=
<div><br></div><div>/Jesper</div><div><br></div></font></span></div><div cl=
ass=3D""><div class=3D"h5"><div class=3D"gmail_extra"><br><br><div class=3D=
"gmail_quote">
On 21 May 2014 19:03, Tom Hukins <span dir=3D"ltr"><<a href=3D"mailto:to=
[email protected]" target=3D"_blank">[email protected]</a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-=
left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;p=
adding-left:1ex"><div>On Wed, May 21, 2014 at 06:57:58PM +0200, Jesper Pers=
son wrote:<br>
> this perl code<br>
> ------------------------------------------<br>
> use LWP::Simple qw($ua get);<br>
> $ua->timeout(5);<br>
> $ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} =3D 0;<br>
> $contents =3D get("<a href=3D"https://www.butler.edu/boa/default.=
aspx" target=3D"_blank">https://www.butler.edu/boa/default.aspx</a>");=
<br>
> ------------------------------------------<br>
><br>
> just tried fresh perl installs.<br>
><br>
> strawberry perl <a href=3D"http://5.16.3.1" target=3D"_blank">5.16.3.1=
</a>: =C2=A0doesnt hang.<br>
><br>
> strawberry perl <a href=3D"http://5.18.0.1" target=3D"_blank">5.18.0.1=
</a>: =C2=A0doesnt hang.<br>
><br>
> strawberry perl <a href=3D"http://5.18.1.1" target=3D"_blank">5.18.1.1=
</a>: =C2=A0doesnt hang.<br>
><br>
> strawberry perl <a href=3D"http://5.18.2.1" target=3D"_blank">5.18.2.1=
</a>: =C2=A0hangs.<br>
><br>
> strawberry perl <a href=3D"http://5.18.2.2" target=3D"_blank">5.18.2.2=
</a>: =C2=A0hangs.<br>
<br>
</div>I expect you're using different versions of LWP with these differ=
ent<br>
versions of Perl and you're encountering the change documented at<br>
<a href=3D"https://metacpan.org/pod/LWP#PERL_LWP_SSL_VERIFY_HOSTNAME" targe=
t=3D"_blank">https://metacpan.org/pod/LWP#PERL_LWP_SSL_VERIFY_HOSTNAME</a><=
br>
<span><font color=3D"#888888"><br>
Tom<br>
</font></span></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>
--001a11c146d07dfe4704fa47f55d--