Net::HTTPS 6.04: read_entity_body hanging

--089e01419d7c82e3a704d76509e1
Content-Type: text/plain; charset=ISO-8859-1

Some LWP requests are hanging *most* of the time.  Sometimes they work.  At
times it seems dependent if gzip is used or not.   This behavior seems to
have started with recent LWP upgrades.   If I use Net::HTTPS 6.04 (most
current) I see the problem.  Reverting to Net::HTTPS 6.02 appears to fix
the issue.


I'm not clear if it's related to this ticket:
https://rt.cpan.org/Public/Bug/Display.html?id=81237

Below are example command line requests and where they are hanging in LWP.


LWP::UserAgent is up to date. (6.04)
LWP::Protocol::https is up to date. (6.03)
Net::SSL is up to date. (2.85)
openssl098e.x86_64
CentOS 6.2.

Net::HTTPS (6.04) / Net-HTTP-6.05 -- hangs.
Net::HTTPS (6.04) / Net-HTTP-6.04 -- hangs.
Net::HTTPS (6.02) / Net-HTTP-6.03 -- works ok.


The page fetched below is behind an F5 load balancer that handles the SSL
and gzip compression.


In LWP::Protocol::http I've modified the code like this so it's easy to see
that read_entity_body is hanging.

      READ:
        {
warn "Reading $size from $socket\n";
            $n = $socket->read_entity_body($buf, $size);
warn "Read $n bytes\n";
            unless (defined $n) {
                redo READ if $!{EINTR} || $!{EAGAIN};
                die "read failed: $!";
            }
            redo READ if $n == -1;
        }

For example:

$ perl -MLWP::UserAgent -e 'LWP::UserAgent->new->get( "
https://app.ithenticate.com/en_us/login", accept_encoding => "gzip" );'
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x164d0c8)
Read 605 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x164d0c8)
(hangs here)
^C


$ perl -MLWP::UserAgent -e 'LWP::UserAgent->new->get( "
https://app.ithenticate.com/en_us/login", *accept_encoding => "nope"* );'
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 653 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 459 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 2896 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 1448 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 2880 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x26ea0c8)
Read 0 bytes

Yet, it's not consistent.  Here's without gzip and it does hang.

$ perl -MLWP::UserAgent -e 'LWP::UserAgent->new->get( "
https://app.ithenticate.com/en_us/login", accept_encoding => "nope" );'
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x14720c8)
Read 653 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x14720c8)
Read 471 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x14720c8)
Read 4096 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x14720c8)
( hangs here again )
^C

Then again it works:

$ perl -MLWP::UserAgent -e 'LWP::UserAgent->new->get( "
https://app.ithenticate.com/en_us/login", accept_encoding => "nope" );'
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 653 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 459 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 2896 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 1448 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 2880 bytes
Reading 4096 from LWP::Protocol::https::Socket=GLOB(0x19530c8)
Read 0 bytes

Sorry, not a lot to go on.

-- 
Bill Moseley
[email protected]

--089e01419d7c82e3a704d76509e1--