Re: transfer-encoding buglet

Bob McElrath <[email protected]> Tue, 14 Oct 2003 07:38:32 -0700
Newsgroups	gmane.comp.lang.perl.modules.lwp,gmane.comp.web.filterproxy.devel
Message-ID	<[email protected]>
--HkMjoL2LAeBLhbFV
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

Gisle Aas [[email protected]] wrote:
> Bob McElrath <[email protected]> writes:
>=20
> > This version doesn't cause a warning if the "Connection" header isn't
> > present.
>=20
> Perhaps this warnings shows becase nothing sets 'http_connection'. =20

Of course.  The warning was a bug in my original patch.  The patch is
trying to fix the transfer-encoding bug indicated below.

> Do you have further patches to Net::HTTP not shown here.

No more patches.

> > Bob McElrath [[email protected]] wrote:
> > > According to RFC 2616, section 3.6:
> > >     Whenever a transfer-coding is applied to a message-body, the set =
of
> > >     transfer-codings MUST include "chunked", unless the message is
> > >     terminated by closing the connection.
> > >=20
> > > libwww currently does not correctly implement the "unless" clause whe=
re
> > > the connection is closed but a transfer-coding header is present.
>=20
> I agree.
>=20
> > > Attached is a small patch which fixes this.  A URL that I've run
> > > across that exhibit the problem with FilterProxy are:
> > >=20
> > >     http://larve.net/people/hugo/2000/07/ml-mutt
>=20
> I don't see any problem with this one.  Do you have other examples?

Without patch:
    (1)<mcelrath@navi:/home/mcelrath> lwp-request -U http://larve.net/peopl=
e/hugo/2000/07/ml-mutt
    GET http://larve.net/people/hugo/2000/07/ml-mutt
    User-Agent: lwp-request/2.01

    <HTML>
    <HEAD><TITLE>An Error Occurred</TITLE></HEAD>
    <BODY>
    <H1>An Error Occurred</H1>
    500 Chunked must be last Transfer-Encoding 'deflate'
    </BODY>
    </HTML>

With patch:
    (0)<mcelrath@draal:/home/mcelrath> lwp-request -ue http://larve.net/peo=
ple/hugo/2000/07/ml-mutt
    GET http://larve.net/people/hugo/2000/07/ml-mutt
    Cache-Control: max-age=3D21600
    Connection: close
    Date: Tue, 14 Oct 2003 14:25:30 GMT
    ETag: "8buij7:ub1p6un0"
    Server: Jigsaw/2.2-20010823 jre/1.2.2_009 javacomp/1.2.15
    Content-Location: http://larve.net/people/hugo/2000/07/ml-mutt.html
    Content-Type: text/html;charset=3Dus-ascii
    Content-Type: text/html; charset=3Dus-ascii
    Expires: Tue, 14 Oct 2003 20:25:30 GMT
    Last-Modified: Thu, 16 Jan 2003 04:11:55 GMT
    11-PolicyRef: /2000/08/p3p-policyref
    Client-Date: Tue, 14 Oct 2003 14:17:52 GMT
    Client-Peer: 18.29.5.151:80
    Client-Response-Num: 1
    Client-Transfer-Encoding: deflate,chunked
    Link: </stylesheets/base>; /=3D"/"; rel=3D"stylesheet"; type=3D"text/cs=
s"
    Opt: "http://www.w3.org/2000/P3Pv1";ns=3D11
    Title: Using Mutt to send emails to mailing lists

    [...document...]

The 500 error is generated by LWP, not the HTTP server.  The actual
headers sent by the server are (snooping with ngrep):
     HTTP/1.0 200 OK
     Connection: close
     Date: Tue, 14 Oct 2003 14:34:21 GMT
     Transfer-Encoding: deflate
     Opt: "http://www.w3.org/2000/P3Pv1";ns=3D11
     Content-Location: http://larve.net/people/hugo/2000/07/ml-mutt.html
     Content-Type: text/html;charset=3Dus-ascii
     Etag: "8buij7:ub1p6un0"
     Expires: Tue, 14 Oct 2003 20:34:21 GMT
     Last-Modified: Thu, 16 Jan 2003 04:11:55 GMT
     Server: Jigsaw/2.2-20010823 jre/1.2.2_009 javacomp/1.2.15..11-PolicyRe=
f: /2000/08/p3p-policyref

I have no control over this server.  It would appear that it is using a
HTTP/1.1 header (Transfer-Encoding) in a HTTP/1.0 response, which is
probably not so kosher.  But the method read_entity_body that my patch
applies to does not check the HTTP version anyway.

Cheers,
Bob McElrath [Univ. of California at Davis, Department of Physics]

    "Knowledge will forever govern ignorance, and a people who mean to
    be their own governors, must arm themselves with the power knowledge
    gives. A popular government without popular information or the means
    of acquiring it, is but a prologue to a farce or a tragedy or
    perhaps both."
        - James Madison

--HkMjoL2LAeBLhbFV
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/jApojwioWRGe9K0RAirUAKDhmDeD4yvJzhlh+cg1rxZhPEQo2ACffsiV
6R/Webt6jvctE6PTS/Xc0Jc=
=8vze
-----END PGP SIGNATURE-----

--HkMjoL2LAeBLhbFV--