Re: 2004-11-18 change to fd_read_body

Dear Hrvoje,

> A patch by Leonid Petrov changed fd_read_body by adding these lines:
> 
>      /* when retrieving from http-proxy wget sometimes does not trust the
>       * file length reported by server.
>       * this check is to tell wget not to stubbornly try to read again and
>       * again until another errno code was received. */
>      if ( ret == -1 && errno == ETIMEDOUT && sum_read == toread &&
toread > 0 )
>     break;
> 
> I believe this change is incorrect because it basically reimplements
> the "rb_read_exactly" flag, in a fairly non-obvious way.  The change
> can also break FTP downloads, where the size is in some cases merely a
> hint to provide better progress bars.

  Maybe you are right. Frankly speaking I cannot claim that I understood
what I have done. This fix appeared as a response to Mads' bug report.
Hi thought it was due to my LFS patch, but it turned out was a bug in
the CVS version. Can I ask you a favor to test whether your unpatch
still fixes the problem which Mads reported? Please find his bug report
attached.

Leonid
2005.03.15_21:37:06

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Attachment:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Date: Fri, 5 Nov 2004 12:48:13 +0100
From: Mads Martin Joergensen <[email protected]>
To: Leonid Petrov <[email protected]>
Subject: wget+LFS and proxy

Hey Leonid,

I've been using your LFS patch for wget and enjoying it. Thanks for it.
But it seems something is a bit fishy with the proxy support now.

Look at this:

12:42 mmj@f211:~/ > time wget \
http://belnet.dl.sourceforge.net/sourceforge/docbook/docbook-dsssl-doc-1.79.tar.gz
--12:44:14--
http://belnet.dl.sourceforge.net/sourceforge/docbook/docbook-dsssl-doc-1.79.tar.gz
           => `docbook-dsssl-doc-1.79.tar.gz.2'
Resolving proxy.suse.de... 2001:780:101:0:250:8bff:feaf:3108,
149.44.160.33
Connecting to proxy.suse.de|2001:780:101:0:250:8bff:feaf:3108|:3128...
failed: Connection refused.
Connecting to proxy.suse.de|149.44.160.33|:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 225,797 [application/x-gzip]

100%[==============================================>] 225,797
--.--K/s    ETA 00:00

12:45:15 (3.67 KB/s) - `docbook-dsssl-doc-1.79.tar.gz.2' saved
[225,797/225,797]

wget   0.01s user 0.01s system 0% cpu 1:00.63 total

And now without proxy:

12:45 mmj@f211:~/ > time wget --proxy=off \
http://belnet.dl.sourceforge.net/sourceforge/docbook/docbook-dsssl-doc-1.79.tar.gz
--12:46:09--
http://belnet.dl.sourceforge.net/sourceforge/docbook/docbook-dsssl-doc-1.79.tar.gz
           => `docbook-dsssl-doc-1.79.tar.gz.3'
Resolving belnet.dl.sourceforge.net... 193.190.198.97
Connecting to belnet.dl.sourceforge.net|193.190.198.97|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 225,797 [application/x-gzip]

100%[==============================================>] 225,797
129.36K/s

12:46:11 (128.99 KB/s) - `docbook-dsssl-doc-1.79.tar.gz.3' saved
[225,797/225,797]

wget --proxy=off   0.00s user 0.01s system 0% cpu 2.197 total

There's a pretty big difference allthough the one with the proxy goes up
to 100% just as fast as the one without it. It just sits there--seems
it's waiting for something to timeout.

Date: Fri, 12 Nov 2004 08:27:32 +0100
From: Mads Martin Joergensen <[email protected]>
To: Leonid Petrov <[email protected]>
Subject: Re: wget+LFS and proxy
Message-ID: <[email protected]>
References: <[email protected]>
<[email protected]>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <[email protected]>
User-Agent: Mutt/1.5.6i
Status: RO

* Leonid Petrov <[email protected]> [Nov 12. 2004 03:22]:
> > I've been using your LFS patch for wget and enjoying it. Thanks for it.
> > But it seems something is a bit fishy with the proxy support now.
> 
>   First, what is fishy? That what the speed with proxy is slower? Did you 
> check the old version of wget? Is the difference the same? Do you have 
> evidence that it is persistent?

Yes, I've collected more of the same reports all over the company.

>   Secondly, what should be done in order to reproduce this "bug"?

Use a proxy for http downloads.

-- 
Mads Martin Joergensen, http://mmj.dk