[ ijbswa-Actionsfile feedback-3394462 ] Some websites are very slow (some minutes)

Actionsfile feedback item #3394462, was opened at 2011-08-19 11:22
Message generated for change (Comment added) made by fabiankeil
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=460288&aid=3394462&group_id=11118

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: filter: other problem
Group: None
>Status: Closed
Resolution: None
Priority: 5
Private: No
Submitted By: Andy (ihatesfregistra)
Assigned to: Fabian Keil (fabiankeil)
Summary: Some websites are very slow (some minutes)

Initial Comment:
Some websites take a few minutes to load with Privoxy, but load in a reasonable amount of time (few seconds) without Privoxy.

An example is:
http://www.bauhaus-design24.com/uk/

With Privoxy I just aborted loading the page after 4 minutes. The page was not completely loaded and many images where marked as broken.
Sometimes I just get the Privoxy error message "No server or forwarder data received".
Without Privoxy the page loads within a few seconds including all pictures.

This happens with Privoxy 3.0.17 with default configuration (I commented out user.filter and user.action).

Privoxy runs on a server (Debian) - CPU load of the computer ist ca. zero according to top.
The client runs Seamonkey (same result with other web browsers)  - CPU load also about zero.
Internet traffic is very low (a few kBytes/s for a short period, than zero for a short period, than
again a few kBytes/s, ...).


----------------------------------------------------------------------

>Comment By: Fabian Keil (fabiankeil)
Date: 2011-08-24 16:44

Message:
You're welcome.

----------------------------------------------------------------------

Comment By: Andy (ihatesfregistra)
Date: 2011-08-24 14:07

Message:
I agree, it doesn't look like a Privoxy bug.

I am not sure about a PMTU discovery or ICMP problem, e.g. because the pf
man page says that ICMP error messages are matched against the
corresponding connections in the state table. I will have to dig a bit
deeper before I know for sure, but that is another story and off topic.

Nonetheless thank you very much for your great support.


----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2011-08-23 19:08

Message:
Thanks, I take this as a confirmation that this isn't a Privoxy issue.

I believe those symptoms could be explained by a MTU discovery problem
due to ICMP being filtered too restrictively, in which case this would be
a
configuration issue and not a bug in pf.

----------------------------------------------------------------------

Comment By: Andy (ihatesfregistra)
Date: 2011-08-23 17:56

Message:
> I assume that when you tested the page without Privoxy, the request
didn't
> actually come from the server running Privoxy, is that correct?
>
> Can you try reproducing the problem without Privoxy with requests
> originating from the Debian server?

Right.
I have now tried to connect to the website with lynx and w3m running
directly on the Privoxy server (there is no X11 installed).

With pf scrub(reassemble tcp) on the router the connection is unreliable.
I think between 20 and 50 percent of my trials failed with both browsers (I
didn't try it very often and these text based web browsers don't try to
fetch all those images).

Without scrub on the router it seems to work perfectly.


----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2011-08-22 19:31

Message:
Thanks a lot for the log files.

According to them, the connections get indeed terminated by the other end
(or any system routing the packets) before any data makes it through.

Even if I reuse the client data from one of the failed requests I can't
reproduce the problem here.

I assume that when you tested the page without Privoxy, the request didn't
actually come from the server running Privoxy, is that correct?

Can you try reproducing the problem without Privoxy with requests
originating from the Debian server?

----------------------------------------------------------------------

Comment By: Andy (ihatesfregistra)
Date: 2011-08-21 17:28

Message:
Great to hear that you are also using pf.

I just uploaded a log file with all mentioned debug values (incl. 32768).
I had to split it into multiple parts, as it was to large for upload
otherwise.

In case it is important: Privoxy runs on Debian with kernel "2.6.26-2-686
#1 SMP Mon Aug 30 07:01:57 UTC 2010 i686".

Please let me know if you need more information.


----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2011-08-20 20:19

Message:
The fact that you can only reproduce the problem with scrubbing enabled
doesn't really rule out a Privoxy issue, so feel free to attach a logfile
anyway.

Adding "debug 32768" to the other recommended debug directives would be
especially useful here.

While I'm using pf (4.5 on FreeBSD 9.0) with reassemble tcp too, our
connections are likely to be different enough to cause different
fragmentation.

----------------------------------------------------------------------

Comment By: Andy (ihatesfregistra)
Date: 2011-08-19 21:07

Message:
First of all thank you for the quick reply.

After I read your comment about not being able to reproduce it, I did some
more thinking and testing. It seems that the problem only exists in
combination with an OpenBSD pf "scrub (reassemble tcp)" rule on the
router.

Unfortunately I am not yet running the latest version of pf and there is a
chance that the problem is fixed there. Therefore I don't think that it
makes much sense to look at the log files now.

I don't know when I will install the current OpenBSD/pf version, but if
the problem still exists afterwards, I will try to contact the pf
developers.

Sorry for wasting your time, but it helped me a lot to get closer to the
root of the problem. Thank you very much.



----------------------------------------------------------------------

Comment By: Fabian Keil (fabiankeil)
Date: 2011-08-19 15:15

Message:
I can't reproduce this. The page loads in less than 20 seconds here.

Please reproduce the problem with logging enabled as described at:
http://www.privoxy.org/user-manual/contact.html#CONTACT-BUGS
and attach the log file.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=460288&aid=3394462&group_id=11118

------------------------------------------------------------------------------
EMC VNX: the world's simplest storage, starting under $10K
The only unified storage solution that offers unified management 
Up to 160% more powerful than alternatives and 25% more efficient. 
Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev