Polipo jams up under load

grarpamp <[email protected]>
Newsgroups gmane.comp.web.polipo.user
Message-ID <CAD2Ti2_LjFtr6qzBeBq89kdcVZuyVRaSw10UmrsOwKYEC0zgfw@mail.gmail.com>
I may have posted a note months ago about this.

When forking off a constant level of 15-50 concurrent http requests
through polipo, polipo will lock up and/or fail to properly close
down the connection, filehandles, sockets, etc. If I recall, there
were lots of TIME_WAIT2's, I think, don't quote me. Whatever state
it was, it showed up in fstat, sockstat and netstat -a on FreeBSD 8.x
i386. My parent at the time was Tor. I wasn't able to track it down, it
occurred seemingly at random, with maybe 15% of the production runs
working fine.  When it locked, no further connections would go
through polipo thus requiring a kill. Also, polipo would often fail
to proxy reliably as indicated by wget's logged retry count of 2
or more, often into the 5-15 range. The OS and stack itself was
working just fine throughout. All software was current.  Polipo was
simply not suitable for production as far as I could make of it.
Then I pointed the same parallel fetching script at Privoxy via the
same http_proxy environment variables and the problem disappeared.
Been using Privoxy ever since but thought I should give people here
a heads up as to this experience.

You might be able to replicate by fetching the Alexa top 100 in round
robin via Tor in parallel concurrency settings from 15-50 or so till it jams
up.

proxyAddress=127.0.0.1
proxyPort=8080
socksParentProxy=127.0.0.1:9050
socksProxyType=socks5
allowedPorts=0-65535
localDocumentRoot=
disableConfiguration=true
disableServersList=false
forbiddenTunnelsFile=
forbiddenFile=
diskCacheRoot=
uncachableFile=
logLevel=0xFF

------------------------------------------------------------------------------
Free Next-Gen Firewall Hardware Offer
Buy your Sophos next-gen firewall before the end March 2013 
and get the hardware for free! Learn more.
http://p.sf.net/sfu/sophos-d2d-feb
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.