polipo memory growing apparently without bound; atoms?

Josh Triplett <[email protected]>
Newsgroups gmane.comp.web.polipo.user
Message-ID <20091015101315.GA8533@feather>
Hello,

We use polipo as part of a web crawler, and it greatly simplifies our
code to not have to deal with caching ourselves; thank you for polipo.

We seem to have run into an issue where polipo's memory usage grows
without bound.  After running for a while, polipo will grow to 100M or
more.  We tried configuring polipo to use less memory, but this did not
help.

Investigating, I found the following entries in polipo's log:

Discarded all objects, 1 + 3 objects left (4 chunks and 307966 atoms used).
Discarded all objects, 1 + 3 objects left (5 chunks and 313599 atoms used).
Discarded all objects, 1 + 3 objects left (5 chunks and 319717 atoms used).
Discarded all objects, 0 + 3 objects left (4 chunks and 325469 atoms used).
Discarded all objects, 0 + 3 objects left (4 chunks and 331110 atoms used).
Discarded all objects, 0 + 3 objects left (4 chunks and 336715 atoms used).
Discarded all objects, 1 + 3 objects left (6 chunks and 342191 atoms used).
Discarded all objects, 1 + 3 objects left (5 chunks and 347533 atoms used).
Discarded all objects, 1 + 3 objects left (5 chunks and 351679 atoms used).

This suggests that the problem may lie with polipo's use of atoms, the
number of which continues to grow.

We'd like to try to resolve this issue.  Please let us know if we can
provide any further information.  Also happy to test any potential
fixes.

- Josh Triplett

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
lmpx.com only provides a reader for public news (NNTP) servers. It is not affiliated with the servers or forums shown here and is not responsible for the content of articles, which is written by their respective authors.