Re: zope2/zodb cache size question
Jürgen Herrmann <[email protected]>
| Newsgroups | gmane.comp.web.zope.devel |
|---|---|
| Organization | XLhost.de GmbH |
| Message-ID | <[email protected]> |
On Thu, 22 Apr 2010 12:54:55 -0400, Jim Fulton <[email protected]> wrote: > On Thu, Apr 22, 2010 at 9:00 AM, Jürgen Herrmann > <[email protected]> wrote: >> >> hi there! >> >> today i ran a check script which iterates over approx 150000 objects >> and does some sanity check calculations on these. during this loop >> i saw the zope process use up to about 4.5gb memory. the database >> has ~3.5million objects in it. >> >> i set the zodb cache size for the mount point in question to 10000 >> objects. obviously this limit is not honoured during one transaction: > > Right. > >> so two questions here: >> - would the byte-limited zodb cache setting help here? > > No. The limits aren't really limits. They are more like > suggestions. :) In particular, they are only enforced at > transaction (or subtransaction) boundaries, when a connection is > opened or closed, or when applications call certain APIs explicitly. are these apis exposed somewhere? calling these periodically sounds cleaner than the aproach below. > >> - if no - how can i iterate over a big set of objects without >> forcing them to stay in the cache for the whole transaction? >> after all i just need each object once during the iteration. > > As you're iterating through the objects, if you know you aren't going > to need an object again or otherwise think it would be OK to free it > from memory, you can call _p_deactivate on it. If the object hasn't > been modified, then it's state will be released and it will become a > ghost. as my objects contain references to OOBTrees, do i also have to call _p_deactivate() on these explicitly, or is it enough to call it on the containing object? thanks for your help! best regards, jürgen -- >> XLhost.de - eXperts in Linux hosting ® << XLhost.de GmbH Jürgen Herrmann, Geschäftsführer Boelckestrasse 21, 93051 Regensburg, Germany Geschäftsführer: Volker Geith, Jürgen Herrmann Registriert unter: HRB9918 Umsatzsteuer-Identifikationsnummer: DE245931218 Fon: +49 (0)800 XLHOSTDE [0800 95467833] Fax: +49 (0)800 95467830 WEB: http://www.XLhost.de IRC: #[email protected] _______________________________________________ Zope-Dev maillist - [email protected] https://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope )