Re: BDB uses far more memory than it's supposed to; ideas?

Maybe I am completely off base here, but according to
http://www.oracle.com/technology/documentation/berkeley-db/je/relnotes.html
You must set the memory using a few different configuration
properties.  I searched the Freenet source and was unable to find:
EnvironmentConfig.setCacheSize
je.maxMemory
je.maxMemoryPercent

Are you using a command or function other than these?


On 1/30/07, Matthew Toseland <toad-EI5O+8PHWbJeeLb3ft/[email protected]> wrote:
> After various reports of OOMs, and after high CPU usage which might have
> been related to excessive GCing, I did some profiling.
>
> Having analysed the first 70% of memory on the dump, with a 100MB
> overall limit, the Berkeley DB Java Edition that we use for the database
> uses at least 54.5MB of RAM. databaseMaxMemory is not overridden in the
> config, and therefore should be at the default value of 20MB.
>
> The documentation says:
> "Note that the cache does not include transient objects created by the JE
> library, such as cursors, locks and transactions."
>
> It is possible that the cache only refers to the first stack trace,
> which is 17MB. However there are many other traces which appear to be
> dealing with the database log (BDB is a log-structured database), many
> of them rather large - for example the next 3 traces of 10MB, 5MB and
> 5MB respectively.
>
> Any ideas? We can:
> - Just ignore it. We may have to increase the default memory limit. But
>   anecdotally OOMs happen when you have lots of queued requests.
> - Use another database. BDB has been fairly unreliable, hence the code
>   in the store to reconstruct the database (store index) from the store
>   file by deleting the database and parsing each key.
> - Given the second item, if we had a reliable database we could store
>   queued requests in it, thus limiting the overall memory usage
>   regardless of the size of the request queue. But that would be a
>   significant amount of work even with a database.
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
>
> iD8DBQFFv3hRA9rUluQ9pFARAlC/AJ9u0JzMj746VIK2hSYA76NNbjxs6ACeMS8E
> TTux6RxGK0Eo4PGlKQzCldM=
> =c7LM
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Tech mailing list
> [email protected]
> http://emu.freenetproject.org/cgi-bin/mailman/listinfo/tech
>


-- 
I may disagree with what you have to say, but I shall defend, to the
death, your right to say it. - Voltaire
Those who would give up Liberty, to purchase temporary Safety, deserve
neither Liberty nor Safety. - Ben Franklin