Re: ideas for returning memory to the OS

* Brian Crowder <[email protected]> wrote:

Hi,

> Enrico:  I think we're having a communication problem here.  You seem
> to think we're trying to work around bugs in the general allocators
> provided by the operating systems.  We're not.  We're trying to
> improve the performance of our application using knowledge that no
> operating system allocator has -- or could ever have -- of its
> behavior.  

Yeah, we've mixed up different issues in one thread. Partially 
my fault - I should not try to come from one topic to another ;-O
But at least we're talking about these things here (more than I
could expect from other moz lists ;-P)

Once of my major problems is the enormous code complexit. IMHO 
this slows down development and also brings bad performance
(to get back OT ;-)).

For example, memory management problems were far from being that 
bad if moz would be split off into several smaller subsystems with 
tiny and strict interfaces, which are only started on demand.
One of those subsystems could be bookmark+history storage: 
If this would be done by an separate server, we wouldn't have to
cope w/ actual storage formats, ldap, etc, within mozilla itself.
Another important point is moving out plugins to their own
processes.

We could learn very much from Plan9 here. I'm currently working
on a designing synthetic filesystem interfaces for those things:

* http://oss-qm.metux.de/index.php/9forge

> roc's suggestion in this thread is that each of the different 
> platforms has a specific means of hinting that a page is ready 
> for reuse, and that we should (on each platform individually)
> use those hinting procedures to improve our performance (with 
> respect to memory utilization).

Right. But all this should happen within (and hidden behind the 
interfaces of) some dedicated memory management library.
So maybe lets call it "libnewmalloc" and all the moz code
would now use libnewmalloc's functions instead of libc's.
And this lib won't be directly included within the moz tree, 
but instead *imported*.

> There's nothing inherently wrong with any of these OSs general 
> purpose allocators; we simply need better, more specialized, 
> behavior than they can provide. 

Yeah, you need new semantics, so an new interface and an new library.

> How can you possible argue for modularity in our application while 
> also criticizing glibc for being "Simply too large!", 

Easily. Glibc should be modularized too. But this is not our 
topic here (if you like, we can continue that issue on the 
proper lists ;-p)

> and then tell us that we should be trying to push the functionality 
> we need down into glibc? Why should it be glibc's problem to worry 
> about how mozilla/firefox use memory? 

As you already said above: communication problem. 
I thought you wanted to stay within malloc() semantics and we
were talking about those chunks already free()'ed, but not yet
given back to the OS.

Obviously that's not the case: we need an allocator which allows 
assigning additional attributes to (from the memory allocator's
view) still allocated chunks. That's an different quality, that
needs an own interface with new semantics.


cu
-- 
---------------------------------------------------------------------
 Enrico Weigelt    ==   metux IT service - http://www.metux.de/
---------------------------------------------------------------------
 Please visit the OpenSource QM Taskforce:
 	http://wiki.metux.de/public/OpenSource_QM_Taskforce
 Patches / Fixes for a lot dozens of packages in dozens of versions:
	http://patches.metux.de/
---------------------------------------------------------------------