Re: ideas for returning memory to the OS

> Uh, tricky issue.
>
> IMHO, memory allocation should be (exclusively) done by libc, because
> it's optimized to the OS/kernel and provides an (quite;-o) portable
> interface. If an certain libc isn't yet good enough, improve *this*
> instead of trying to work around that in individual applications.

This isn't that tricky an issue.  The general purpose allocators of
each of the top-tier operating systems provide these routines so that
apps that understand their memory needs -can- be tuned more
optimally.  The library (libc, you suggest, though in reality there
are non-open-source libraries involved on at least one of our
platforms) does indeed, as you suggest, provide a general-purpose
allocation facility well-tuned to the platform.  It does not, however,
provide specialized allocators well-tuned to the application.

> I don't think, unmapping is that great deal, it just saves some
> pagetable entries. BUT: it's important to keep fragmentation within
> pages low. The key point is trying to put as much traffic as you can
> to an as small as possible set of pages (so reuse where you can;-))
> and the kernel will handle the rest.
>
> Unused pages lurking around and not being accessed for longer time are
> no problem, since they will just silently sleep in swap.
>

Swap is certainly not unlimited, and whether or not it is technically
sound criticism or not, Firefox is often criticized for consuming lots
of memory.  Whether that memory is actually in use or actually
impacting the performance of the system doesn't matter.  Our users
(sometimes) perceive us as being memory-hungry.  If we can easily
assuage those concerns, let's.

> > In fact all three of our main platforms have a way to tell the OS
> > that the contents of a committed page are no longer needed, need
> > not be paged out, so the RAM can be be used by another process,
> > but without messing around with munmap and VMA fragmentation!
>
> Right, that avoids unnecessary swap-out (and later swap-in) for
> unused pages. An sane malloc() should be expected to already
> utilize this.
>

Evidently, they don't; see Stuart's several blog posts on the subject.

> No, I don't think, mozilla's code should do anything in that area.
> Please fix it at the source -> libc.
>

We aren't the keepers of the libc code (not even glibc), and certainly
can't drive the development of msvcrt.  How do you propose we solve
these issues (even if they are issues of perception) on Windows -- the
platform the vast majority of our users are running firefox on?

-- crowder