Re: bloatware

On Sun, Mar 30, 2003 at 01:06:57PM -0800, [email protected] wrote:
> > Sender: [email protected]
> > From: Martin Konold <[email protected]>
> 
> material elided.
> 
> >
> > > Various stuff has bloated over the years, but it is generally on the
> > > applications side.  Sometimes you think the X server is bloating, when it
> > > is really that applications have a pixmap leak, for example.
> >
> > In order to show that X is bloated I generally kill all apps except a single
> > xterm and then check the memory usage.
> >
> 
> To begin with, what you see is VM; if the memory is never referenced
> again, it is fine for it to page out.
> 
> X has to allocate space for all the memory clients ask.  There is no good
> way to reclaim it right now.  Your VM usage of the X server will be:
>   o the size of X itself
>   o however much I/O space is required to map the hardware registers and VRAM
>   o the heap used for allocation of X data structures and Pixmaps.
> 
> So what you are seeing is ***the high water mark of what all the bloatware
> has asked the X server on their behalf*** added up to.
> 
> Ok, what can be done about this?
> 
> A number of things...
> 
> 1) We can try, as a stop-gap, to see if cranking down the MMAP threshold
>    in glibc (the point at which glibc calls mmap to allocate pages of
>    memory to satisfy allocations) helps.  Then if many pixmaps are larger
>    than that threshold, at least the VM gets returned when the pixmaps
>    get freed.  But we can't just set this arbitrarily low, as memory
>    allocation is very performance sensitive in respect to any windowing
>    operation (the region code in the X server pounds on malloc/free),
>    and a few apps do some amazing things with regions.
> 
> 2) If we separate pixmap allocation from the general heap, it would be
>    easy to have the X server do a copying garbage collector and reclaim

Yes, that would be a good idea.

>    large amounts of VM, as all pixmap access is through internal data
>    structures.  This would also avoid large amounts of heap fragmentation
>    where window data structures currently get scattered through the VM of
>    the X server (around the much larger pixmap allocations), and is
>    worth doing anyway as you'd like pixmap allocation to
>    be something a device driver can do anyway (so they'd be out in VRAM
>    when appropriate).   When clients exit, it would probably then
>    be worth copying pixmaps around to compact the virtual memory address
>    space, and one could then free the left over pages easily, returning
>    VM to the operating system.

It could be used with the onchip memory manager too, could it not, and
possibly with the new DRI memory manager also ?

> I was in the middle of 1) when things recently blew up, so I don't have
> anything to report yet; maybe something in a few weeks.  2) will take
> more work, and if someone is interested, please let us (Keith and I) know;
> it shouldn't be too hard to implement.

Well, i would assuredly be interested in doing something like that.
Altough i have not yet done any GC work, i maintain the debian package
of ocaml, have good contact with ocaml's upstream and work with
functional programing. The ocaml GC is considered to be one of the best
around, it is a copy and sweep garbage collector, with a separation for
short time data and longer lived data.

I don't think that you could transpose the same concepts used for a GC
for a functional language, where data is often same sized, rather small,
and there are part of them who are extremely short lived (local
variables and such). It would be worth to study pixmap usage trace or
something such, but i believe that things work differently for a pixmap
GC, and additionnaly, since you are using low level programing languages
and have to maintain binary compatibility, it is not as easy to move the
pixmaps around, isn't it ?

That said, i don't think that writing a good GC for this case is an easy
task, writing good GC is mostly a difficult task, especially if you want
it to be efficient.

Anyway, as said, i am interrested, i have not much time right now for
it, but this could change nextly.

Friendly,

Sven Luther