Re: bloatware

> Sender: [email protected]
> From: Martin Konold <[email protected]>

material elided.

>
> > Various stuff has bloated over the years, but it is generally on the
> > applications side.  Sometimes you think the X server is bloating, when it
> > is really that applications have a pixmap leak, for example.
>
> In order to show that X is bloated I generally kill all apps except a single
> xterm and then check the memory usage.
>

To begin with, what you see is VM; if the memory is never referenced
again, it is fine for it to page out.

X has to allocate space for all the memory clients ask.  There is no good
way to reclaim it right now.  Your VM usage of the X server will be:
  o the size of X itself
  o however much I/O space is required to map the hardware registers and VRAM
  o the heap used for allocation of X data structures and Pixmaps.

So what you are seeing is ***the high water mark of what all the bloatware
has asked the X server on their behalf*** added up to.

Ok, what can be done about this?

A number of things...

1) We can try, as a stop-gap, to see if cranking down the MMAP threshold
   in glibc (the point at which glibc calls mmap to allocate pages of
   memory to satisfy allocations) helps.  Then if many pixmaps are larger
   than that threshold, at least the VM gets returned when the pixmaps
   get freed.  But we can't just set this arbitrarily low, as memory
   allocation is very performance sensitive in respect to any windowing
   operation (the region code in the X server pounds on malloc/free),
   and a few apps do some amazing things with regions.

2) If we separate pixmap allocation from the general heap, it would be
   easy to have the X server do a copying garbage collector and reclaim
   large amounts of VM, as all pixmap access is through internal data
   structures.  This would also avoid large amounts of heap fragmentation
   where window data structures currently get scattered through the VM of
   the X server (around the much larger pixmap allocations), and is
   worth doing anyway as you'd like pixmap allocation to
   be something a device driver can do anyway (so they'd be out in VRAM
   when appropriate).   When clients exit, it would probably then
   be worth copying pixmaps around to compact the virtual memory address
   space, and one could then free the left over pages easily, returning
   VM to the operating system.

I was in the middle of 1) when things recently blew up, so I don't have
anything to report yet; maybe something in a few weeks.  2) will take
more work, and if someone is interested, please let us (Keith and I) know;
it shouldn't be too hard to implement.

All this was much less of a concern when displays were typically 1 or
8 bits/pixel, but these days most displays are 16 or 32 bits/pixel,
compounding the heap usage problem.

                                    - Jim
--
Jim Gettys
Cambridge Research Laboratory
HP Labs, Hewlett-Packard Company
[email protected]