Re: GC leaks debugging

On Sat, Apr 9, 2011 at 12:55 PM, Erik Groeneveld <[email protected]> wrote:
> On Sat, Apr 9, 2011 at 3:14 AM, Boehm, Hans <[email protected]> wrote:
>
> I only kept the last dump of GC_DUMP_REGULARLY because they became GBs
> big.  It is attached.
>
>> That test program is pushing the envelope in a couple of different ways:
>>
>> 1. It potentially causes lots of fragmentation if some of those objects are not immediately reclaimed.

I wrote a little Python script to analyze the heap dump from GC_dump
(attached earlier).  It finds all large blocks like:

 0x7f8198277000 size 180224 not black listed

then sorts them on the address, and scans it all to see if they are adjacent.

None of the 13,000+ blocks are.

It also calculates the gaps between the blocks and adds them.  It
leaves out the 13 largest gaps as they are likely to represent
different heap sections and not GC allocated space.  It adds up to 296
Mb, roughly the amount of space in use by the program according to
GC_dump, 273 Mb.

So indeed there is a lot fragmentation...

Any ideas about how to fight this?

I believe that I am now able to revisit my earlier suspicion about the
GC losing control by not collecting enough.  If it did collect more
often, it could avoid fragmentation.  (Is that true?) But now, when
fragmentation grows, it grows the heap and collects even less
frequently, thus allowing for more fragmentation.  And so on.  It this
a possible scenario?

Erik