Re: [Gc] Is there a way to walk the entire heap of live objects

Apologies if this is a duplicate - I ran into trouble with the size due to a graph I included.

I used the code that Peter sent me as-is - it works very well - thanks!

I’m writing a Common Lisp implementation that interoperates with C++ and uses LLVM as the back end.  I’ve written a static analyzer for my C++ code to help me clean it up and add features. When I run the static analyzer on the 165 C++ source files using the Boehm GC it blows up.  

At the end of processing 165 source files the process consumes about 30GB (gigabytes) of memory.

So I run the static analyzer on one source file 10 times and use the code that Peter sent me and walk the reachable objects and add up their memory footprint.   There are a handful of objects that don’t have valid headers - I filter them out and currently don’t count them. I find that the total reachable memory used by the system remains pretty constant at 62MB to 34MB (it spikes and then goes down). 

But the total memory used by my process goes up and up and up very quickly!

I ran the OS X program “heap” on the process and it reports “non-object” heap memory - which I assume is allocated by the Boehm GC (this assumption may be wrong - please correct me if you know better).

Here is a graph that shows all of the memory usage each time after parsing/generating an AST/searching the AST for the same C++ file. A bit of explanation… The Y axis is log(Bytes)  The red points are what “heap” reports as “non-object” memory - I assume this is total Boehm memory. The green points are the total reachable Boehm memory that has a valid header that I built.   The blue points are the reachable Boehm memory that was allocated just since the start of loading the most recent C++ file. I measure this by writing an integer marker into the header of each newly allocated object and changing that marker each time I read the source file - this lets me track what objects are allocated since the last operation.

Graph link: http://imgur.com/tSN5jRL



This can’t be memory fragmentation can it?      
Is Boehm not reusing memory properly?   
Am I not configuring Boehm properly?