Re: Memory fragmentation?

Colin Barrett wrote:

> One thing that wasn't made super clear, though Daniel did touch on it, 
> was that when we all of the memory on a page has been freed, that page 
> can be reclaimed by the OS.

Well, it's a little more complicated than that, and depends on what one 
means by "reclaimed by the OS." A simplistic example:

Suppose your fragmented heap consists of 4096 1-byte allocations, and 
each is on a separate 4K page. That's 16MB of address space (4096*4K), 
and if you're touching each of those 1 byte values the OS needs to keep 
everything in 16MB of real memory. That's bad.

Now allocate an 8K region on your heap, and you'll need to touch two 
more pages because there's no contiguous free space big enough. Also 
kind of bad.

Here's the tricky bit: if you free those 4096 1-byte buffers, the VM 
size of your heap won't change (still 16MB+8KB): it only shrinks from 
the top, and is blocked by that 8K allocation. This isn't really a 
concern; you're not touching that 16MB of memory so it'll get paged out 
and not mapped to real memory.

If you then free that 8KB chunk, the malloc implementation could 
potentially tell the OS to shrink the heap's VM region (which would make 
the process's reported VM size decrease). But afaik not all 
implementations do that, and not all OSes support it.

Justin