Re: avoiding child death by size limit

E R wrote:
> Hi,
> 
> I have a problem where a mod_perl handler will allocate a lot of
> memory when processing a request, and this causes Apache to kill the
> child due to exceeding the configure child size limit.
> 
> However, the memory allocated will get freed up or re-used by the next
> request - I think the memory is just fragmented enough to be
> automatically reclaimed by the memory allocator (I've heard that some
> mallocs can return memory to the OS in 1 MB chunks.)
> 
> Are there any special techniques people use to avoid this situation?
> Does SizeLimit count actual memory used or does it just look at the
> process size?
> 
This is not a direct answer to your question, and it begs for a more 
authoritative answer.

I have had problems similar to yours, which I solved by turning what was 
original part of my mod_perl handler (or script), into a separate 
process.  I know it is not elegant, but it seems to work well in my case.
One of the problems is that, as far as I know, once perl has obtained 
some memory from the OS, it will never give it back until perl itself 
exits.  And since by definition with mod_perl normally the perl 
interpreter lives as long as the Apache process it is inside of, that 
means almost never.
But if perl runs as an external process for a request, then of course it 
exits at the end of it, and the memory is returned.

The exact problem I had, was that some processing I had to do involved 
parsing XML, and some XML parsing module in the chain (XML::Twig ?) was 
leaking a chunk of memory at each request. I ended up with 
multi-megabyte Apache children all the time.
So I off-loaded this parsing into a separate process, which wrote (to 
disk) its results in the form of a Storable structure.  When the 
external process was done, the main mod_perl handler sucked in the data 
back from the Storable file and deleted it.
Not elegant, but it's been working flawlessly for several years now.