Re: mod_perl memory

On Mon, Mar 15, 2010 at 11:26 PM, Pavel Georgiev <[email protected]> wrote:
> I have a perl script running in mod_perl that needs to write a large amount of data to the client, possibly over a long period. The behavior that I observe is that once I print and flush something, the buffer memory is not reclaimed even though I rflush (I know this cant be reclaimed back by the OS).
>
> Is that how mod_perl operates and is there a way that I can force it to periodically free the buffer memory, so that I can use that for new buffers instead of taking more from the OS?

That is how Perl operates.  Mod_Perl is just Perl embedded in the
Apache Process.

You have a few options:
  * Buy more memory. :)
  * Delegate resource intensive work to a different process (I would
NOT suggest a forking a child in Apache).
  * Tie the buffer to a file on disk, or db object, that can be
explicitly reclaimed
  * Create a buffer object of a fixed size and loop.
  * Use compression on the data stream that you read into a buffer.

You could also architect your system to mitigate resource usage if the
large data serve is not a common operation:
  * Proxy those requests to a different server which is optimized to
handle large data serves.
  * Execute the large data serves with CGI rather than Mod_Perl.

I'm sure there are probably other options as well.

-wjt