Re: mod_perl memory

You could use Apache2::SizeLimit ("because size does matter") which  
evaluates the size of Apache httpd processes when they complete HTTP  
Requests, and kills those that grow too large. (Note that  
Apache2::SizeLimit can only be used for non-threaded MPMs, such as  
prefork.) Since it operates at the end of a Request, SizeLimit has the  
advantage that it doesn't interrupt Request processing and the  
disadvantage that it won't prevent a process from becoming oversized  
while processing a Request. To reduce the regular load of  
Apache2::SizeLimit it can be configured to check the size  
intermittently by setting the parameter CHECK_EVERY_N_REQUESTS. These  
parameters can be configured in a <Perl> section in httpd.conf, or a  
Perl start-up file.

That way, if your script allocates too much memory the process will be  
killed when it finishes handling the request. The MPM will eventually  
start another process if necessary.

BR
A

On Mar 16, 2010, at 9:30 AM, William T wrote:

> On Mon, Mar 15, 2010 at 11:26 PM, Pavel Georgiev <[email protected]>  
> wrote:
>> I have a perl script running in mod_perl that needs to write a  
>> large amount of data to the client, possibly over a long period.  
>> The behavior that I observe is that once I print and flush  
>> something, the buffer memory is not reclaimed even though I rflush  
>> (I know this cant be reclaimed back by the OS).
>>
>> Is that how mod_perl operates and is there a way that I can force  
>> it to periodically free the buffer memory, so that I can use that  
>> for new buffers instead of taking more from the OS?
>
> That is how Perl operates.  Mod_Perl is just Perl embedded in the
> Apache Process.
>
> You have a few options:
>  * Buy more memory. :)
>  * Delegate resource intensive work to a different process (I would
> NOT suggest a forking a child in Apache).
>  * Tie the buffer to a file on disk, or db object, that can be
> explicitly reclaimed
>  * Create a buffer object of a fixed size and loop.
>  * Use compression on the data stream that you read into a buffer.
>
> You could also architect your system to mitigate resource usage if the
> large data serve is not a common operation:
>  * Proxy those requests to a different server which is optimized to
> handle large data serves.
>  * Execute the large data serves with CGI rather than Mod_Perl.
>
> I'm sure there are probably other options as well.
>
> -wjt
>

Arthur P. Goldberg, PhD

Research Scientist in Bioinformatics
Plant Systems Biology Laboratory
www.virtualplant.org

Visiting Academic
Computer Science Department
Courant Institute of Mathematical Sciences
www.cs.nyu.edu/artg

[email protected]
New York University
212 995-4918
Coruzzi Lab
8th Floor Silver Building
1009 Silver Center
100 Washington Sq East
New York NY 10003-6688