Re: Cache refresh each 50 queries?

Hi.

On 04.10.2016 10:17, SUZUKI Arthur wrote:
[...]

>
> On 03.10.2016 16:57, SUZUKI Arthur wrote:
>>> Hello List,
>>> I'm sending this message to know if there are some hints/tips to help with the problem
>>> we're facing.
>>> The problem is that for a same query repeated over time,
>>> reply time can be as short as 5ms and as long as 1s.
>>>
>>> Since there is neither correlation with CPU load or RAM usage, nor any networking
>>> constraints,
>>> we think it is due to the cache refresh mechanism.
>>>
>>> The problem occurs at random times, with a probability of around 1/50 queries (empirical
>>> data).
>>>
>>> Is there any configuration option which could help?
>>>
>>> Our current httpd.conf contains the following:
>>>
>>> PerlModule ModPerl::Registry
>>> PerlOptions +Parent
>>> PerlSwitches -I/home/koha/src
>>>
>>> <Files "ilsdi.pl">
>>>    SetHandler perl-script
>>>    # more faster, link with worker
>>>    PerlResponseHandler ModPerl::Registry
>>>    # less faster, link with prefork
>>>    #PerlResponseHandler ModPerl::PerlRun
>>>    Options +ExecCGI
>>>    PerlOptions +ParseHeaders
>>> </Files>
>>>
>>> Thanks a lot in advance for your replies.
>>> Best regards,
>

You still do not provide enough information about your server configuration (the part not 
mod_perl specific) to really make a narrow guess, but let me try anyway :

One thing that might explain a seemingly-long processing time for a request approximately 
once in 50 calls, is if there was something special and resource-intensive which happens 
once per approximately 50 calls, right ?

One thing which /might/ explain this, /perhaps/ and depending on your configuration, is if 
approximately once in 50 calls, Apache had to fork a new child process, to handle the 
call. This new child process would then start with a brand-new, fresh Perl interpreter, 
which might need to recompile some modules before it even starts handling the request.

Look for example here : http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_
and your configuration line above :
  PerlOptions +Parent

In that configuration, the cgi-bin script which you are running (and all its 
dependencies), is not compiled by the embedded perl until it has been run at least once. 
So the first execution will take more time than the following ones.
Now if the rest of your configuration is so that every 50 requests or so, Apache starts a 
new child with a new perl interpreter, that may explain the symptoms that you are observing.

When Apache handles a request (if it is a "prefork" MPM), it will look for a child Apache 
which is free, to pass the request for execution.  If the server is lightly loaded, it may 
be that the child to which it passes the next request is always the same, because it is 
always done with the previous request, and free, when the next request comes in.
But if this child process somehow has a limit to how many requests it may process before 
it dies, and this limit was around 50, then somewhere around the 50th request, another 
child would get the next request, and for this child it would be the first one (or the 
first one with /this/ cgi-bin script).

There are roughly similar phenomenons that might happen in other kinds of Apache MPM's, 
which is why it is important to know which one you are using, and with which setup parameters.

And yes, there are ways to improve this.  But again, we would need to know more about your 
configuration in order to make suggestions.