RE: Cache refresh each 50 queries?

Sounds like André is on the right track. I've certainly run into a similar
issue (with a non-Koha app).

However, because I was using the Catalyst framework, I was able to just
preload the entire app, so that the Perl modules were loaded into the Apache
master process before forking, and that did the trick. That app is a lot
smaller than Koha though too. This case is a bit more complicated since Koha
isn't really a MVC app, but you could look at the Koha Plack examples and
see which modules they pre-load. 

You might also want to run ilsdi.pl with Devel::NYTProf to identify what
exactly is slowing down that 1s long query.

I've only played with PerlOptions +Parent a bit, but I'm guessing that you
have multiple mod_perl apps, and that we're not actually seeing your entire
httpd.conf configuration relating to Koha, right? 

David Cook
Systems Librarian
Prosentient Systems
72/330 Wattle St
Ultimo, NSW 2007
Australia

Office: 02 9212 0899
Direct: 02 8005 0595


> -----Original Message-----
> From: André Warnier [mailto:[email protected]]
> Sent: Tuesday, 4 October 2016 10:02 PM
> To: [email protected]
> Subject: Re: Cache refresh each 50 queries?
> 
> Hi.
> 
> On 04.10.2016 10:17, SUZUKI Arthur wrote:
> [...]
> 
> >
> > On 03.10.2016 16:57, SUZUKI Arthur wrote:
> >>> Hello List,
> >>> I'm sending this message to know if there are some hints/tips to
> >>> help with the problem we're facing.
> >>> The problem is that for a same query repeated over time, reply time
> >>> can be as short as 5ms and as long as 1s.
> >>>
> >>> Since there is neither correlation with CPU load or RAM usage, nor
> >>> any networking constraints, we think it is due to the cache refresh
> >>> mechanism.
> >>>
> >>> The problem occurs at random times, with a probability of around
> >>> 1/50 queries (empirical data).
> >>>
> >>> Is there any configuration option which could help?
> >>>
> >>> Our current httpd.conf contains the following:
> >>>
> >>> PerlModule ModPerl::Registry
> >>> PerlOptions +Parent
> >>> PerlSwitches -I/home/koha/src
> >>>
> >>> <Files "ilsdi.pl">
> >>>    SetHandler perl-script
> >>>    # more faster, link with worker
> >>>    PerlResponseHandler ModPerl::Registry
> >>>    # less faster, link with prefork
> >>>    #PerlResponseHandler ModPerl::PerlRun
> >>>    Options +ExecCGI
> >>>    PerlOptions +ParseHeaders
> >>> </Files>
> >>>
> >>> Thanks a lot in advance for your replies.
> >>> Best regards,
> >
> 
> You still do not provide enough information about your server
configuration
> (the part not mod_perl specific) to really make a narrow guess, but let me
try
> anyway :
> 
> One thing that might explain a seemingly-long processing time for a
request
> approximately once in 50 calls, is if there was something special and
> resource-intensive which happens once per approximately 50 calls, right ?
> 
> One thing which /might/ explain this, /perhaps/ and depending on your
> configuration, is if approximately once in 50 calls, Apache had to fork a
new
> child process, to handle the call. This new child process would then start
with
> a brand-new, fresh Perl interpreter, which might need to recompile some
> modules before it even starts handling the request.
> 
> Look for example here :
> http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_
> and your configuration line above :
>   PerlOptions +Parent
> 
> In that configuration, the cgi-bin script which you are running (and all
its
> dependencies), is not compiled by the embedded perl until it has been run
at
> least once.
> So the first execution will take more time than the following ones.
> Now if the rest of your configuration is so that every 50 requests or so,
> Apache starts a new child with a new perl interpreter, that may explain
the
> symptoms that you are observing.
> 
> When Apache handles a request (if it is a "prefork" MPM), it will look for
a
> child Apache which is free, to pass the request for execution.  If the
server is
> lightly loaded, it may be that the child to which it passes the next
request is
> always the same, because it is always done with the previous request, and
> free, when the next request comes in.
> But if this child process somehow has a limit to how many requests it may
> process before it dies, and this limit was around 50, then somewhere
around
> the 50th request, another child would get the next request, and for this
child
> it would be the first one (or the first one with /this/ cgi-bin script).
> 
> There are roughly similar phenomenons that might happen in other kinds of
> Apache MPM's, which is why it is important to know which one you are
using,
> and with which setup parameters.
> 
> And yes, there are ways to improve this.  But again, we would need to know
> more about your configuration in order to make suggestions.