Re: Cache refresh each 50 queries?

Hello David, André,

the server we're working with is fully dedicated to Koha.

Thank you André for your hints about Apache configurations.

@David, same thing, I'll try to preload C4::ILSDI:Services and see if it 
helps.

Thanks to both of you, I'll let you know about the results.
Cheers,
Arthur

Le 05/10/2016 à 01:20, David Cook a écrit :
> Sounds like André is on the right track. I've certainly run into a similar
> issue (with a non-Koha app).
>
> However, because I was using the Catalyst framework, I was able to just
> preload the entire app, so that the Perl modules were loaded into the Apache
> master process before forking, and that did the trick. That app is a lot
> smaller than Koha though too. This case is a bit more complicated since Koha
> isn't really a MVC app, but you could look at the Koha Plack examples and
> see which modules they pre-load.
>
> You might also want to run ilsdi.pl with Devel::NYTProf to identify what
> exactly is slowing down that 1s long query.
>
> I've only played with PerlOptions +Parent a bit, but I'm guessing that you
> have multiple mod_perl apps, and that we're not actually seeing your entire
> httpd.conf configuration relating to Koha, right?
>
> David Cook
> Systems Librarian
> Prosentient Systems
> 72/330 Wattle St
> Ultimo, NSW 2007
> Australia
>
> Office: 02 9212 0899
> Direct: 02 8005 0595
>
>
>> -----Original Message-----
>> From: André Warnier [mailto:[email protected]]
>> Sent: Tuesday, 4 October 2016 10:02 PM
>> To: [email protected]
>> Subject: Re: Cache refresh each 50 queries?
>>
>> Hi.
>>
>> On 04.10.2016 10:17, SUZUKI Arthur wrote:
>> [...]
>>
>>> On 03.10.2016 16:57, SUZUKI Arthur wrote:
>>>>> Hello List,
>>>>> I'm sending this message to know if there are some hints/tips to
>>>>> help with the problem we're facing.
>>>>> The problem is that for a same query repeated over time, reply time
>>>>> can be as short as 5ms and as long as 1s.
>>>>>
>>>>> Since there is neither correlation with CPU load or RAM usage, nor
>>>>> any networking constraints, we think it is due to the cache refresh
>>>>> mechanism.
>>>>>
>>>>> The problem occurs at random times, with a probability of around
>>>>> 1/50 queries (empirical data).
>>>>>
>>>>> Is there any configuration option which could help?
>>>>>
>>>>> Our current httpd.conf contains the following:
>>>>>
>>>>> PerlModule ModPerl::Registry
>>>>> PerlOptions +Parent
>>>>> PerlSwitches -I/home/koha/src
>>>>>
>>>>> <Files "ilsdi.pl">
>>>>>     SetHandler perl-script
>>>>>     # more faster, link with worker
>>>>>     PerlResponseHandler ModPerl::Registry
>>>>>     # less faster, link with prefork
>>>>>     #PerlResponseHandler ModPerl::PerlRun
>>>>>     Options +ExecCGI
>>>>>     PerlOptions +ParseHeaders
>>>>> </Files>
>>>>>
>>>>> Thanks a lot in advance for your replies.
>>>>> Best regards,
>> You still do not provide enough information about your server
> configuration
>> (the part not mod_perl specific) to really make a narrow guess, but let me
> try
>> anyway :
>>
>> One thing that might explain a seemingly-long processing time for a
> request
>> approximately once in 50 calls, is if there was something special and
>> resource-intensive which happens once per approximately 50 calls, right ?
>>
>> One thing which /might/ explain this, /perhaps/ and depending on your
>> configuration, is if approximately once in 50 calls, Apache had to fork a
> new
>> child process, to handle the call. This new child process would then start
> with
>> a brand-new, fresh Perl interpreter, which might need to recompile some
>> modules before it even starts handling the request.
>>
>> Look for example here :
>> http://perl.apache.org/docs/2.0/user/config/config.html#C_Parent_
>> and your configuration line above :
>>    PerlOptions +Parent
>>
>> In that configuration, the cgi-bin script which you are running (and all
> its
>> dependencies), is not compiled by the embedded perl until it has been run
> at
>> least once.
>> So the first execution will take more time than the following ones.
>> Now if the rest of your configuration is so that every 50 requests or so,
>> Apache starts a new child with a new perl interpreter, that may explain
> the
>> symptoms that you are observing.
>>
>> When Apache handles a request (if it is a "prefork" MPM), it will look for
> a
>> child Apache which is free, to pass the request for execution.  If the
> server is
>> lightly loaded, it may be that the child to which it passes the next
> request is
>> always the same, because it is always done with the previous request, and
>> free, when the next request comes in.
>> But if this child process somehow has a limit to how many requests it may
>> process before it dies, and this limit was around 50, then somewhere
> around
>> the 50th request, another child would get the next request, and for this
> child
>> it would be the first one (or the first one with /this/ cgi-bin script).
>>
>> There are roughly similar phenomenons that might happen in other kinds of
>> Apache MPM's, which is why it is important to know which one you are
> using,
>> and with which setup parameters.
>>
>> And yes, there are ways to improve this.  But again, we would need to know
>> more about your configuration in order to make suggestions.
>
>

-- 
Arthur SUZUKI
Service informatique des bibliothèques
BIBLIOTHÈQUES UNIVERSITAIRES
Université Jean Moulin Lyon 3
6 Cours Albert Thomas - B.P. 8242 – 69355 Lyon Cedex 08
ligne directe : +33 (0)4 78 78 79 16 | http://bu.univ-lyon3.fr
L'Université Jean Moulin est membre fondateur de l'Université de Lyon