RE: Moving ExecCGI to mod_perl - performance and custom 'modules' [EXT]
Steven Haigh <[email protected]>
| Newsgroups | gmane.comp.apache.mod-perl |
|---|---|
| Message-ID | <[email protected]> |
In fact, I just realised that 'ab' test is rather restrictive.... So here's a bit more of an extended test: # ab -k -n 1000 -c 32 Apache + ExecCGI: Requests per second: 14.26 [#/sec] (mean) Time per request: 2244.181 [ms] (mean) Time per request: 70.131 [ms] (mean, across all concurrent requests) Apache + mod_perl (ModPerl::PerlRegistry): Requests per second: 132.14 [#/sec] (mean) Time per request: 242.175 [ms] (mean) Time per request: 7.568 [ms] (mean, across all concurrent requests) Interestingly, without Keepalives, the story is much the same: # ab -n 1000 -c 32 Apache + ExecCGI: Requests per second: 14.15 [#/sec] (mean) Time per request: 2260.875 [ms] (mean) Time per request: 70.652 [ms] (mean, across all concurrent requests) Apache + mod_perl (ModPerl::PerlRegistry): Requests per second: 154.48 [#/sec] (mean) Time per request: 207.140 [ms] (mean) Time per request: 6.473 [ms] (mean, across all concurrent requests) Running some benchmarks across various parts of my site made me realise I also had some RewriteRules in the apache config that still had H=cgi-script - changed those to H=perl-script and saw similar improvements: ExecCGI - Requests per second: 11.84 [#/sec] (mean) mod_perl - Requests per second: 130.97 [#/sec] (mean) That's quite some gains for a days work. -- Steven Haigh 📧 [email protected] <mailto:[email protected]> 💻 https://www.crc.id.au <https://www.crc.id.au/> On Sun, Feb 7, 2021 at 23:58, Steven Haigh <[email protected]> wrote: > Interestingly, I did get things working with ModPerl::PerlRegistry. > > What I couldn't find *anywhere* is that the data I was loading in > Template Toolkit was included in the file in the __DATA__ area - > which causes mod_perl to fall over! > > The only way I managed to find this was the following error in the > *system* /var/log/httpd/error_log (didn't show up in the vhost > error_log!): > readline() on unopened filehandle DATA at > /usr/lib64/perl5/vendor_perl/Template/Provider.pm line 638. > > Took me a LONG time to find a vague post that reading in lines from > <DATA> kills mod_perl. Not sure why - but I stripped all the > templates out and put them in a file instead and re-wrote that bit of > code, and things started working. > > I had to fix a few lib path issues, but after getting my head around > that, most things seem to work as before - however I don't notice > much of an improvement in execution times, I do see this improvement > using 'ab -n 100 -c32': > > Apache + ExecCGI: Requests per second: 13.50 [#/sec] (mean) > Apache + mod_perl: Requests per second: 59.81 [#/sec] (mean) > > This is obviously a good thing. > > I haven't gotten into the preload or DBI sharing yet - as that'll end > up needing a bit of a rewrite of code to take advantage of. I'd be > open to suggestions here from those who have done it in the past to > save me going down some dead ends :D > > -- > Steven Haigh > > 📧 [email protected] <mailto:[email protected]> > 💻 https://www.crc.id.au <https://www.crc.id.au/> > > On Sun, Feb 7, 2021 at 12:49, James Smith <[email protected]> wrote: >> As welsey said – try Registry, that was the standard way of using >> mod_perl to cache perl in the server – but your problem might be >> due to the note in PerlRun… >> >> <https://perl.apache.org/docs/2.0/api/ModPerl/PerlRun.html#Description> >> META: document that for now we don't chdir() into the script's dir, >> because it affects the whole process under threads. >> ModPerl::PerlRunPrefork >> <https://perl.apache.org/docs/20/api/ModPerl/PerlRunPrefork.html> >> should be used by those who run only under prefork MPM. >> {tbh most people don’t use mod perl under threads anyway as there >> isn’t really a gain from using them} >> >> It suggests you use ModPerl/PerlRunPrefork – as this does an >> additional step to cd to the script directory – which might be >> your issue…. >> >> >> >> *From:*Steven Haigh <[email protected]> >> *Sent:* 07 February 2021 01:00 >> *To:* [email protected] >> *Subject:* Moving ExecCGI to mod_perl - performance and custom >> 'modules' [EXT] >> >> >> >> Hi all, >> >> >> >> So for many years I've been slack and writing perl scripts to do >> various things - but never needed more than the normal apache >> +ExecCGI and Template Toolkit. >> >> >> >> One of my sites has become a bit more popular, so I'd like to spend >> a bit of time on performance. Currently, I'm seeing ~300-400ms of >> what I believe to be execution time of the script loading, running, >> and then blatting its output to STDOUT and the browser can go do its >> thing. >> >> >> >> I believe most of the delay would be to do with loading perl, its >> modules etc etc >> >> >> >> I know that the current trend would be to re-write the entire site >> in a more modern, daemon based solution - and I started down the >> Mojolicious path - but the amount of re-writing to save 1/3rd of a >> second seems to be excessive >> >> >> >> Would I be correct in thinking that mod_perl would help in this case? >> >> >> >> I did try a basic test, but I have a 'use functions' in all my >> scripts that loads a .pm with some global vars and a lot of common >> subs - and for whatever reason (can't find anything on Google as to >> why), none of the subs are recognised in the main script when loaded >> via ModPerl::PerlRun. >> >> >> >> So throwing it out to the list - am I on the right track? wasting my >> time? or just a simple mistake? >> >> >> >> -- >> >> Steven Haigh 📧[email protected] >> <mailto:[email protected]>💻https://www.crc.id.au [crc.id.au] >> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.crc.id.au_&d=DwMFaQ&c=D7ByGjS34AllFgecYw0iC6Zq7qlm8uclZFI0SqQnqBo&r=oH2yp0ge1ecj4oDX0XM7vQ&m=bosoTbkecbnrPukObNK-5Duc1p3JTllIM7_FHhBYKW4&s=vQDi0ezyEZDOz86GVraerPdT76UjN2in3UdPh8fglRM&e=> >> >> -- The Wellcome Sanger Institute is operated by Genome Research >> Limited, a charity registered in England with number 1021457 and a >> company registered in England with number 2742969, whose registered >> office is 215 Euston Road, London, NW1 2BE.