Re: ### Allocation failure
David Hakim <dhakim-Gkm/TONP9n1Wk0Htik3J/[email protected]> Tue, 22 Apr 2003 20:16:40 -0400
| Newsgroups | gmane.comp.lang.moto.devel |
|---|---|
| Message-ID | <[email protected]> |
On Wednesday, April 16, 2003, at 09:35 AM, Scott Rogers wrote:
> David,
>
> I ran a similiar test using the following PHP files:
>
> And received NO errors on Linux, with 417 requests per second and
> 708 errors on FreeBSD with 627 requests per second.
>
> So, there is some difference in the two boxes, whether it's
> configuration
> (memory?) or an OS difference, I can't say.
>
> Should I set a session variable in PHP to get an apples to apples
> comparision
> of sessions?
Because of differences in the meaning of sessions to Moto and PHP its
not entirely apples to apples. Creating a session in moto creates a
hashtable (actually a couple of them) in shared memory for storage of
session objects. Its all the hashtables that fill up the memory. 32M is
the default heap maximum, you can raise it with the SharedHeapSize
config option. My understanding is that PHP doesn't offer storage of
objects in memory between pageviews without some extra work.
I'm surprised to see PHP perform as well as it did actually :) Maybe
you could try making the test more complicated. Add a for loop around
the method call for instance. I'm wondering if the reason we see
performance as close as we do is because the highest order cost we are
looking at here is inside of httpd itself.
-Dave
>
>
>
> === BEGIN hello.php ===
> <?php
>
>
> require_once("my_classes.php");
>
> ?>
>
> <html>
> <head>
> <style type="text/css">
> </style>
> </head>
> <body>
>
> <hr>
> <?php
> $hw = new HelloWorld();
> $message = $hw->sayHelloTo("bob");
>
> echo "$message <br/>";
>
> echo $hw->sayHelloTo("bobie");
>
> echo "<hr>";
>
> $box = new Box("Loud People Out Front");
>
>
> echo $box->renderStandard()
>
> ?>
> <hr>
> </body>
> </html>
> === END hello.php ===
>
> === BEGIN my_classes.php ===
> <?php
> class HelloWorld
> {
> function sayHelloTo($name)
> {
> return "Hello $name ";
> }
> }
>
>
> class Box
> {
> var $_title = "";
>
> function Box($title)
> {
> $this->_title = $title;
> }
>
> function renderStandard()
> {
> return "Title: $this->_title";
> }
> }
> ?>
> === END my_classes.php ===
>
>
>
> On Wednesday 16 April 2003 03:25 pm, David Hakim wrote:
>> Cool! So you got 50,000 transactions through in right about 1 minute
>> ... not too shabby :) I don't like those failures on the FreeBSD box .
>> Its possible those are related to apache children dying and
>> re-spawning. If there are other apache modules you could run the ab
>> test on on FreeBSD I'd really like to see whether they fare any better
>> with regard to failure rate.
>>
>> I'd also be real curious how any concurrent tests you run with apache
>> bench change your results.
>>
>> -Dave
>>
>> On Wednesday, April 16, 2003, at 07:06 AM, Scott Rogers wrote:
>>> David,
>>>
>>> I changed my httpd.conf file as you suggested to turn off sessions.
>>> On both
>>> FreeBSD and Linux I experienced the same behavior as you.
>>>
>>> The Linux box reported 844 requests per second with no failures and
>>> the BSD box reported 1045 requests per second with 562 failures.
>>>
>>> So it does appear to be session related.
>>>
>>> Thanks.
>>>
>>> My httpd.conf below
>>> === BEGIN httpd.conf ===
>>> <IfModule mod_tmoto1.c>
>>> <Location /tmoto1>
>>> Tmoto1Option Location /tmoto1
>>> Tmoto1Option Session.Disable TRUE
>>> SetHandler tmoto1
>>> </Location>
>>> </IfModule>
>>> === END httpd.conf ===
>>>
>>> On Wednesday 16 April 2003 01:45 am, David Hakim wrote:
>>>> Interesting, I'm pretty confident this is Sessions building up. I
>>>> replicated the allocation failure with sessions enabled no problem
>>>> (yep, right around 6-7K transactions/sessions 32M is used up) ,
>>>> however
>>>> as soon as I disabled them I was able to run the ab test without
>>>> incident. Please post the relevant portions of your httpd.conf. Here
>>>> are mine for the compiled module on my httpd.conf (running on a g4
>>>> portable 500Mhz Mac OS 10.2.5 with lots-o-other-things running :) )
>>>>
>>>> =====httpd.conf====
>>>> LoadModule hello_module libexec/httpd/mod_hello.so
>>>>
>>>> <IfModule mod_hello.c>
>>>> <Location /hello>
>>>> HelloOption Location /hello
>>>> SetHandler hello
>>>> HelloOption Session.Disable TRUE
>>>> </Location>
>>>> </IfModule>
>>>> =================
>>>>
>>>> And here was my ab output:
>>>>
>>>> =================
>>>> % ab -t 0 'http://localhost/hello/hello.moto'
>>>> This is ApacheBench, Version 1.3d <$Revision: 1.1.1.9 $> apache-1.3
>>>> Copyright (c) 1996 Adam Twiss, Zeus Technology Ltd,
>>>> http://www.zeustech.net/
>>>> Copyright (c) 1998-2002 The Apache Software Foundation,
>>>> http://www.apache.org/
>>>>
>>>> Benchmarking localhost (be patient)
>>>> Completed 5000 requests
>>>> Completed 10000 requests
>>>> Completed 15000 requests
>>>> Completed 20000 requests
>>>> Completed 25000 requests
>>>> Completed 30000 requests
>>>> Completed 35000 requests
>>>> Completed 40000 requests
>>>> Completed 45000 requests
>>>> Finished 50000 requests
>>>> Server Software: Apache/1.3.27
>>>> Server Hostname: localhost
>>>> Server Port: 80
>>>>
>>>> Document Path: /hello/hello.moto
>>>> Document Length: 198 bytes
>>>>
>>>> Concurrency Level: 1
>>>> Time taken for tests: 249.111 seconds
>>>> Complete requests: 50000
>>>> Failed requests: 0
>>>> Broken pipe errors: 0
>>>> Total transferred: 16550000 bytes
>>>> HTML transferred: 9900000 bytes
>>>> Requests per second: 200.71 [#/sec] (mean)
>>>> Time per request: 4.98 [ms] (mean)
>>>> Time per request: 4.98 [ms] (mean, across all concurrent
>>>> requests)
>>>> Transfer rate: 66.44 [Kbytes/sec] received
>>>>
>>>> Connnection Times (ms)
>>>> min mean[+/-sd] median max
>>>> Connect: 0 1 NaN 0 56638
>>>> Processing: 2 3 7.7 2 800
>>>> Waiting: 0 2 7.1 2 800
>>>> Total: 2 4 NaN 2 56641
>>>>
>>>> Percentage of the requests served within a certain time (ms)
>>>> 50% 2
>>>> 66% 2
>>>> 75% 3
>>>> 80% 3
>>>> 90% 3
>>>> 95% 6
>>>> 98% 7
>>>> 99% 19
>>>> 100% 56641 (last request)
>>>>
>>>> Also, consider adding this script somewhere on your server so you
>>>> can
>>>> watch moto's memory, sessions and other persistent objects while
>>>> benchmarking:
>>>>
>>>> $use("codex.util")
>>>> $use("codex.http")
>>>>
>>>> <html>
>>>> <body bgcolor="#FFFFFF">
>>>> <table border=0 cellpadding=0>
>>>> <tr>
>>>> <td valign=top>
>>>> Total Memory = $(getTotalMemory() / (1024*1024 ))M
>>>> = $(getTotalMemory() / (1024 ))K
>>>> = $(getTotalMemory())B <br>
>>>> Free Memory = $(getFreeMemory() / (1024*1024 ))M
>>>> = $(getFreeMemory() / (1024 ))K
>>>> = $(getFreeMemory())B <br>
>>>> Used Memory = $((getTotalMemory()-getFreeMemory()) /
>>>> (1024*1024 ))M
>>>> = $((getTotalMemory()-getFreeMemory()) /
>>>> (1024 ))K
>>>> =
>>>> $((getTotalMemory()-getFreeMemory()))B<br>
>>>> Objects Allocated = $(getANodes()) <br>
>>>> Available Segments = $(getFNodes()) <br>
>>>> Total Context Objects =
>>>> $(getContext().getObjectCount())
>>>> <br>
>>>> Total Sessions = $(getContext().getSessionCount())
>>>> <br>
>>>> States Per Session =
>>>> $(getContext().getStatesPerSession()) <br>
>>>> Session Timeout =
>>>> $(getContext().getSessionTimeout())
>>>> <br>
>>>> </td><td
>>>> height="100"
>>>> width="50"
>>>> bgcolor="0000FF"
>>>> valign=bottom
>>>>
>>>>> <img
>>>>
>>>> border="0"
>>>> src="rbar.gif"
>>>> width="50"
>>>> height="$(100 -
>>>> <int>((<float>getFreeMemory()/getTotalMemory())*100))"
>>>>
>>>>> </td>
>>>>
>>>> <tr>
>>>> </table>
>>>> </body>
>>>> =================
>>>>
>>>> -Dave
>>>>
>>>> On Tuesday, April 15, 2003, at 11:45 AM, Scott Rogers wrote:
>>>>> Dave,
>>>>>
>>>>> I did as you asked. I turned off sessions for the compiled
>>>>> application and
>>>>> received the same result on the FreeBSD box. FYI I also modified
>>>>> codex/sharedmem.h to execute the 'debug()' function when the error
>>>>> occurs. So
>>>>> I have a 15mg httd error log with the tree dumps. I don't
>>>>> understand
>>>>> how to
>>>>> interperet the output however.
>>>>>
>>>>> I ran the app from the command line and received the following
>>>>> output:
>>>>>
>>>>> === BEGIN output ===
>>>>>
>>>>>> /usr/local/moto/bin/moto -d hello.moto
>>>>>
>>>>> /* */
>>>>>
>>>>>
>>>>>
>>>>> <html>
>>>>> <head>
>>>>> <style type="text/css">
>>>>> </style>
>>>>> </head>
>>>>> <body>
>>>>>
>>>>> <hr>
>>>>>
>>>>> Hello bob <br/>
>>>>> Hello bobie
>>>>> <hr/>
>>>>>
>>>>>
>>>>>
>>>>> Title: Loud People Out Front
>>>>>
>>>>>
>>>>> <hr>
>>>>> </body>
>>>>> </html>
>>>>>
>>>>> ---------------------------------------------------------
>>>>>
>>>>> Size Range Tree :
>>>>> |[0x28165000,0x2a165000)| = 33554432 id=12 pid=712816190
>>>>>
>>>>> Allocated Range Tree :
>>>>> According to lo-mem we have 1 nodes allocated
>>>>> === END output ===
>>>>>
>>>>> On Tuesday 15 April 2003 05:21 pm, David Hakim wrote:
>>>>>> Interesting ... I'm really surprised that this failure is
>>>>>> occurring
>>>>>> with the compiled module.
>>>>>>
>>>>>> Anyhow ... its pretty clear then that there is a memory leak at
>>>>>> least
>>>>>> somewhere. Since your test page isn't really using codex.http,
>>>>>> could
>>>>>> you run
>>>>>>
>>>>>> moto -d <test file>
>>>>>>
>>>>>> from the command line and verify that there are no leaks there ?
>>>>>>
>>>>>> Second, could you add the following directives to your httpd.conf:
>>>>>>
>>>>>> MotoOption Session.Disable TRUE
>>>>>>
>>>>>> changing the above to
>>>>>>
>>>>>> <CompiledModule>Option Session.Disable TRUE
>>>>>>
>>>>>> for your compiled module and re-run the ab tests ?
>>>>>>
>>>>>> -Dave
>>>>>>
>>>>>> On Tuesday, April 15, 2003, at 09:06 AM, Scott Rogers wrote:
>>>>>>> David,
>>>>>>>
>>>>>>> I experienced the same behavior on the FreeBSD box in interperted
>>>>>>> and
>>>>>>> compiled
>>>>>>> mode. I did not try interperted on the Linux box.
>>>>>>>
>>>>>>> I did nothing to turn off sessions. So, if sessions are on 'out
>>>>>>> of
>>>>>>> the box'
>>>>>>> then they were on for these tests.
>>>>>>>
>>>>>>> Scott Rogers
>>>>>>>
>>>>>>> On Tuesday 15 April 2003 03:50 pm, David Hakim wrote:
>>>>>>>> Is this under the interpreter or with a module compiled with
>>>>>>>> mmc ?
>>>>>>>> Also, are sessions turned off for this application ?
>>>>>>>>
>>>>>>>> -Dave
>>>>>>>>
>>>>>>>> On Tuesday, April 15, 2003, at 07:03 AM, Scott Rogers wrote:
>>>>>>>>> Hey All,
>>>>>>>>>
>>>>>>>>> I recently did some performance testing of Moto on both FreeBSD
>>>>>>>>> 4.6.2
>>>>>>>>> and
>>>>>>>>> Redhat Linux 7.2.
>>>>>>>>>
>>>>>>>>> FreeBSD
>>>>>>>>> Apache: 1.3.26
>>>>>>>>> Moto: 0.19.0
>>>>>>>>> 256mg ram
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> RedHat
>>>>>>>>> Apache 1.3.27
>>>>>>>>> Moto: 0.19.0
>>>>>>>>> 1gig ram
>>>>>>>>>
>>>>>>>>> Using ApacheBench (ab with source distributions) in the
>>>>>>>>> following
>>>>>>>>> manner:
>>>>>>>>> ab -t 0 http://localhost/tmoto1/hello.moto
>>>>>>>>>
>>>>>>>>> This sends 50,000 connections to the server, one at a time.
>>>>>>>>> Somewhere
>>>>>>>>> between
>>>>>>>>> 5000 and 10,000 connections, Moto starts sending "###
>>>>>>>>> Allocation
>>>>>>>>> failure"
>>>>>>>>> messages to the Apache error log file.
>>>>>>>>>
>>>>>>>>> I tracked this down to line 568 of src/codex/sharedmem.c. It
>>>>>>>>> seems
>>>>>>>>> there is
>>>>>>>>> not enough contiguous memory, according to function
>>>>>>>>> "locateLPFNode"
>>>>>>>>> at
>>>>>>>>> line
>>>>>>>>> #516.
>>>>>>>>>
>>>>>>>>> It seems when Moto is unable to return a large enough block of
>>>>>>>>> memory
>>>>>>>>> from
>>>>>>>>> it's internal stack, it issues the above warning/error message
>>>>>>>>> to
>>>>>>>>> stderr.
>>>>>>>>>
>>>>>>>>> The problem is, it logs the message so many times it fills up
>>>>>>>>> my
>>>>>>>>> /var
>>>>>>>>> file
>>>>>>>>> system in a matter of seconds!
>>>>>>>>>
>>>>>>>>> Also, it is important to note, that on the FreeBSD system Moto
>>>>>>>>> consumes all
>>>>>>>>> available RAM, but not on the Linux box with 1gig of Ram.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Here is the code excerpt:
>>>>>>>>>
>>>>>>>>> === BEGIN hello.moto ===
>>>>>>>>> $use("codex.http")
>>>>>>>>> #include("my_classes.moto")
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <html>
>>>>>>>>> <head>
>>>>>>>>> <style type="text/css">
>>>>>>>>> </style>
>>>>>>>>> </head>
>>>>>>>>> <body>
>>>>>>>>>
>>>>>>>>> <hr>
>>>>>>>>> ${
>>>>>>>>> HelloWorld hw = new HelloWorld();
>>>>>>>>> String message = hw.sayHelloTo("bob");
>>>>>>>>> }$
>>>>>>>>> $(message + "<br/>")
>>>>>>>>> $(hw.sayHelloTo("bobie"))
>>>>>>>>> <hr/>
>>>>>>>>>
>>>>>>>>> ${
>>>>>>>>> Box box = new Box("Loud People Out Front");
>>>>>>>>> }$
>>>>>>>>>
>>>>>>>>> $(box.renderStandard())
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> <hr>
>>>>>>>>> </body>
>>>>>>>>> </html>
>>>>>>>>> === END hello.moto ===
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> === BEGIN my_classes.moto ===
>>>>>>>>> ${
>>>>>>>>> class HelloWorld {
>>>>>>>>> String sayHelloTo(String name){
>>>>>>>>> return "Hello " + name + " ";
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> class Box {
>>>>>>>>> boolean _initalized = false;
>>>>>>>>> String _title = "";
>>>>>>>>>
>>>>>>>>> Box(String title) {
>>>>>>>>> this._title = title;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> String renderStandard() {
>>>>>>>>> return "Title: " + _title;
>>>>>>>>> }
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> }$
>>>>>>>>> === END my_classes.moto ===
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> Scott Rogers
>
>