Re: Latest Stack Performance

I have now been able to reproduce the performance numbers 
that have been reported.  As a matter of fact, I have been 
able to run the Shootme UAS application at a sustained 
call rate of over 90 CAPS on my 2 GHz MacBook Pro!

My original performance test is still running half as 
fast, at 45 CAPS, but I believe that I understand why. 
 The original Shootme UAS application is designed to send 
a BYE REQUEST to the UAC as soon as the call is completed. 
 My version of the Shootme UAS was modified to not 
initiate the BYE REQUEST, but instead rely on the UAC call 
generator to initiate the BYE.  From studying the SIP 
traces, it is apparent that it takes as much time to 
process the INVITE REQUEST as it does to process the BYE 
REQUEST.  On the other hand, generating a BYE is 
relatively insignificant.  This then accounts for the 2:1 
performance difference.

 From a black box analysis perspective, I would be 
inclined to believe that request handling is a performance 
hot-spot in the current stack.

-Mardy

On Mon, 17 Jul 2006 10:32:30 -0400
  Marden P. Marshall <[email protected]> wrote:
> 
>
>From: "Marden P. Marshall" <[email protected]>
> Date: Thu, 13 Jul 2006 16:11:58 -0400
> To: <[email protected]>
> Subject: Re: [nist-sip] Latest Stack Performance
> 
> I performed my benchmarks using the version of the stack
> that was checked into dev.java.net over the July 4th
> weekend.  Are you suggesting that I try an earlier 
>version
> from the NIST repository?
> 
> -Mardy
> 
> On Thu, 13 Jul 2006 14:01:57 -0400
>  "M. Rangnathan" <[email protected]> wrote:
>> Hmmm.. Something is amiss indeed. Looks like something
>>bad happened over the last week or two. Check out the
>>version from about a week ago and you should see the
>>reported numbers. I'll take a look and see what the mater
>>could be.
>> 
>> Ranga
>> 
>> Marden P. Marshall wrote:
>> 
>>> First, great job on the stack!
>>>
>>> I've been doing some basic performance benchmarking of
>>>the latest 
>>> stack code.  Before I talk about the actual performance
>>>numbers, let 
>>> me first describe the test setup. I used a variation of
>>>the UAS code: 
>>> "test/load/concurrency/Shootme.java".  The only
>>>functional changes
>>> that I made was to remove the delay in responding to the
>>>INVITE and to 
>>> disable the generation of the BYE. The UAC is
>>>responsible for 
>>> generating both  the INVITE and the BYE.  I then use a
>>>SIP load 
>>> generator application called sipp
>>>(http://sipp.sourceforge.net) to
>>> generate a load directed to the UAS.  The call scenario
>>>that I used is 
>>> as follows:
>>>
>>>       INVITE ---------->
>>>          100 <----------
>>>          180 <----------
>>>          200 <----------
>>>          ACK ---------->
>>>          BYE ---------->
>>>          200 <----------
>>>
>>> I then ran the UAS code on a Pentium-M PC running at
>>>2.5GHz using 
>>> version 1.5.0_06 JVM.  I ran the UAC traffic generator
>>>on another 
>>> machine so as not to interfere with the UAS performance.
>>> So now for 
>>> the results.
>>>
>>> First, I am seeing about a 25% performance improvement
>>>between the old 
>>> stack and the current version.  But I am not seeing the
>>>performance 
>>> numbers that others have reported.  With the new code
>>>and this 
>>> hardware configuration, I am only able to get about 24
>>>CPS.  I believe 
>>> that someone reported getting about 70 CAPS on a similar
>>>hardware 
>>> platform.  I am not sure if the call scenarios are the
>>>same, as that 
>>> can make a huge difference.
>>>
>>> I would be curious to know what other peoples experience
>>>has been in 
>>> benchmarking the code and wether or not my results are
>>>in-line.
>>>
>>> -Mardy
>>> _______________________________________________
>>> nist-sip mailing list
>>> [email protected]
>>> http://www-x.antd.nist.gov/mailman/listinfo/nist-sip
>>>
>> 
>> 
>> -- 
>> M. Ranganathan 
>> Advanced Networking Technologies Division,
>> National Institute of Standards and Technology (NIST),
>> 100 Bureau Drive, Stop 8920, Gaithersburg, MD 20899.
>>tel:301 975 3664 , fax:301 590 0932
>>http://w3.antd.nist.gov/
>> Advanced Networking Technologies For the People!
>> 
> 
> _______________________________________________
> nist-sip mailing list
> [email protected]
> http://www-x.antd.nist.gov/mailman/listinfo/nist-sip