Re: Latest Stack Performance

Mardy,

That is very cool. Thanks for reporting it. However, the problem may 
have been a stupid bug of mine. It turns out that

(int) Math.random() * 10000 == 0


Which means, the branch ids that I was generating were not random at all 
and moreover would clash if two were generated in the period of a 
milisecond. Ahem... well OK this problem is fixed and hence the improved 
results.

Ranga.

Marden P. Marshall wrote:

> I have now been able to reproduce the performance numbers that have 
> been reported.  As a matter of fact, I have been able to run the 
> Shootme UAS application at a sustained call rate of over 90 CAPS on my 
> 2 GHz MacBook Pro!
>
> My original performance test is still running half as fast, at 45 
> CAPS, but I believe that I understand why. The original Shootme UAS 
> application is designed to send a BYE REQUEST to the UAC as soon as 
> the call is completed. My version of the Shootme UAS was modified to 
> not initiate the BYE REQUEST, but instead rely on the UAC call 
> generator to initiate the BYE.  From studying the SIP traces, it is 
> apparent that it takes as much time to process the INVITE REQUEST as 
> it does to process the BYE REQUEST.  On the other hand, generating a 
> BYE is relatively insignificant.  This then accounts for the 2:1 
> performance difference.
>
> From a black box analysis perspective, I would be inclined to believe 
> that request handling is a performance hot-spot in the current stack.
>
> -Mardy
>
> On Mon, 17 Jul 2006 10:32:30 -0400
>  Marden P. Marshall <[email protected]> wrote:
>
>>
>>
>> From: "Marden P. Marshall" <[email protected]>
>> Date: Thu, 13 Jul 2006 16:11:58 -0400
>> To: <[email protected]>
>> Subject: Re: [nist-sip] Latest Stack Performance
>>
>> I performed my benchmarks using the version of the stack
>> that was checked into dev.java.net over the July 4th
>> weekend.  Are you suggesting that I try an earlier version
>> from the NIST repository?
>>
>> -Mardy
>>
>> On Thu, 13 Jul 2006 14:01:57 -0400
>>  "M. Rangnathan" <[email protected]> wrote:
>>
>>> Hmmm.. Something is amiss indeed. Looks like something
>>> bad happened over the last week or two. Check out the
>>> version from about a week ago and you should see the
>>> reported numbers. I'll take a look and see what the mater
>>> could be.
>>>
>>> Ranga
>>>
>>> Marden P. Marshall wrote:
>>>
>>>> First, great job on the stack!
>>>>
>>>> I've been doing some basic performance benchmarking of
>>>> the latest stack code.  Before I talk about the actual performance
>>>> numbers, let me first describe the test setup. I used a variation of
>>>> the UAS code: "test/load/concurrency/Shootme.java".  The only
>>>> functional changes
>>>> that I made was to remove the delay in responding to the
>>>> INVITE and to disable the generation of the BYE. The UAC is
>>>> responsible for generating both  the INVITE and the BYE.  I then use a
>>>> SIP load generator application called sipp
>>>> (http://sipp.sourceforge.net) to
>>>> generate a load directed to the UAS.  The call scenario
>>>> that I used is as follows:
>>>>
>>>>       INVITE ---------->
>>>>          100 <----------
>>>>          180 <----------
>>>>          200 <----------
>>>>          ACK ---------->
>>>>          BYE ---------->
>>>>          200 <----------
>>>>
>>>> I then ran the UAS code on a Pentium-M PC running at
>>>> 2.5GHz using version 1.5.0_06 JVM.  I ran the UAC traffic generator
>>>> on another machine so as not to interfere with the UAS performance.
>>>> So now for the results.
>>>>
>>>> First, I am seeing about a 25% performance improvement
>>>> between the old stack and the current version.  But I am not seeing 
>>>> the
>>>> performance numbers that others have reported.  With the new code
>>>> and this hardware configuration, I am only able to get about 24
>>>> CPS.  I believe that someone reported getting about 70 CAPS on a 
>>>> similar
>>>> hardware platform.  I am not sure if the call scenarios are the
>>>> same, as that can make a huge difference.
>>>>
>>>> I would be curious to know what other peoples experience
>>>> has been in benchmarking the code and wether or not my results are
>>>> in-line.
>>>>
>>>> -Mardy
>>>> _______________________________________________
>>>> nist-sip mailing list
>>>> [email protected]
>>>> http://www-x.antd.nist.gov/mailman/listinfo/nist-sip
>>>>
>>>
>>>
>>> -- 
>>> M. Ranganathan Advanced Networking Technologies Division,
>>> National Institute of Standards and Technology (NIST),
>>> 100 Bureau Drive, Stop 8920, Gaithersburg, MD 20899.
>>> tel:301 975 3664 , fax:301 590 0932
>>> http://w3.antd.nist.gov/
>>> Advanced Networking Technologies For the People!
>>>
>>
>> _______________________________________________
>> nist-sip mailing list
>> [email protected]
>> http://www-x.antd.nist.gov/mailman/listinfo/nist-sip
>
> _______________________________________________
> nist-sip mailing list
> [email protected]
> http://www-x.antd.nist.gov/mailman/listinfo/nist-sip
>


-- 
M. Ranganathan 

Advanced Networking Technologies Division,
National Institute of Standards and Technology (NIST),
100 Bureau Drive, Stop 8920, Gaithersburg, MD 20899. 
tel:301 975 3664 , fax:301 590 0932 http://w3.antd.nist.gov/
Advanced Networking Technologies For the People!