[ grinder-Bugs-3046909 ] Very high test time standard deviation

Bugs item #3046909, was opened at 2010-08-17 02:59
Message generated for change (Settings changed) made by philipa
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=118598&aid=3046909&group_id=18598

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Console
Group: None
>Status: Closed
Resolution: None
Priority: 5
Private: No
Submitted By:  Paul W (phawieze)
Assigned to: Philip Aston (philipa)
Summary: Very high test time standard deviation

Initial Comment:
Hi, 

I've ran a succesfull test with the grinder and it really helped to measure the performance of my system. 

However there is still one issue with the results that i can't explain. The mean test times seem to be calculated correctly. However the standard deviation is in some places ten times as high, which has to mean that grinder recorded very high negative and positive results. Is this in some way related to the faq entry 1.5.1 (grinder reports negative test times)? The difference is that i'm running the agents and console on Windows 7. Or is it related to 1.5.6 (limits to accurate timings). The difference with this issue is that my mean times seem to be correct and that just the standard deviation ain't correct. 

Test	Description		Succesfull 	                     Mean Test	                     Test time standard deviaton
			tests	Errors	time 	
Test 100	Page 1	 	500	0	451		844
Test 101	GET livelink.exe	500	0	118		569
Test 102	GET livelink.exe	500	0	88,7		191
Test 103	GET livelink.css	500	0	15,6		126
Test 104	GET global.js		500	0	11,8		135
Test 105	GET globalmenus.js	500	0	3,12		0,832
Test 106	GET globalmenu.js	500	0	3,05		1,17
Test 107	GET menu.js		500	0	11,1		167
Test 108	GET functionmenu.js	500	0	3,34		1,6
Test 109	GET newsticker.js	500	0	3,2		2,9

Grinder version 3.4
Java version 1.6.0_11-b03  (Version 6 build 11)

----------------------------------------------------------------------

Comment By: Philip Aston (philipa)
Date: 2011-08-01 09:42

Message:
I think its far more likely that the large standard deviations are actually
correct. You can verify this by examining the data file, and looking at the
timings for individual requests. Bear in mind that requests that set up an
HTTP connection (for example) might take a lot longer than subsequent
requests that re-use the connection. Also, server side performance issues
(e.g. garbage collection), can cause requests to take much longer than
normal. 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=118598&aid=3046909&group_id=18598

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and 
threat landscape has changed and how IT managers can respond. Discussions 
will include endpoint security, mobile security and the latest in malware 
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/