Re: best effort vs. given req rate - strange response times

On Mon, 2005/02/14 (MST), <[email protected]> wrote:

> I am currently implementing a secure HTTP proxy for an information
> security laboratory (see http://www.infsec.ethz.ch/lab/appliedlab) and
> use web polygraph for performance testing. I want to test the proxy's
> performance a) with a realistic low request rate of .6 req/sec and b) in
> best effort mode.
> ...
> the actual test consist of a single phase during 20 minutes. I ran the
> test with different configurations of the proxy (filtering modules
> switched on or off) but always encountered a very strange behaviour:
> mean response times are _lower_ in the best effort mode than with the .6
> req/sec request rate...
>
> example from one test:
>
> best effort: 180 xAct/sec = 13.65 Mbits/sec, mean response time 5msec
> .6 req/sec: 0.60 xAct/sec = 0.05 Mbits/sec, mean response time 9 msec

There are several differences between the two tests that may explain lower  
best effort response time. For example, with a single best effort robot,  
the proxy handles one request at a time. There are no concurrent requests  
(except for requests for embedded objects, if they exist in your  
environment). With a constant mean request rate robot, you may get bursts  
of concurrent requests. The proxy might slow down responses during a  
burst. To [indirectly] verify this theory, study the "Concurrency Levels"  
graphs produced by the "reporter" tool and increasing the number of  
best-effort robots to match the concurrency level of the other test.

Another reason could be the mean rate of transactions itself. It is  
possible that something in the proxy slows down isolated transactions  
compared to transactions that occur without a delay. For example, the  
proxy may have some kind of select(2) or idle loop optimizations that lead  
to a few milliseconds of extra sleep time when a new transaction arrives  
after a pause. Polygraph might have a similar problem as it is rarely used  
for very low request rate testing. To [indirectly] verify this theory, you  
can increase the number of rated robots or increase the single robot rate  
to ensure that there is usually a few pending requests at the proxy.

Note that the difference within a few milliseconds is usually within  
Polygraph test precision. A consistent difference most likely indicates  
that there is something interesting going on, but any two comparable  
results are still a statistical tie.

It would be great if you inform the list of your progress and, hopefully,  
the actual reason behind response time difference.

Thank you,

Alex.
P.S. You can further simplify the test by removing embedded objects, but  
that would be a less realistic, micro-level test, of course.