Re: Soak tests
"Dr Steven A. Wright" <[email protected]>
| Newsgroups | gmane.ietf.bmwg |
|---|---|
| Message-ID | <[email protected]> |
I agree with your comment, but was also trying to be sensitive to Pierre's concern as to where to draw the line between a performance benchmarking metric and a diagnostic procedure. My intuition was too recast the soak test around a performance benchmark for the duration of "stable operation under load". Duration is a nice linear time scale for a performance metric. At some point the user has to decide how long of a duration is satisfactory for their application, but that is a separate argument that _may_ be sensitive to the application. the definition of "stable operation under load" seems quite likely to be application specific.... > On Jul 20, 2016, at 8:01 PM, MORTON, ALFRED C (AL) <[email protected]> wrote: > > Thanks for your comment, Steve. > > There is some value in continuing to test after an error, > to see if the system recovers to the pre-error throughput > for example, but I agree that there are likely to be events > after which it would make sense to observe recovery if it happens > and stop. > > Al > >> -----Original Message----- >> From: bmwg [mailto:[email protected]] On Behalf Of Dr Steven A. >> Wright >> Sent: Wednesday, July 20, 2016 5:56 AM >> To: [email protected] >> Subject: [bmwg] Soak tests >> >> >> We had some discussion around soak tests, as to what the appropriate >> duration should be and whether ( or what) the performance metric was >> under measurement. >> >> I would suggest that these tests should be measured as the duration of >> stable operation under load. The measurement period starts when full >> load is achieved on the system under test and terminated when (1) >> received load (throughput) variation exceeds some threshold (5%?) or (2) >> some system exception/ failure occurs. The test may be terminated after >> some user defined period. The user defined period for the test is >> usually greater than 24 hrs to catch intermittent/ probabilistic system >> scheduling conflicts that interfere with stable operation. >> >> Regards >> Steven Wright >> >> _______________________________________________ >> bmwg mailing list >> [email protected] >> https://www.ietf.org/mailman/listinfo/bmwg