Lies, damned lies, and statistics or thoughts on report-time-results
Jens Thiele <[email protected]> Thu, 10 Nov 2022 12:44:04 +0100
| Newsgroups | gmane.lisp.scheme.gauche |
|---|---|
| Message-ID | <[email protected]> |
Hi,
I really like time-these/report and report-time-results, but for the
problem at hand it does report misleading results because it reports the
rate per cputime and not per realtime:
"
(define (cputime result) (+ (usertime result) (systime result)))
(define (rate result) (/. (~ result'count) (cputime result)))
"
Benchmark: ran suisse, germany, south-america, each for at least 3.0 cpu seconds.
suisse: 4.121 real, 3.120 cpu (2.690 user + 0.430 sys)@ 801.28/s n=2500
germany: 6.590 real, 3.200 cpu (1.890 user + 1.310 sys)@2131.25/s n=6820
south-america: 3.055 real, 3.020 cpu (2.700 user + 0.320 sys)@ 774.17/s n=2338
Rate suisse germany south-america
suisse 801/s -- 0.376 1.035
germany 2131/s 2.660 -- 2.753
south-america 774/s 0.966 0.363 --
What's happening?
For the problem at hand quite some time is spent on disk i/o especially
in the germany case (it could also be a network socket or some sleep
...)
Maybe report-time-results optionally could use the realtime for rate
calculation?
greetings,
jens