Re: Lies, damned lies, and statistics or thoughts on report-time-results

Good point.  I remember I looked at Perl's Benchmark module to write it,
but I'm not quite sure why I used cputime only.  I'll add options.



On Thu, Nov 10, 2022 at 2:01 AM Jens Thiele <[email protected]> wrote:

> Hi,
>
> I really like time-these/report and report-time-results, but for the
> problem at hand it does report misleading results because it reports the
> rate per cputime and not per realtime:
> "
> (define (cputime result)  (+ (usertime result) (systime result)))
> (define (rate result)     (/. (~ result'count) (cputime result)))
> "
>
> Benchmark: ran suisse, germany, south-america, each for at least 3.0 cpu
> seconds.
>          suisse: 4.121 real, 3.120 cpu (2.690 user + 0.430 sys)@ 801.28/s
> n=2500
>         germany: 6.590 real, 3.200 cpu (1.890 user + 1.310 sys)@2131.25/s
> n=6820
>   south-america: 3.055 real, 3.020 cpu (2.700 user + 0.320 sys)@ 774.17/s
> n=2338
>
>                   Rate suisse germany south-america
>          suisse  801/s     --   0.376         1.035
>         germany 2131/s  2.660      --         2.753
>   south-america  774/s  0.966   0.363            --
>
> What's happening?
>
> For the problem at hand quite some time is spent on disk i/o especially
> in the germany case (it could also be a network socket or some sleep
> ...)
>
> Maybe report-time-results optionally could use the realtime for rate
> calculation?
>
> greetings,
> jens
>
>
> _______________________________________________
> Gauche-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/gauche-devel
>

_______________________________________________
Gauche-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/gauche-devel