Re: Lies, damned lies, and statistics or thoughts on report-time-results

Now you can say (time-these/report config samples :metric 'real)  to use
real (wallclock) time to compute rates.  Metrix can take either real, cpu
(default), user, or sys.

On Thu, Nov 10, 2022 at 12:16 PM Shiro Kawai <[email protected]> wrote:

> Good point.  I remember I looked at Perl's Benchmark module to write it,
> but I'm not quite sure why I used cputime only.  I'll add options.
>
>
>
> On Thu, Nov 10, 2022 at 2:01 AM Jens Thiele <[email protected]> wrote:
>
>> Hi,
>>
>> I really like time-these/report and report-time-results, but for the
>> problem at hand it does report misleading results because it reports the
>> rate per cputime and not per realtime:
>> "
>> (define (cputime result)  (+ (usertime result) (systime result)))
>> (define (rate result)     (/. (~ result'count) (cputime result)))
>> "
>>
>> Benchmark: ran suisse, germany, south-america, each for at least 3.0 cpu
>> seconds.
>>          suisse: 4.121 real, 3.120 cpu (2.690 user + 0.430 sys)@ 801.28/s
>> n=2500
>>         germany: 6.590 real, 3.200 cpu (1.890 user + 1.310 sys)@2131.25/s
>> n=6820
>>   south-america: 3.055 real, 3.020 cpu (2.700 user + 0.320 sys)@ 774.17/s
>> n=2338
>>
>>                   Rate suisse germany south-america
>>          suisse  801/s     --   0.376         1.035
>>         germany 2131/s  2.660      --         2.753
>>   south-america  774/s  0.966   0.363            --
>>
>> What's happening?
>>
>> For the problem at hand quite some time is spent on disk i/o especially
>> in the germany case (it could also be a network socket or some sleep
>> ...)
>>
>> Maybe report-time-results optionally could use the realtime for rate
>> calculation?
>>
>> greetings,
>> jens
>>
>>
>> _______________________________________________
>> Gauche-devel mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/gauche-devel
>>
>

_______________________________________________
Gauche-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/gauche-devel