Re: improving the benchmarking API
fady esam via ublas <[email protected]> Wed, 12 Sep 2018 07:06:21 +0200
| Newsgroups | gmane.comp.lib.boost.ublas |
|---|---|
| Message-ID | <CALnUhKP6-K3K=TO7akcGFicjRguDv6yu6eRWBpXJMitEit_kXw@mail.gmail.com> |
--===============2870727172179710142== Content-Type: multipart/related; boundary="000000000000e963cd0575a58d64" --000000000000e963cd0575a58d64 Content-Type: multipart/alternative; boundary="000000000000e963cb0575a58d63" --000000000000e963cb0575a58d63 Content-Type: text/plain; charset="UTF-8" Hi, I did a similar implementation in my gsoc project covering most of the operations.. But a small diference in my implementation that running the executable of each operation produces the file containing the benchmarking data directly Thanks, Fady Essam On Wed, Sep 12, 2018, 4:08 AM Stefan Seefeld via ublas < [email protected]> wrote: > Hi there, > > I have been looking at the existing benchmarks, to see how to extend them > to cover more functions as well as alternative implementations. The > existing benchmarks have a few shortcomings that I would like to address: > > * a single benchmark executable will measure a range of operations, and > write output to stdout. It's impossible to benchmark individual operations > > * operations are measured with a single set of inputs. It would be very > helpful to be able to run operations on a range of inputs, to see how they > perform over a variety of problem sizes. > > * the generated output should be easily machine-readable, so it can be > post-processed into benchmark reports (including performance charts). > > > (The above will be particularly useful as we are preparing PRs to include > support for OpenCL backends (work that has been done by Fady Essam as a > GSoC project). > > > I have attempted to prototype a few new benchmarks (matrix-matrix > products, as well as matrix-vector products, for a variety of value-types), > together with a simple script to produce graphs. For example, the attached > plot was produced running: > > ``` > > .../mm_prod -t float > mm_prod_float.txt > > .../mm_prod -t double > mm_prod_double.txt > > .../mm_prod -t fcomplex > mm_prod_fcomplex.txt > > .../mm_prod -t dcomplex > mm_prod_dcomplex.txt > > plot.py mm_prod_*.txt > > ``` > > I'd appreciate any feedback, both on the general concepts, as well as the > code, which is here: https://github.com/boostorg/ublas/pull/57 > > Thanks, > > [image: Stefan] > > -- > > ...ich hab' noch einen Koffer in Berlin... > > > _______________________________________________ > ublas mailing list > [email protected] > https://lists.boost.org/mailman/listinfo.cgi/ublas > Sent to: [email protected] > --000000000000e963cb0575a58d63 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">Hi,=C2=A0<div dir=3D"auto"><br></div><div dir=3D"auto">I = did a similar implementation in my gsoc project covering most of the operat= ions..</div><div dir=3D"auto"><br></div><div dir=3D"auto">But a small difer= ence in my implementation that running the executable of each operation pro= duces the file containing the benchmarking data directly</div><div dir=3D"a= uto"><br></div><div dir=3D"auto">Thanks,</div><div dir=3D"auto">Fady Essam<= /div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr">On Wed, Sep 12, = 2018, 4:08 AM Stefan Seefeld via ublas <<a href=3D"mailto:[email protected]= ost.org">[email protected]</a>> wrote:<br></div><blockquote class=3D= "gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding= -left:1ex"> =20 =20 =20 <div text=3D"#000000" bgcolor=3D"#FFFFFF"> <p>Hi there,</p> <p>I have been looking at the existing benchmarks, to see how to extend them to cover more functions as well as alternative implementations. The existing benchmarks have a few shortcomings that I would like to address:</p> <p>* a single benchmark executable will measure a range of operations, and write output to stdout. It's impossible to benchmark individual operations</p> <p>* operations are measured with a single set of inputs. It would be very helpful to be able to run operations on a range of inputs, to see how they perform over a variety of problem sizes.</p> <p>* the generated output should be easily machine-readable, so it can be post-processed into benchmark reports (including performance charts).</p> <p><br> </p> <p>(The above will be particularly useful as we are preparing PRs to include support for OpenCL backends (work that has been done by Fady Essam as a GSoC project).<br> </p> <p><br> </p> <p>I have attempted to prototype a few new benchmarks (matrix-matrix products, as well as matrix-vector products, for a variety of value-types), together with a simple script to produce graphs. For example, the attached plot was produced running:</p> <p>```<br> </p> <p>=C2=A0.../mm_prod -t float > mm_prod_float.txt</p> <p>.../mm_prod -t double > mm_prod_double.txt</p> <p>.../mm_prod -t fcomplex > mm_prod_fcomplex.txt</p> <p>.../mm_prod -t dcomplex > mm_prod_dcomplex.txt</p> <p>plot.py mm_prod_*.txt</p> <p>```<br> </p> <p>I'd appreciate any feedback, both on the general concepts, as well as the code, which is here: <a class=3D"m_-7577290477398864347mo= z-txt-link-freetext" href=3D"https://github.com/boostorg/ublas/pull/57" tar= get=3D"_blank" rel=3D"noreferrer">https://github.com/boostorg/ublas/pull/57= </a><br> </p> <p>Thanks,<br> </p> <div class=3D"m_-7577290477398864347moz-signature"><br> <div class=3D"m_-7577290477398864347moz-signature"><img src=3D"cid:pa= [email protected]" alt=3D"Stefan" height=3D"45" width=3D"7= 3"><br> <pre>--=20 ...ich hab' noch einen Koffer in Berlin... </pre> </div> </div> </div> _______________________________________________<br> ublas mailing list<br> <a href=3D"mailto:[email protected]" target=3D"_blank" rel=3D"noreferre= r">[email protected]</a><br> <a href=3D"https://lists.boost.org/mailman/listinfo.cgi/ublas" rel=3D"noref= errer noreferrer" target=3D"_blank">https://lists.boost.org/mailman/listinf= o.cgi/ublas</a><br> Sent to: <a href=3D"mailto:[email protected]" target=3D"_blank" rel=3D= "noreferrer">[email protected]</a><br> </blockquote></div> --000000000000e963cb0575a58d63-- --000000000000e963cd0575a58d64 Content-Type: image/png; name="signature.png" Content-Disposition: inline; filename="signature.png" Content-Transfer-Encoding: base64 Content-ID: <> X-Attachment-Id: iVBORw0KGgoAAAANSUhEUgAAAEkAAAAtCAYAAADiFkaQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz AAAVhwAAFYcBshnuugAAABl0RVh0U29mdHdhcmUAd3d3Lmlua3NjYXBlLm9yZ5vuPBoAAAVDSURB VGiB5dpprF1TFMDx33u0iqpWW6LmqWpIVUkoNVbM81ShiJifIakgxAc1VEzxwRiC8EHwgaAilBhC g6BClJIYaqohSM1t6fVh7afXfffcd+55d8I/uXnvnL3PWuvus/daa699+e8xCFfhlnYb0qkMw40o 4dA229KRDMLL+AnvYYX2mtOZXIj5+B4HttmWjmQkfsCreLDNtnQsV+ErfIsxbbalI1kDi/AnTm6z LR3LpSKazUJXm23JRauNHIyF+ByjWqy7boagB59gGR4QIbnZHCeW2Z4t0DUgNsXh4k2Owxti+l/e At0vt0jPgNgUn2FC2b2NsFjkK81M6E4UDntYE3UMmC6Rl9xUpW2WmE2bNUn3OPzsX7A/OxC/q+4w rxGDtE3GszvgmIJ6h+CtJH/7gjJaxiO4P6PtSvEltqjStifmYXhBvXeJAPF2wedbxmgswd4Z7bfg R30j3Dr4FFsV1HsmZmMOpheU0TJ6hE9YKaN9Hm6vuNeF53F+QZ2T8IFYYouxZkE5LeNhPJHRNgXf iVlTzikiPeguoG8UFmBXkWE/WkBGS+kWg1Btuo/AXOxRcX8kvhFfsl668BhmpusPMK2AnJaytXDK k7EyrhNlihfFbnztKs/chocK6jsbLwn/Nl5E1NULymoZ08RWYDUMxUE4UiSR1dhQVAuL5EwrieDQ u0QvEzlYx3M93q+j/724oUG638UJDZLVVJ4SPiIPY8XWYa0G6N1cpB0jGiCrLsojTVY4r2QTEWny MBN34Ot6jKqgt/xyFF4Q/i8vQ3EA9jHAMs5kkaCtmqNvt8hRLsjRdwJ+U92R52VHvCPqRq/In0Cu hRl4Uszk0gDt8Jr8kWdMUnh0P/0GiS3LPem6W/6Z2st2+CPpW08Ei81z6D1HnLn16pulAc5+cR1C Jgmjd8honyAG/QvhP8aLrcSjWLFOuw5PuhYLZ/1hP/3XFfWliWX3dhKuoTKxrYsVxQnDdmLNlvrp v376+1VG+0TLd+ZLcRZOFVP/jzpt+65MzvH+meEPFynHJtg2fcaLlzM39dkYV4sN9Rd16u7DfWJw ds/R99zUd5WM9mmpvYTTsH/6v6eAXd2Wl0RK4mXOw0d4M9k9XfjU4cKhl8SW6XSRgBbdSPdhryR8 do6+M/FLjfapSdYPIhs/IF3fnUN2t76llddFhr1ELPWRNZ7fUpRRFogl2ju4S8TJ7tQcNtTkpSRw t3763SlqOVkcmeTcmq5HikH9DTvXeG6UOL+vjJofJ3lz+rGrl/KUZpQo+l2UZCxUvIYFdhGnHO+J yl8Wj4m3m8WpyaD9yu71pHuLxCBUJoMjhO86Xd/606/p2Stqm98vTyc5P+FxcXjQgyPEpnuc2t/7 b25Pgq6t0efVpCSLGUnG1hX3DxP5Tkk48Hl4VuQ+9wunW8kIy5fMQI+MBos95hUiLXlCROEFYpaX RIrxsUiHTpZxjjcsdVomsttqfCky6CzmiEHIeisbill2lCin1DpQ3CoZv1S+RLcoXdgA+4qKxvyk 92dR1egTpCakxl+EoyxniBjt8zKUjU7t8xtgOLGVKKm9vJtBFw62fLAqK6yIJG6pWL97ld3fMT00 JUP4viJMZ7XXy0lJX7Ujq1YwRHYNH+HQlojwe7FwqOeJyDY445nR4hcejeISMUjHNlBmw9ldnM6W ROn1OdUdbLO4WXMPOBvGajgDh2j9L0YeFr6xyMHB/4bX0qftdPJb2kjkU22nUwdpmNjS5K2ANpVO HaSx6e/CtlqR6NRB6q0GfN5WKxKdOkhjRF72TLsNgb8Aty4jMPtjABcAAAAASUVORK5CYII= --000000000000e963cd0575a58d64-- --===============2870727172179710142== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline