-warmup
Ahmet Göksu <[email protected]>
| Newsgroups | gmane.comp.fonts.freetype.devel |
|---|---|
| Message-ID | <c65df83b-83ae-4d5a-a168-94bba8abeba7@Spark> |
Hi, I have updated the bench code. It is warming up as the given number of seconds with -w flag before every benchmark test. There are still differences like 100%.. Also, 1 sec warmup means (test count)*(font count) 70 secs for the results. I am thinking of what else can be done and waiting for your test. Best, Goksu goksu.in On 3 Aug 2023 05:45 +0300, Werner LEMBERG <[email protected]>, wrote: > > > I have done the changes you want. > > Thanks! > > > > 36.5% run difference is baaaad. AFAICS, you haven't yet worked on > > > omitting 'warmup' iterations, right? > > > > I am planning to increase the iteration count by 10% and ignore the > > results for them. > > You mean you are going to ignore the first 10% of the iterations? > This might be a good default. However, I still think that a > `--warmup` command-line option makes sense to control this. > > > Trying to figure out the benchmarking program but actually drowning > > in 1500 lines of code. > > :-) I hope you can eventually find what you need. > > > Werner
benchmark.html
(text/html, 14.6 KB)
<html>
<head>
<style>
table {
}
th, td {
padding: 3px;
text-align: center;
}
th {
background-color: #ccc;
color: black;
}
.warning{
color: red;
}
.col1 {
background-color: #eee;
}
.highlight {
background-color: #0a0;
}
</style>
</head>
<body>
<h1>Freetype Benchmark Results</h1>
<h2 class="warning">Warning: Baseline and Benchmark have the same commit ID!</h2>
<h2>Info</h2>
<table border="1">
<tr><th>Info</th><th>Baseline</th><th>Benchmark</th></tr>
<tr><td class="col1">Parameters</td><td>-c 500 -w 1</td><td>-c 500 -w 1</td></tr>
<tr><td class="col1">Commit ID</td><td><a href="https://gitlab.freedesktop.org/freetype/freetype/-/commit/fae5e8c941a2ab051dbb1c56e6190dbd0f3791f9">fae5e8c9</a></td><td><a href="https://gitlab.freedesktop.org/freetype/freetype/-/commit/fae5e8c941a2ab051dbb1c56e6190dbd0f3791f9">fae5e8c9</a></td></tr>
<tr><td class="col1">Commit Date</td><td>2023-08-01 17:37:55 +0300</td><td>2023-08-01 17:37:55 +0300</td></tr>
<tr><td class="col1">Branch</td><td>GSoC-2023-Ahmet</td><td>GSoC-2023-Ahmet</td></tr>
</table><br/>* Smaller values mean faster operation<br/>
<h2>Results for Roboto_subset.ttf</h2>
<table border="1">
<tr><th>Test</th><th>N</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/baseline/Roboto_subset.txt">Baseline</a> (ms)</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/benchmark/Roboto_subset.txt">Benchmark</a> (ms)</th> <th>Difference (%)</th></tr>
<tr><td class="col1">Load</td><td>59880</td> <td class="highlight">0.250</td><td class="lowlight">0.270</td><td>-8.0</td></tr>
<tr><td class="col1">Load_Advances (Normal)</td><td>59880</td> <td class="lowlight">0.311</td><td class="highlight">0.243</td><td>21.9</td></tr>
<tr><td class="col1">Load_Advances (Fast)</td><td>59880</td> <td class="highlight">0.002</td><td class="lowlight">0.002</td><td>0.0</td></tr>
<tr><td class="col1">Load_Advances (Unscaled)</td><td>59880</td> <td class="highlight">0.001</td><td class="lowlight">0.002</td><td>-100.0</td></tr>
<tr><td class="col1">Render</td><td>59880</td> <td class="lowlight">0.220</td><td class="highlight">0.218</td><td>0.9</td></tr>
<tr><td class="col1">Get_Glyph</td><td>59880</td> <td class="highlight">0.086</td><td class="lowlight">0.086</td><td>0.0</td></tr>
<tr><td class="col1">Get_Char_Index</td><td>46906</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Iterate CMap</td><td>499</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">New_Face</td><td>499</td> <td class="highlight">0.022</td><td class="lowlight">0.022</td><td>0.0</td></tr>
<tr><td class="col1">Embolden</td><td>59880</td> <td class="highlight">0.115</td><td class="lowlight">0.115</td><td>0.0</td></tr>
<tr><td class="col1">Stroke</td><td>57120</td> <td class="highlight">1.628</td><td class="lowlight">1.629</td><td>-0.1</td></tr>
<tr><td class="col1">Get_BBox</td><td>59880</td> <td class="highlight">0.055</td><td class="lowlight">0.055</td><td>0.0</td></tr>
<tr><td class="col1">Get_CBox</td><td>59880</td> <td class="lowlight">0.043</td><td class="highlight">0.042</td><td>2.3</td></tr>
<tr><td class="col1">New_Face & load glyph(s)</td><td>59880</td> <td class="highlight">0.053</td><td class="lowlight">0.054</td><td>-1.9</td></tr>
<tr><td class="col1">TOTAL</td><td class="col1">1407648</td> <td class="col1">2.788</td><td class="col1">2.740</td><td class="col1">-6.1</td></tr>
</table><br/>
<h2>Results for Arial_subset.ttf</h2>
<table border="1">
<tr><th>Test</th><th>N</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/baseline/Arial_subset.txt">Baseline</a> (ms)</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/benchmark/Arial_subset.txt">Benchmark</a> (ms)</th> <th>Difference (%)</th></tr>
<tr><td class="col1">Load</td><td>47405</td> <td class="lowlight">0.390</td><td class="highlight">0.376</td><td>3.6</td></tr>
<tr><td class="col1">Load_Advances (Normal)</td><td>47405</td> <td class="lowlight">0.344</td><td class="highlight">0.340</td><td>1.2</td></tr>
<tr><td class="col1">Load_Advances (Fast)</td><td>47405</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Load_Advances (Unscaled)</td><td>47405</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Render</td><td>47405</td> <td class="highlight">0.179</td><td class="lowlight">0.186</td><td>-3.9</td></tr>
<tr><td class="col1">Get_Glyph</td><td>47405</td> <td class="highlight">0.075</td><td class="lowlight">0.076</td><td>-1.3</td></tr>
<tr><td class="col1">Get_Char_Index</td><td>46906</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Iterate CMap</td><td>499</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">New_Face</td><td>499</td> <td class="lowlight">0.028</td><td class="highlight">0.025</td><td>10.7</td></tr>
<tr><td class="col1">Embolden</td><td>47405</td> <td class="highlight">0.103</td><td class="lowlight">0.103</td><td>0.0</td></tr>
<tr><td class="col1">Stroke</td><td>47405</td> <td class="highlight">1.324</td><td class="lowlight">1.329</td><td>-0.4</td></tr>
<tr><td class="col1">Get_BBox</td><td>47405</td> <td class="highlight">0.049</td><td class="lowlight">0.051</td><td>-4.1</td></tr>
<tr><td class="col1">Get_CBox</td><td>47405</td> <td class="highlight">0.035</td><td class="lowlight">0.036</td><td>-2.9</td></tr>
<tr><td class="col1">New_Face & load glyph(s)</td><td>47405</td> <td class="lowlight">0.060</td><td class="highlight">0.054</td><td>10.0</td></tr>
<tr><td class="col1">TOTAL</td><td class="col1">1138718</td> <td class="col1">2.591</td><td class="col1">2.580</td><td class="col1">0.9</td></tr>
</table><br/>
<h2>Results for TimesNewRoman_subset.ttf</h2>
<table border="1">
<tr><th>Test</th><th>N</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/baseline/TimesNewRoman_subset.txt">Baseline</a> (ms)</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/benchmark/TimesNewRoman_subset.txt">Benchmark</a> (ms)</th> <th>Difference (%)</th></tr>
<tr><td class="col1">Load</td><td>47405</td> <td class="highlight">0.455</td><td class="lowlight">0.464</td><td>-2.0</td></tr>
<tr><td class="col1">Load_Advances (Normal)</td><td>47405</td> <td class="lowlight">0.419</td><td class="highlight">0.412</td><td>1.7</td></tr>
<tr><td class="col1">Load_Advances (Fast)</td><td>47405</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Load_Advances (Unscaled)</td><td>47405</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Render</td><td>47405</td> <td class="lowlight">0.201</td><td class="highlight">0.197</td><td>2.0</td></tr>
<tr><td class="col1">Get_Glyph</td><td>47405</td> <td class="highlight">0.078</td><td class="lowlight">0.082</td><td>-5.1</td></tr>
<tr><td class="col1">Get_Char_Index</td><td>46906</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Iterate CMap</td><td>499</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">New_Face</td><td>499</td> <td class="highlight">0.027</td><td class="lowlight">0.027</td><td>0.0</td></tr>
<tr><td class="col1">Embolden</td><td>47405</td> <td class="lowlight">0.140</td><td class="highlight">0.139</td><td>0.7</td></tr>
<tr><td class="col1">Stroke</td><td>40185</td> <td class="highlight">1.545</td><td class="lowlight">1.545</td><td>0.0</td></tr>
<tr><td class="col1">Get_BBox</td><td>47405</td> <td class="highlight">0.058</td><td class="lowlight">0.059</td><td>-1.7</td></tr>
<tr><td class="col1">Get_CBox</td><td>47405</td> <td class="highlight">0.037</td><td class="lowlight">0.041</td><td>-10.8</td></tr>
<tr><td class="col1">New_Face & load glyph(s)</td><td>47405</td> <td class="highlight">0.069</td><td class="lowlight">0.076</td><td>-10.1</td></tr>
<tr><td class="col1">TOTAL</td><td class="col1">1124278</td> <td class="col1">3.033</td><td class="col1">3.046</td><td class="col1">-1.8</td></tr>
</table><br/>
<h2>Results for Tahoma_subset.ttf</h2>
<table border="1">
<tr><th>Test</th><th>N</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/baseline/Tahoma_subset.txt">Baseline</a> (ms)</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/benchmark/Tahoma_subset.txt">Benchmark</a> (ms)</th> <th>Difference (%)</th></tr>
<tr><td class="col1">Load</td><td>47405</td> <td class="highlight">0.265</td><td class="lowlight">0.289</td><td>-9.1</td></tr>
<tr><td class="col1">Load_Advances (Normal)</td><td>47405</td> <td class="lowlight">0.245</td><td class="highlight">0.239</td><td>2.4</td></tr>
<tr><td class="col1">Load_Advances (Fast)</td><td>47405</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Load_Advances (Unscaled)</td><td>47405</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Render</td><td>47405</td> <td class="highlight">0.166</td><td class="lowlight">0.166</td><td>0.0</td></tr>
<tr><td class="col1">Get_Glyph</td><td>47405</td> <td class="lowlight">0.075</td><td class="highlight">0.071</td><td>5.3</td></tr>
<tr><td class="col1">Get_Char_Index</td><td>46906</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Iterate CMap</td><td>499</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">New_Face</td><td>499</td> <td class="lowlight">0.025</td><td class="highlight">0.024</td><td>4.0</td></tr>
<tr><td class="col1">Embolden</td><td>47405</td> <td class="lowlight">0.118</td><td class="highlight">0.108</td><td>8.5</td></tr>
<tr><td class="col1">Stroke</td><td>47405</td> <td class="lowlight">1.244</td><td class="highlight">1.236</td><td>0.6</td></tr>
<tr><td class="col1">Get_BBox</td><td>47405</td> <td class="highlight">0.047</td><td class="lowlight">0.047</td><td>0.0</td></tr>
<tr><td class="col1">Get_CBox</td><td>47405</td> <td class="highlight">0.034</td><td class="lowlight">0.034</td><td>0.0</td></tr>
<tr><td class="col1">New_Face & load glyph(s)</td><td>47405</td> <td class="lowlight">0.053</td><td class="highlight">0.052</td><td>1.9</td></tr>
<tr><td class="col1">TOTAL</td><td class="col1">1138718</td> <td class="col1">2.276</td><td class="col1">2.270</td><td class="col1">1.0</td></tr>
</table><br/>
<h2>Results for Verdana_subset.ttf</h2>
<table border="1">
<tr><th>Test</th><th>N</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/baseline/Verdana_subset.txt">Baseline</a> (ms)</th> <th>* <a href="/Users/gks/Desktop/freetype/objs/benchmark/Verdana_subset.txt">Benchmark</a> (ms)</th> <th>Difference (%)</th></tr>
<tr><td class="col1">Load</td><td>47405</td> <td class="lowlight">0.251</td><td class="highlight">0.220</td><td>12.4</td></tr>
<tr><td class="col1">Load_Advances (Normal)</td><td>47405</td> <td class="lowlight">0.226</td><td class="highlight">0.217</td><td>4.0</td></tr>
<tr><td class="col1">Load_Advances (Fast)</td><td>47405</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Load_Advances (Unscaled)</td><td>47405</td> <td class="lowlight">0.002</td><td class="highlight">0.001</td><td>50.0</td></tr>
<tr><td class="col1">Render</td><td>47405</td> <td class="highlight">0.169</td><td class="lowlight">0.174</td><td>-3.0</td></tr>
<tr><td class="col1">Get_Glyph</td><td>47405</td> <td class="highlight">0.073</td><td class="lowlight">0.075</td><td>-2.7</td></tr>
<tr><td class="col1">Get_Char_Index</td><td>46906</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">Iterate CMap</td><td>499</td> <td class="highlight">0.001</td><td class="lowlight">0.001</td><td>0.0</td></tr>
<tr><td class="col1">New_Face</td><td>499</td> <td class="lowlight">0.024</td><td class="highlight">0.023</td><td>4.2</td></tr>
<tr><td class="col1">Embolden</td><td>47405</td> <td class="highlight">0.103</td><td class="lowlight">0.104</td><td>-1.0</td></tr>
<tr><td class="col1">Stroke</td><td>47405</td> <td class="highlight">1.230</td><td class="lowlight">1.319</td><td>-7.2</td></tr>
<tr><td class="col1">Get_BBox</td><td>47405</td> <td class="highlight">0.046</td><td class="lowlight">0.050</td><td>-8.7</td></tr>
<tr><td class="col1">Get_CBox</td><td>47405</td> <td class="highlight">0.034</td><td class="lowlight">0.040</td><td>-17.6</td></tr>
<tr><td class="col1">New_Face & load glyph(s)</td><td>47405</td> <td class="highlight">0.050</td><td class="lowlight">0.056</td><td>-12.0</td></tr>
<tr><td class="col1">TOTAL</td><td class="col1">1138718</td> <td class="col1">2.211</td><td class="col1">2.282</td><td class="col1">1.3</td></tr>
</table><br/>
<center>Freetype Benchmark</center>
</body>
</html>