Re: Dispatch vs EnsureDispatch performance
Mark Hammond <[email protected]> Mon, 26 Aug 2024 16:20:06 -0400
| Newsgroups | gmane.comp.python.windows |
|---|---|
| Message-ID | <[email protected]> |
This is a multi-part message in MIME format. --===============4081882766719453681== Content-Type: multipart/alternative; boundary="------------X2rVftZIq0XxpIDI2xXrdsfx" Content-Language: en-US This is a multi-part message in MIME format. --------------X2rVftZIq0XxpIDI2xXrdsfx Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit On 2024-07-17 10:30 a.m., Sven Bardos via python-win32 wrote: > Hi Mark, > > I found some time to debug and profile my issue a little bit more. > In my scenario I call a COM method which returns a list of 14195 > integers plus the amount of the found objects. > > If I profile that call I get the following: > > ncalls tottime percall cumtime percall filename:lineno(function) > ... > 1 0.000 0.000 0.025 0.025 __init__.py:572(_ApplyTypes_) > 1 0.000 0.000 0.007 0.007 __init__.py:603(_get_good_object_) > 14195 0.001 0.000 0.001 0.000 > __init__.py:608(_get_good_single_object_) > 14198/1 0.005 0.000 0.007 0.007 > __init__.py:614(_get_good_object_) > ... > > So _get_good_object_ is called ~14200 times. > If I do that in a loop 100x, I get an execution time of ~3.3s > > If I change the code in the _ApplyTypes_ function to: > > def _ApplyTypes_(self, dispid, wFlags, retType, argTypes, user, > resultCLSID, *args): > return self._oleobj_.InvokeTypes(dispid, 0, wFlags, retType, argTypes, > *args) > # return self._get_good_object_( > # self._oleobj_.InvokeTypes(dispid, 0, wFlags, retType, argTypes, > *args), > # user, > # resultCLSID, > # ) > > that is, avoiding the _get_good_object_() calls, the execution time is > ~1.6s for 100 iterations. Which is ~50% of the original time. The > result is the same, in this case. I'm certainly up for looking at optimizations to how this works, but I don't think it's as simple as not calling the function - there are cases where it would return different objects. I don't think an instance method is needed (ie, maybe we could just call the final implementation and maybe even inline some of those calls, but if you say the result is identical, it looks a little like there are just 2 isinstance() calls which return False in this scenario, so maybe this might also offer optimization ideas (eg, if the type in question for your examples are builtin types like int/string/etc, maybe these could be checked before making any other calls?). HTH, Mark > > Do you have any thoughts on this? > > Best regards, > Sven > > Am Mi., 17. Apr. 2024 um 17:19 Uhr schrieb Mark Hammond > <[email protected]>: > > I'm mildly surprised by that - a profiler might show some > low-hanging fruit, and/or might show different characteristics > when many more functions are used. However, the primary reason for > EnsureDispatch is for better support of the object model - there's > far more context available and this less chance of upsetting some > COM objects - eg, when `foo.bar` is seen, EnsureDispatch knows for > sure that `bar` is a method, but dynamic dispatch doesn't know if > the resulting object is going to be called or not. > > HTH, > > Mark > > On 2024-04-17 2:07 a.m., Sven Bardos via python-win32 wrote: >> Hi, >> >> shouldn't be EnsureDispatch be faster than Dispatch once the code >> generation is done? >> >> I've measured it by calling 6000 COM calls like this: >> >> dirpath = Path('C:/Users/sbardos/AppData/Local/Temp/gen_py/3.10/') >> if dirpath.exists() and dirpath.is_dir(): >> shutil.rmtree(dirpath) >> >> app = Dispatch("CT.Application") >> job = app.CreateJobObject() >> >> start = timer() >> >> for i in range(2000): >> cnt, devIds = job.GetAllDeviceIds() >> cnt, sheetIds = job.GetSheetIds() >> dev = job.CreateDeviceObject() >> >> end = timer() >> print(f"Time ellapsed (late): {end - start}s") >> and the ensure Dispatch version: >> app = EnsureDispatch("CT.Application") >> job = app.CreateJobObject() >> start = timer() >> for i in range(2000): >> cnt, devIds = job.GetAllDeviceIds(None) >> cnt, sheetIds = job.GetSheetIds(None) >> dev = job.CreateDeviceObject() >> end = timer() >> print(f"Time ellapsed (early): {end - start}s") >> EnsureDispatch is a little bit slower ~4.2s compared to ~4.0s. >> If I don't get a performance boost with EnsureDispatch, is there >> even a point using it? >> >> Thanks, >> Sven >> >> _______________________________________________ >> python-win32 mailing list >> [email protected] >> https://mail.python.org/mailman/listinfo/python-win32 > > > _______________________________________________ > python-win32 mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-win32 --------------X2rVftZIq0XxpIDI2xXrdsfx Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> <p>On 2024-07-17 10:30 a.m., Sven Bardos via python-win32 wrote:</p> <blockquote type="cite" cite="mid:CAP10ovyvggv-ecB6AWc7KCg+_Vvxfo3n_QtWDgF=_v-UMAaTeA@mail.gmail.com"> <meta http-equiv="content-type" content="text/html; charset=UTF-8"> <div dir="ltr"> <div dir="ltr">Hi Mark, <div><br> <div>I found some time to debug and profile my issue a little bit more.</div> </div> <div>In my scenario I call a COM method which returns a list of 14195 integers plus the amount of the found objects.</div> <div><br> </div> <div>If I profile that call I get the following:</div> <div><br> </div> <div><font face="garamond, times new roman, serif">ncalls tottime percall cumtime percall filename:lineno(function)<br> </font></div> <div><font face="garamond, times new roman, serif">...</font></div> <div><font face="garamond, times new roman, serif">1 0.000 0.000 0.025 0.025 __init__.py:572(_ApplyTypes_)<br> 1 0.000 0.000 0.007 0.007 __init__.py:603(_get_good_object_)<br> 14195 0.001 0.000 0.001 0.000 __init__.py:608(_get_good_single_object_)<br> 14198/1 0.005 0.000 0.007 0.007 __init__.py:614(_get_good_object_)<br> </font></div> <div>...</div> <div><br> </div> <div>So _get_good_object_ is called ~14200 times.</div> <div>If I do that in a loop 100x, I get an execution time of ~3.3s</div> <div><br> </div> <div>If I change the code in the <font face="garamond, times new roman, serif">_ApplyTypes_ </font>function to:</div> <div><br> </div> <div style="color:rgb(204,204,204);background-color:rgb(31,31,31);font-family:Consolas,"Courier New",monospace;font-size:14px;line-height:19px;white-space:pre"><div> <span style="color:rgb(86,156,214)">def</span> <span style="color:rgb(220,220,170)">_ApplyTypes_</span>(<span style="color:rgb(156,220,254)">self</span>, <span style="color:rgb(156,220,254)">dispid</span>, <span style="color:rgb(156,220,254)">wFlags</span>, <span style="color:rgb(156,220,254)">retType</span>, <span style="color:rgb(156,220,254)">argTypes</span>, <span style="color:rgb(156,220,254)">user</span>, <span style="color:rgb(156,220,254)">resultCLSID</span>, <span style="color:rgb(212,212,212)">*</span><span style="color:rgb(156,220,254)">args</span>):</div><div> <span style="color:rgb(197,134,192)">return</span> <span style="color:rgb(156,220,254)">self</span>._oleobj_.InvokeTypes(<span style="color:rgb(156,220,254)">dispid</span>, <span style="color:rgb(181,206,168)">0</span>, <span style="color:rgb(156,220,254)">wFlags</span>, <span style="color:rgb(156,220,254)">retType</span>, <span style="color:rgb(156,220,254)">argTypes</span>, <span style="color:rgb(212,212,212)">*</span><span style="color:rgb(156,220,254)">args</span>)</div><div> <span style="color:rgb(106,153,85)"># return self._get_good_object_(</span></div><div> <span style="color:rgb(106,153,85)"># self._oleobj_.InvokeTypes(dispid, 0, wFlags, retType, argTypes, *args),</span></div><div> <span style="color:rgb(106,153,85)"># user,</span></div><div> <span style="color:rgb(106,153,85)"># resultCLSID,</span></div><div> <span style="color:rgb(106,153,85)"># )</span></div></div> <div><br> </div> <div>that is, avoiding the _get_good_object_() calls, the execution time is ~1.6s for 100 iterations. Which is ~50% of the original time. The result is the same, in this case.</div> </div> </div> </blockquote> <p>I'm certainly up for looking at optimizations to how this works, but I don't think it's as simple as not calling the function - there are cases where it would return different objects. I don't think an instance method is needed (ie, maybe we could just call the final implementation and maybe even inline some of those calls, but if you say the result is identical, it looks a little like there are just 2 isinstance() calls which return False in this scenario, so maybe this might also offer optimization ideas (eg, if the type in question for your examples are builtin types like int/string/etc, maybe these could be checked before making any other calls?).</p> <p>HTH,</p> <p>Mark<br> </p> <blockquote type="cite" cite="mid:CAP10ovyvggv-ecB6AWc7KCg+_Vvxfo3n_QtWDgF=_v-UMAaTeA@mail.gmail.com"> <div dir="ltr"> <div dir="ltr"> <div><br> </div> <div>Do you have any thoughts on this?</div> <div><br> </div> <div>Best regards,</div> <div>Sven</div> <div> </div> </div> <br> <div class="gmail_quote"> <div dir="ltr" class="gmail_attr">Am Mi., 17. Apr. 2024 um 17:19 Uhr schrieb Mark Hammond <<a href="mailto:[email protected]" moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a>>:<br> </div> <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <p>I'm mildly surprised by that - a profiler might show some low-hanging fruit, and/or might show different characteristics when many more functions are used. However, the primary reason for EnsureDispatch is for better support of the object model - there's far more context available and this less chance of upsetting some COM objects - eg, when `foo.bar` is seen, EnsureDispatch knows for sure that `bar` is a method, but dynamic dispatch doesn't know if the resulting object is going to be called or not.</p> <p>HTH,</p> <p>Mark<br> </p> <div>On 2024-04-17 2:07 a.m., Sven Bardos via python-win32 wrote:<br> </div> <blockquote type="cite"> <div dir="ltr">Hi, <div><br> <div>shouldn't be EnsureDispatch be faster than Dispatch once the code generation is done?</div> <div><br> </div> <div>I've measured it by calling 6000 COM calls like this:</div> <div><br> </div> <div> <div style="color:rgb(204,204,204);background-color:rgb(31,31,31);font-family:Consolas,"Courier New",monospace;font-size:14px;line-height:19px;white-space:pre-wrap"><div><span style="color:rgb(156,220,254)">dirpath</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(78,201,176)">Path</span>(<span style="color:rgb(206,145,120)">'C:/Users/sbardos/AppData/Local/Temp/gen_py/3.10/'</span>)</div><div><span style="color:rgb(197,134,192)">if</span> <span style="color:rgb(156,220,254)">dirpath</span>.<span style="color:rgb(220,220,170)">exists</span>() <span style="color:rgb(86,156,214)">and</span> <span style="color:rgb(156,220,254)">dirpath</span>.<span style="color:rgb(220,220,170)">is_dir</span>():</div><div> <span style="color:rgb(78,201,176)">shutil</span>.<span style="color:rgb(156,220,254)">rmtree</span>(<span style="color:rgb(156,220,254)">dirpath</span>)</div></div> </div> <div><br> </div> <div> <div style="line-height:19px"> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"><span style="color:rgb(156,220,254)">app</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(220,220,170)">Dispatch</span>(<span style="color:rgb(206,145,120)">"CT.Application"</span>)</div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"><span style="color:rgb(156,220,254)">job</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">app</span>.CreateJobObject()</div> <br> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"><span style="color:rgb(156,220,254)">start</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">timer</span>()</div> <br> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"><span style="color:rgb(197,134,192)">for</span> <span style="color:rgb(156,220,254)">i</span> <span style="color:rgb(197,134,192)">in</span> <span style="color:rgb(78,201,176)">range</span>(<span style="color:rgb(181,206,168)">2000</span>):</div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"> <span style="color:rgb(156,220,254)">cnt</span>, <span style="color:rgb(156,220,254)">devIds</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">job</span>.GetAllDeviceIds()</div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"> <span style="color:rgb(156,220,254)">cnt</span>, <span style="color:rgb(156,220,254)">sheetIds</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">job</span>.GetSheetIds()</div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"> <span style="color:rgb(156,220,254)">dev</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">job</span>.CreateDeviceObject()</div> <br> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"><span style="color:rgb(156,220,254)">end</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">timer</span>()</div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"><span style="color:rgb(220,220,170)">print</span>(<span style="color:rgb(86,156,214)">f</span><span style="color:rgb(206,145,120)">"Time ellapsed (late): </span><span style="color:rgb(86,156,214)">{</span><span style="color:rgb(156,220,254)">end</span> <span style="color:rgb(212,212,212)">-</span> <span style="color:rgb(156,220,254)">start</span><span style="color:rgb(86,156,214)">}</span><span style="color:rgb(206,145,120)">s"</span>)</div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"> </div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)">and the ensure Dispatch version:</div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31)"> </div> <div> <div style="color:rgb(204,204,204);font-family:Consolas,"Courier New",monospace;font-size:14px;white-space:pre-wrap;background-color:rgb(31,31,31);line-height:19px"><div><span style="color:rgb(156,220,254)">app</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(220,220,170)">EnsureDispatch</span>(<span style="color:rgb(206,145,120)">"CT.Application"</span>)</div><div><span style="color:rgb(156,220,254)">job</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">app</span>.CreateJobObject()</div> <div><span style="color:rgb(156,220,254)">start</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">timer</span>()</div> <div><span style="color:rgb(197,134,192)">for</span> <span style="color:rgb(156,220,254)">i</span> <span style="color:rgb(197,134,192)">in</span> <span style="color:rgb(78,201,176)">range</span>(<span style="color:rgb(181,206,168)">2000</span>):</div><div> <span style="color:rgb(156,220,254)">cnt</span>, <span style="color:rgb(156,220,254)">devIds</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">job</span>.GetAllDeviceIds(<span style="color:rgb(86,156,214)">None</span>)</div><div> <span style="color:rgb(156,220,254)">cnt</span>, <span style="color:rgb(156,220,254)">sheetIds</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">job</span>.GetSheetIds(<span style="color:rgb(86,156,214)">None</span>)</div><div> <span style="color:rgb(156,220,254)">dev</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">job</span>.CreateDeviceObject()</div> <div><span style="color:rgb(156,220,254)">end</span> <span style="color:rgb(212,212,212)">=</span> <span style="color:rgb(156,220,254)">timer</span>()</div><div><span style="color:rgb(220,220,170)">print</span>(<span style="color:rgb(86,156,214)">f</span><span style="color:rgb(206,145,120)">"Time ellapsed (early): </span><span style="color:rgb(86,156,214)">{</span><span style="color:rgb(156,220,254)">end</span> <span style="color:rgb(212,212,212)">-</span> <span style="color:rgb(156,220,254)">start</span><span style="color:rgb(86,156,214)">}</span><span style="color:rgb(206,145,120)">s"</span>)</div><div> </div><div> </div></div> EnsureDispatch is a little bit slower ~4.2s compared to ~4.0s.<br> If I don't get a performance boost with EnsureDispatch, is there even a point using it?<br> <br> Thanks,<br> Sven<br> </div> </div> </div> </div> </div> <br> <fieldset></fieldset> <pre>_______________________________________________ python-win32 mailing list <a href="mailto:[email protected]" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">[email protected]</a> <a href="https://mail.python.org/mailman/listinfo/python-win32" target="_blank" moz-do-not-send="true" class="moz-txt-link-freetext">https://mail.python.org/mailman/listinfo/python-win32</a> </pre> </blockquote> </div> </blockquote> </div> </div> <br> <fieldset class="moz-mime-attachment-header"></fieldset> <pre wrap="" class="moz-quote-pre">_______________________________________________ python-win32 mailing list <a class="moz-txt-link-abbreviated" href="mailto:[email protected]">[email protected]</a> <a class="moz-txt-link-freetext" href="https://mail.python.org/mailman/listinfo/python-win32">https://mail.python.org/mailman/listinfo/python-win32</a> </pre> </blockquote> </body> </html> --------------X2rVftZIq0XxpIDI2xXrdsfx-- --===============4081882766719453681== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ python-win32 mailing list [email protected] https://mail.python.org/mailman/listinfo/python-win32 --===============4081882766719453681==--