Re: Slowdown on GAE using pre-compiled templates

On Wed, 14 Oct 2009, Arun Kumar wrote:

> i came across a problem using Cheetah on Google App Engine (GAE). When i 
> used pre-compiled templates, the time to render some 500 rows of a table 
> was over 140 seconds. Using dynamically compiled templates the page was 
> rendering in about 200ms.
> 
> After chatting with rtyler on irc and doing some profile tests, we 
> managed to narrow the problem down to the valueFromFrameOrSearchList 
> call from Cheetah/NameMapper.py. Which appears to be the main difference 
> between the two. The dynamically compiled templates uses 
> valueFromSearchList (and not VFFSL).

To help back this up with some of the numbers we were looking at, here's
some of the profiler results from running the pre-compiled template with
useStackFrames=True:

	85442921 function calls (85412686 primitive calls) in 141.942 CPU seconds

	   Ordered by: internal time
	   List reduced from 153 to 80 due to restriction <80>

	   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
	160087/160064   48.591    0.000  103.762    0.001 /usr/lib/python2.5/inspect.py:408(getmodule)
	 19768401   26.590    0.000   36.291    0.000 /usr/lib/python2.5/inspect.py:35(ismodule)
	 22574035   11.350    0.000   11.350    0.000 {isinstance}
	 15176510   10.810    0.000   10.810    0.000 {hasattr}
	 14517734    6.690    0.000    6.690    0.000 {method 'get' of 'dict' objects}
	   286859    4.140    0.000    6.590    0.000 /usr/lib/python2.5/posixpath.py:373(normpath)
	   286831    3.410    0.000   13.300    0.000 /home/arun/projects/libraries/google_appengine/google/appengine/tools/dev_appserver.py:1106(__call__)
	   525212    2.690    0.000    2.690    0.000 {built-in method match}
	280312/280112    2.530    0.000   39.211    0.000 /usr/lib/python2.5/inspect.py:381(getsourcefile)
	   140056    2.400    0.000  119.022    0.001 /usr/lib/python2.5/inspect.py:454(findsource)
	   290316    2.050    0.000    7.140    0.000 /usr/lib/python2.5/inspect.py:342(getfile)
	   140056    1.750    0.000  142.182    0.001 /usr/lib/python2.5/inspect.py:820(getframeinfo)
		70029    1.510    0.000    1.510    0.000 {method 'items' of 'dict' objects}
	  2592375    1.360    0.000    1.360    0.000 {method 'append' of 'list' objects


While the slowdown doesn't directly come from Cheetah itself, it comes
from Google App Engine's suboptimal sandboxing implementation of the
'inspect' module (and related features).


> One option to circumvent this problem has been to use the 
> 'useStackFrames=False' #compiler-settings. That seems to get the page to 
> render at about the same speed as the dynamically generated template.
> 
> Let me know if anyone wants the profile tests, I can send you a link to 
> a small GAE app, which you can run (using the GAE SDK) and check results.


Thanks for the mail Arun; I'm looking forward to further refining your
cStringIO patchset to squeeze a little more performance out of Cheetah
:)


Cheers,
-R. Tyler Ballance

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference

_______________________________________________
Cheetahtemplate-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkrWFosACgkQFCbH3D9R4W+xOQCgi0TKd7AYbzBdF2sVNrngP06m
YrcAn2P5bLefb3HGTHcyDFIKiU4EoHZ8
=5dUl
-----END PGP SIGNATURE-----