ZPT Performance Twiddling
Tres Seaver <[email protected]> Sun, 04 Sep 2005 17:46:52 -0400
| Newsgroups | gmane.comp.web.zope.page-templates |
|---|---|
| Message-ID | <[email protected]> |
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
I noticed some render-time issues while optimizing a set of pages for a
client, and decided to try creating a cut-down test case.
For background: my laptop is pretty fast (I've clocked it at 50K+
pystones), and has been able to do around 460 requests/second on a
trivial ZPT page in the root of the ZODB, measured using 'ab -n 1000 -c
4' (ab running on the same machine).
I wanted to see if ZopePageTemplate is measurably slower than "plain"
PageTemplates, but did *not* want to measure security checking (other
than the per-request overhead ZPT incurs to set up the security
context). I therefore set the template up to render dynamic data from
"simple" Python data structures (lists and dicts). That overhead (see
the attached spreadsheet) seems to be about 100-125 usec per rendering,
with the difference *dropping* as the page increases in complexity.
A quick summary:
- A 'null' template (lambda returning a static string), runs in
0.653 usec.
- An empty ZPT runs in 168 usec (the equvalent non-Zope PT
runs in 52 usec).
- A ZPT which just defines a single name runs in 233 usec (PT,
115 usec).
- A ZPT which uses the defined name to fill content of one tag
runs in 299 usec (PT, 174 usec).
- The real slowdown comes with 'tal:repeat': a ZPT which repeats
over an empty sequence runs in 310 usec, increasing to 1.1 msec
for a list of 25 (PT, 193 usec - 1 msec).
- Actually doing replacement inside the loop drives the times up
(ZPT 312 usec - 3.74 msec, PT 194 usec - 3.62 msec).
- Nested loops (the case I need to optimize) are pretty bad:
7 msec for an outer loop of 20 and an inner loop of 2, etc.).
Each additional iteration (in either loop) seems to cost around
85 usec. Note that in this case (20 and 2), the maximum throughput
on the server (without security or publisher overhead) is ~141
requests / second, which is fairly pathetic, given how little
work is actually being done.
I've twiddled with a StringIO replacement whose 'getvalue' returns an
IStreamIterator, rather than the joined string, but don't see any win
there (it comes in slightly slower than the ZPT, but my system isn't
RAM-constrained either). Such a string buffer might still be a win when
handed off to the publisher, assuming that the iterator's chunks are
of a reasonable size (I haven't tried this yet).
I would really like to have a faster TALInterpreter than the one we have
now, but don't have enough zen in the guts of the shared datastructure
produced by the parser to write one.
Any thoughts? I'm attaching the script and the spreadsheet with my
results, in case anyone wants to dig further.
Tres.
- --
===================================================================
Tres Seaver +1 202-558-7113 [email protected]
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFDG2tM+gerLs4ltQ4RAribAKDVdOKBgdo3F0MbL4vLEj/o1LmNrQCfQvM9
AESRMs1o6BPGaaS7R32XFtY=
=Z98D
-----END PGP SIGNATURE-----
_______________________________________________
ZPT mailing list
[email protected]
http://mail.zope.org/mailman/listinfo/zpt
perftest_zpt.py
(application/x-httpd-cgi, 7.2 KB)
""" Speed test of page template rendering over various features
Usage: python perftest_zpt.py [-r] [-c #] [-g #] <which> <features*>
Options:
--------
'-r' -- Print the generated, rendered template and exit.
'-c' -- Specify the number of children (default 26).
'-g' -- Specify the number of grandchildren (default 0).
Arguments
---------
<which> -- one of 'null' (default), 'PT', or 'ZPT'.
<features*> -- one or more of the following (order may be significant):
'tal_define' -- set up the 'info' name (a prereq. for all the others).
'tal_content' -- include a tag which uses 'tal:content'
'tal_replace_structure' -- include a tag which uses 'tal:replace' with
the 'structure' modifier (suppresses HTML quoting)
'tal_repeat_prolog' -- set up a repeat over the children
'child_link' -- include an <a href=...> for the child (requires
'tal_repeat_prolog' before).
'grandchildren' -- build a nested list of children / grandchildren
"""
import getopt
import string
import sys
import timeit
_FEATURES = {
'tal_define': '<div tal:define="info options/info">',
'tal_content': '<h1 tal:content="info/title">TITLE</h1>',
'tal_replace_structure': '<p>Description: '
'<span tal:replace="structure info/description" />'
'</p>',
'tal_repeat_prolog': '<ul>\n'
'<li tal:repeat="child info/children">\n'
'<a href="#"\n',
'child_link': 'tal:attributes="href child/id"\n'
'tal:content="child/title"\n',
'tal_repeat_epilog': '>CHILD</a>\n'
'</li>\n'
'</ul>\n',
'grandchildren': '<ul>\n'
'<li tal:repeat="child info/children">\n'
'<span tal:content="child/id">CHILD ID</span>\n'
'<ul>\n'
'<li tal:repeat="grand child/children">\n'
'<span tal:replace="structure grand/title">TITLE</span>\n'
'</li>\n'
'</ul>\n'
'</li>\n'
'</ul>\n',
'epilog': '</div>',
}
def _cleanupFeatures(features):
features = list(features)
if 'tal_repeat_prolog' in features and 'tal_repeat_epilog' not in features:
features.append('tal_repeat_epilog')
if 'tal_define' in features and 'epilog' not in features:
features.append('epilog')
return tuple(features)
def _generateTempateText(features):
return '\n'.join([_FEATURES[x] for x in features])
def _prepareNull(ignored):
return lambda **kw: 'XYZZY'
def _preparePageTemplate(features):
from Products.PageTemplates.PageTemplate import PageTemplate
template = PageTemplate()
template.write(_generateTempateText(features))
return template
def _prepareZopePageTemplate(features):
import Zope2
from Products.PageTemplates.ZopePageTemplate import ZopePageTemplate
app = Zope2.app()
return ZopePageTemplate(id='test',
text=_generateTempateText(features)).__of__(app)
def _prepareZPT_with_StreamIterator(features):
import Zope2
from ZPublisher.Iterators import IStreamIterator
from Products.PageTemplates.ZopePageTemplate import ZopePageTemplate
app = Zope2.app()
class StreamIterator:
__implements__ = (IStreamIterator,)
def __init__(self):
self._buffers = []
self._length = 0
def write(self, data):
assert isinstance(data, basestring)
self._buffers.append(data)
self._length += len(data)
def __iter__(self):
return self
def next(self):
if len(self._buffers) == 0:
raise StopIteration
result, self._buffers = self._buffers[0], self._buffers[1:]
return result
def getvalue(self):
return self
def __len__(self):
return self._length
def __nonzero__(self):
return True
def asString(self):
return ''.join(self._buffers)
class ZPT_with_StreamIterator(ZopePageTemplate):
def StringIO(self):
return StreamIterator()
return ZPT_with_StreamIterator(id='test',
text=_generateTempateText(features),
).__of__(app)
def _prepareTemplateAndInfo(which, features, num_children, num_grandchildren):
WHICH = {'ZPT' : _prepareZopePageTemplate,
'ZPTSI' : _prepareZPT_with_StreamIterator,
'PT' : _preparePageTemplate,
'null' : _prepareNull,
}
template = WHICH[which](features)
info = {'title': 'Page Title',
'description': 'A longer description',
}
info['children'] = children = []
for child_id in string.uppercase[:num_children]:
child_info = {'id': child_id,
'title': 'Child: %s' % child_id,
'description': 'Describing child: %s' % child_id,
}
child_info['children'] = grandchildren = []
for grandchild_id in string.lowercase[:num_grandchildren]:
grandchild_info = {'id': grandchild_id,
'title': 'Grandchild: %s' % grandchild_id,
'description': 'Describing grandchild: %s'
% grandchild_id,
}
grandchildren.append(grandchild_info)
children.append(child_info)
return template, info
def main():
which = 'null'
num_children = len(string.uppercase)
num_grandchildren = 0
render = False
features = ()
try:
opts, args = getopt.getopt(sys.argv[1:], 'c:g:r?h')
except getopt.GetoptError, e:
print __doc__
print e
sys.exit(1)
for k, v in opts:
if k == '-c':
num_children = int(v)
elif k == '-g':
num_grandchildren = int(v)
elif k == '-r':
render = True
elif k in ('-h', '-?'):
print __doc__
sys.exit(2)
if args:
which = args[0]
features = args[1:]
features = _cleanupFeatures(features)
print ('Which : %s\n'
'Features : %s\n'
'# children : %s\n'
'# grands : %s' % (
which, ', '.join(features), num_children, num_grandchildren))
if render:
template, info = _prepareTemplateAndInfo(which,
features,
num_children,
num_grandchildren)
result = template(info=info)
print getattr(result, 'asString', lambda: result)()
return
NUMBER = 10000
setup = ('from perftest_zpt import _prepareTemplateAndInfo; '
'template, info = _prepareTemplateAndInfo("%s", %s, %s, %s)'
% (which, features, num_children, num_grandchildren)
)
run = 'template(info=info)'
t = timeit.Timer(stmt=run, setup=setup)
try:
times = t.repeat(number=NUMBER)
best = min(times)
print 'Time : %5.3g ms/rendering' % (best * 1000 / NUMBER)
except:
t.print_exc()
if __name__ == '__main__':
main()
zpt_perftest.xls
(application/vnd.ms-excel, 18 KB) - not displayed