Re: [PATCH] Use cStringIO instead of array in DummyTransaction

On Thu, 15 Oct 2009, Tavis Rudd wrote:

> Fyi, if I recall correctly, Cheetah originally used StringIO,
> cStringIO if it was available, but I found ''.join([]) was actually
> faster.  There's probably a note about it in the CHANGES file.


Indeed::

	1.0rc2 (Nov 19, 2005)

	  !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!!

	  See my email to the cheetahtemplate-discuss list on Sat. Nov. 12th for more
	  details on these changes:

	  - faster list-based buffering in DummyTrans, rather than StringIO (my 
		benchmarks showed it to be significantly faster.  collections.deque wasn't 
		any faster than a simple list.) [TR]


That was so long ago! I wasn't even aware of Python at that point I
don't think ;)


Cheers

> 
> On Wed, 14 Oct 2009, [email protected] wrote:
> 
> >
> >On Wed, 14 Oct 2009, Aahz wrote:
> >
> >>On Wed, Oct 14, 2009, R. Tyler Ballance wrote:
> >>>
> >>>+try:
> >>>+    from cStringIO import StringIO
> >>>+except ImportError:
> >>>+    import StringIO
> >>>+
> >>
> >>It's not clear what the goal is.  IIRC, using ''.join() is roughly
> >>comparable in performance with cStringIO (which one is faster varies from
> >>version to version), and if for whatever reason cStringIO is not
> >>available, StringIO is definitely going to perform worse.
> >
> >Gahh you're right, what was I thinking. I didn't actually add a
> >performance test prior to cherry-picking Arun's change. I had *assumed*
> >(first mistake) that cStringIO would be performing the majority of its
> >"work" on stream.getvalue() which isn't the case.
> >
> >Looking at the performance tests that I just ran, as well as the
> >cStringIO.c code, stream.write() will actually copy (memcpy(2)) the
> >buffer into the stream's buffer. Whereas list.append() will add a
> >pointer to a list, periodically over-extending it to accomodate
> >additional append() calls.
> >
> >What I find curious is that StringIO *effectively* does a bunch of
> >list.append() calls, but it is in fact slower than cStringIO (at least
> >on 2.6) as the code is performing some really silly ''.join() calls
> >and restructuring/rebuilding the array in the stream.write() call (given
> >certain conditions).
> >
> >I'll discuss with Arun where he saw the performance improvements with
> >cStringIO when he wakes up, but for now I'm just going to partially back
> >these changes out.
> >
> >
> >Thanks for pointing the fail out Rahz :)
> >
> >
> >Cheers,
> >-R. Tyler Ballance
> >

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference

_______________________________________________
Cheetahtemplate-discuss mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/cheetahtemplate-discuss

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)

iEYEARECAAYFAkrXqc8ACgkQFCbH3D9R4W+euwCcCKPaA4sjFvKiiKaeCWdRrvm5
Z8EAoJiToatnKB30SjZVEAfGjETBTOX5
=i3XU
-----END PGP SIGNATURE-----