Re: [PATCH] Use cStringIO instead of array in DummyTransaction

Fyi, if I recall correctly, Cheetah originally used StringIO,
cStringIO if it was available, but I found ''.join([]) was actually 
faster.  There's probably a note about it in the CHANGES file.

On Wed, 14 Oct 2009, [email protected] wrote:

>
> On Wed, 14 Oct 2009, Aahz wrote:
>
>> On Wed, Oct 14, 2009, R. Tyler Ballance wrote:
>>>
>>> +try:
>>> +    from cStringIO import StringIO
>>> +except ImportError:
>>> +    import StringIO
>>> +
>>
>> It's not clear what the goal is.  IIRC, using ''.join() is roughly
>> comparable in performance with cStringIO (which one is faster varies from
>> version to version), and if for whatever reason cStringIO is not
>> available, StringIO is definitely going to perform worse.
>
> Gahh you're right, what was I thinking. I didn't actually add a
> performance test prior to cherry-picking Arun's change. I had *assumed*
> (first mistake) that cStringIO would be performing the majority of its
> "work" on stream.getvalue() which isn't the case.
>
> Looking at the performance tests that I just ran, as well as the
> cStringIO.c code, stream.write() will actually copy (memcpy(2)) the
> buffer into the stream's buffer. Whereas list.append() will add a
> pointer to a list, periodically over-extending it to accomodate
> additional append() calls.
>
> What I find curious is that StringIO *effectively* does a bunch of
> list.append() calls, but it is in fact slower than cStringIO (at least
> on 2.6) as the code is performing some really silly ''.join() calls
> and restructuring/rebuilding the array in the stream.write() call (given
> certain conditions).
>
> I'll discuss with Arun where he saw the performance improvements with
> cStringIO when he wakes up, but for now I'm just going to partially back
> these changes out.
>
>
> Thanks for pointing the fail out Rahz :)
>
>
> Cheers,
> -R. Tyler Ballance
>

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay 
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference