Re: [Cheetahtemplate-discuss] [PATCH] Use cStringIO instead of array in DummyTransaction

[<ty...@mo...>]

On Thu, 15 Oct 2009, Tavis Rudd wrote:

> Fyi, if I recall correctly, Cheetah originally used StringIO,
> cStringIO if it was available, but I found ''.join([]) was actually
> faster.  There's probably a note about it in the CHANGES file.


Indeed::

	1.0rc2 (Nov 19, 2005)

	  !!!THIS RELEASE REQUIRES RECOMPILATION OF ALL COMPILED CHEETAH TEMPLATES!!!

	  See my email to the cheetahtemplate-discuss list on Sat. Nov. 12th for more
	  details on these changes:

	  - faster list-based buffering in DummyTrans, rather than StringIO (my 
		benchmarks showed it to be significantly faster.  collections.deque wasn't 
		any faster than a simple list.) [TR]


That was so long ago! I wasn't even aware of Python at that point I
don't think ;)


Cheers

> 
> On Wed, 14 Oct 2009, ty...@mo... wrote:
> 
> >
> >On Wed, 14 Oct 2009, Aahz wrote:
> >
> >>On Wed, Oct 14, 2009, R. Tyler Ballance wrote:
> >>>
> >>>+try:
> >>>+    from cStringIO import StringIO
> >>>+except ImportError:
> >>>+    import StringIO
> >>>+
> >>
> >>It's not clear what the goal is.  IIRC, using ''.join() is roughly
> >>comparable in performance with cStringIO (which one is faster varies from
> >>version to version), and if for whatever reason cStringIO is not
> >>available, StringIO is definitely going to perform worse.
> >
> >Gahh you're right, what was I thinking. I didn't actually add a
> >performance test prior to cherry-picking Arun's change. I had *assumed*
> >(first mistake) that cStringIO would be performing the majority of its
> >"work" on stream.getvalue() which isn't the case.
> >
> >Looking at the performance tests that I just ran, as well as the
> >cStringIO.c code, stream.write() will actually copy (memcpy(2)) the
> >buffer into the stream's buffer. Whereas list.append() will add a
> >pointer to a list, periodically over-extending it to accomodate
> >additional append() calls.
> >
> >What I find curious is that StringIO *effectively* does a bunch of
> >list.append() calls, but it is in fact slower than cStringIO (at least
> >on 2.6) as the code is performing some really silly ''.join() calls
> >and restructuring/rebuilding the array in the stream.write() call (given
> >certain conditions).
> >
> >I'll discuss with Arun where he saw the performance improvements with
> >cStringIO when he wakes up, but for now I'm just going to partially back
> >these changes out.
> >
> >
> >Thanks for pointing the fail out Rahz :)
> >
> >
> >Cheers,
> >-R. Tyler Ballance
> >