[PyCS-devel] New in CVS: StringCollector pattern for modules

[Georg B. <gb...@mu...>]

Hi!

I just commited a change to PyCS that speeds up construction of module
output. I added it to referrers.py, searches.py and zeitgeist.py, as
those are the modules with largest output. Other places could use that,
too. It's mostly for situations where strings are concatenated to
produce large output:

s =3D ''
s +=3D 'some HTML code'
=2E.. repeat until doomsday ...

page['body'] =3D s

This will be recoded as:

from string_collector import StringCollector

s =3D StringCollector(set.documentEncoding())

s +=3D 'some HTML code'
=2E.. repeat until doomsday ...

page['body'] =3D str(s)

This uses internally cStringIO, so it's much faster than the old string
+=3D operator. It transparently encodes unicode stuff as the
documentEncoding, so you don't need to worry about unicode strings, too.

Another speedup would be to introduce threading for module calls: that
way module calls won't block the main server while they are gathering
data (referrers.py or searches.py still take up some time to fetch data
from the database). But that will require some more changes, especially
since the metakit database needs to be protected against parallel writes
(and the setting object needs to be protected, too). So that's left for
some other day or programmer ;-)

bye, Georg