[Htmltmpl] Performance on long loops
Brought to you by:
jakubvrana,
tripiecz
|
From: Daniel L. <dan...@gm...> - 2004-05-05 11:05:10
|
Hi,
I'm using htmltmpl in a medium-sized web site and I'm delighted with its ease
of use and flexibility. This is a trivial patch against htmltmpl-1.22 that
greatly helps htmltmpl's performance on pages with very long and/or nested
loops (e.g. a table containing hundreds of rows). Those pages tend to spend a
lot of time in htmltmpl's function "process" where the individual tokens are
analyzed and concatenated. It's the string concatenation that detracts from
htmltmpl's performance: it uses a single output string (out) and simply
appends to it every time a new token is processed. This is fine for smaller
pages, but doing string concatenation 1000s of times is inefficient because
Python strings are immutable, so each "out += something" actually creates a
new string, thus killing performance. A more efficient possibility is to
store all tokens in a list, then join the list using string.join. This way
the string is created only once. I saw performance go up by 50-70% on pages
spending most time in TemplateProcessor.process with no problems until now
(the test suite is passed, too).
Here's the diff:
diff -u htmltmpl-1.22/htmltmpl.py htmltmpl/htmltmpl.py
--- htmltmpl-1.22/htmltmpl.py 2001-12-15 23:11:28.000000000 +0100
+++ htmltmpl/htmltmpl.py 2004-05-05 12:57:25.000000000 +0200
@@ -574,7 +574,7 @@
tokens = template.tokens()
len_tokens = len(tokens)
- out = "" # buffer for processed output
+ out = [] # buffer for processed output
# Recover position at which we ended after processing of last part.
i = self._current_pos
@@ -605,7 +605,7 @@
if DISABLE_OUTPUT not in output_control:
value = str(self.find_value(var, loop_name,
loop_pass,
loop_total, globalp))
- out += self.escape(value, escape)
+ out += [self.escape(value, escape)]
self.DEB("VAR: " + str(var))
elif token == "<TMPL_LOOP":
@@ -731,21 +731,21 @@
# when it was not replaced by the parser.
skip_params = 1
filename = tokens[i + PARAM_NAME]
- out += """
+ out += ["""
<br />
<p>
<strong>HTMLTMPL WARNING:</strong><br />
Cannot include template: <strong>%s</strong>
</p>
<br />
- """ % filename
+ """ % filename]
self.DEB("CANNOT INCLUDE WARNING")
elif token == "<TMPL_GETTEXT":
skip_params = 1
if DISABLE_OUTPUT not in output_control:
text = tokens[i + PARAM_GETTEXT_STRING]
- out += gettext.gettext(text)
+ out += [gettext.gettext(text)]
self.DEB("GETTEXT: " + text)
else:
@@ -756,7 +756,7 @@
# Raw textual template data.
# If output of current block is not disabled, then
# append template data to the output buffer.
- out += token
+ out += [token]
i += 1
# end of the big while loop
@@ -764,7 +764,7 @@
# Check whether all opening statements were closed.
if loop_name: raise TemplateError, "Missing </TMPL_LOOP>."
if output_control: raise TemplateError, "Missing </TMPL_IF> or
</TMPL_UNLESS>"
- return out
+ return "".join(out)
Take care,
Daniel
|