[Pyobjc-dev] Slow String Repeat (was: [Python-Dev] PyBuffer* vs. array.array())

[Christian T. <ti...@ti...>]

Christian Tismer wrote:
> Guido van Rossum wrote:
...

>> Correct; then even better:
>>
>>              singlePlane = array.array('B', [0]) * (width*height*3)
>>
>> i.e. do only one sequence repeat rather than three.

Here an addition to my former note.

Doing some simple analysis of this, I found that
it is generally safer *not* to do huge repetitions
of very small objects.
If you always use intermediate steps, you are
creating some slight overhead, but you will never
step into traps like these:

>  >>> if 1:
> ...     t = time.clock()
> ...     for i in xrange(100):
> ...         s = ' ' * 1000 * 1000
> ...     print time.clock()-t
> ...
> 0.674784644417
>  >>> if 1:
> ...     t = time.clock()
> ...     for i in xrange(100):
> ...         s = ' ' * 1000000
> ...     print time.clock()-t
> ...
> 6.28695295072
>  >>>

Analysis:
The central copying code in stringobject.c is the following
tight loop:

	for (i = 0; i < size; i += a->ob_size)
		memcpy(op->ob_sval+i, a->ob_sval, (int) a->ob_size);

For my example, this memcpy is started for every single
of the one million bytes. So the overhead of memcpy,
let is be a function call or a macro, will be executed
a million times.

On the other hand, doing ' ' * 1000 * 1000 only
has to call memcpy 2000 times.

My advice: Do not go from very small to very large in
one big step, but go to reasonable chunks.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:ti...@ti...>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  pager +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/