numpy-discussion

[Numpy-discussion] Speed degression?

[Steffen L. <ste...@gm...>]

Hi all,

i made some speed tests using the sin-function and the %-operation to compare 
Numeric, numpy 0.9.8 and numpy 0.9.9.2732. 
As result the latest numpy version seems to be very slow in comparison to the 
two other candidates.

Results (in usec per loop):
                                     sin-array            mod-array  
                                    
 Numeric                                134                   18         
  
 numpy 0.9.8                             97                   55

 numpy 0.9.9.2732                       204                  316
   
 numpy 0.9.8 + math                      38

 numpy 0.9.9.2732 + math                161
  
 Numeric + math                          23


The used scripts can be found at the end. 

Can anyone verify my results and explain the observed speed degression?

Thanks,
Steffen


sin-scripts:

/usr/lib/python2.3/timeit.py -s "from Numeric import sin,zeros,arange; 
x=zeros(10, 'd'); x[0]=0.1" "for i in arange(9): x[i+1]=sin(x[i])"
 
/usr/lib/python2.3/timeit.py -s "from numpy import sin,zeros,arange; 
x=zeros(10, 'd'); x[0]=0.1" "for i in arange(9): x[i+1]=sin(x[i])"
 
/usr/lib/python2.3/timeit.py -s "from math import sin; from numpy import 
zeros,arange; x=zeros(10, 'd');x[0]=0.1" "for i in arange(9): 
x[i+1]=sin(x[i])"

/usr/lib/python2.3/timeit.py -s "from math import sin; from Numeric import 
zeros,arange; x=zeros(10, 'd'); x[0]=0.1" "for i in arange(9): 
x[i+1]=sin(x[i]
)"


%-scripts

/usr/lib/python2.3/timeit.py -s "from Numeric import zeros,arange; x=zeros(10, 
'd'); x[0]=0.1" "for i in arange(9): x[i+1]=(x[i]+1.1)%(1.0)"
 
/usr/lib/python2.3/timeit.py -s "from numpy import zeros,arange; x=zeros(10, 
'd'); x[0]=0.1" "for i in arange(9): x[i+1]=(x[i]+1.1)%(1.0)"

Re: [Numpy-discussion] Speed degression?

[Travis O. <oli...@ie...>]

Steffen Loeck wrote:
> Hi all,
>
> i made some speed tests using the sin-function and the %-operation to compare 
> Numeric, numpy 0.9.8 and numpy 0.9.9.2732. 
> As result the latest numpy version seems to be very slow in comparison to the 
> two other candidates.
>
> Results (in usec per loop):
>                                      sin-array            mod-array  
>                                     
>  Numeric                                134                   18         
>   
>  numpy 0.9.8                             97                   55
>
>  numpy 0.9.9.2732                       204                  316
>    
>  numpy 0.9.8 + math                      38
>
>  numpy 0.9.9.2732 + math                161
>   
>  Numeric + math                          23
>
>
> The used scripts can be found at the end. 
>
> Can anyone verify my results and explain the observed speed degression?
>   

I can't verify the results, but it's possible that the indexing code for 
a single integer is a bit slower.  

However, you forgot to test the "use Python lists" approach which is the 
fastest of all on this code. 

python -m timeit -s "x = [0.1]*10;" "for i in xrange(9): 
x[i+1]=(x[i]+1.1)%(1.0)"


Using Numeric/NumPy is always slower than lists if you are doing 
element-by-element processing.   NumPy is slower than Numeric for such 
processing.  But, Numeric was always slower than pure Python too for 
such processing.  So, the moral of the story is:

"don't compute like that with NumPy (or Numeric)"

With that said.   I can add back the 1-d integer check in the indexing 
code to see if that speeds things up...

-Travis

Re: [Numpy-discussion] Speed degression?

[Arnd B. <arn...@we...>]

Hi Travis,

On Thu, 6 Jul 2006, Travis Oliphant wrote:

> Steffen Loeck wrote:
> > Hi all,
> >
> > i made some speed tests using the sin-function and the %-operation to compare
> > Numeric, numpy 0.9.8 and numpy 0.9.9.2732.
> > As result the latest numpy version seems to be very slow in comparison to the
> > two other candidates.
> >
> > Results (in usec per loop):
> >                                      sin-array            mod-array
> >
> >  Numeric                                134                   18
> >  numpy 0.9.8                             97                   55
> >  numpy 0.9.9.2732                       204                  316
> >  numpy 0.9.8 + math                      38
> >  numpy 0.9.9.2732 + math                161
> >  Numeric + math                          23
> >
> > Can anyone verify my results and explain the observed speed degression?
> >
>
> I can't verify the results, but it's possible that the indexing code for
> a single integer is a bit slower.

On the 64Bit machine I get

Results (in usec per loop):
                                     sin-array            mod-array
 Numeric                                34                  7
 numpy 0.9.9.2746                       56                 92
 numpy 0.9.9.2746 + math                42
 Numeric + math                          8

Though this does not do the comparison with 0.9.8,
it can roughly reproduce the ratios above.

In particular the mod seems not just a "bit slower", but more than
a factor 5. Also the " + math" variants for the sin
suffered by a factor of 4...

> However, you forgot to test the "use Python lists" approach which is the
> fastest of all on this code.
>
> python -m timeit -s "x = [0.1]*10;" "for i in xrange(9):
> x[i+1]=(x[i]+1.1)%(1.0)"
>
>
> Using Numeric/NumPy is always slower than lists if you are doing
> element-by-element processing.   NumPy is slower than Numeric for such
> processing.  But, Numeric was always slower than pure Python too for
> such processing.  So, the moral of the story is:
>
> "don't compute like that with NumPy (or Numeric)"
>

Well, the example is a benchmark and not the real code ...
To me it seems that it clearly points out some significant slow-down.

> With that said.   I can add back the 1-d integer check in the indexing
> code to see if that speeds things up...

That would be great - I will re-run the tests on the Opteron then.

Many thanks,

Arnd

Re: [Numpy-discussion] Speed degression?

[Travis O. <oli...@ie...>]

Arnd Baecker wrote:
> Hi
>>> i made some speed tests using the sin-function and the %-operation to compare
>>> Numeric, numpy 0.9.8 and numpy 0.9.9.2732.
>>> As result the latest numpy version seems to be very slow in comparison to the
>>> two other candidates.
>>>       

Arrgggh.

It looks like a pre-mature return statement was causing scalarmath not 
to load and therefore none of it's speed ups are present  until this SVN.

Thanks for the speed test...

-Travis

Re: [Numpy-discussion] Speed degression?

[Arnd B. <arn...@we...>]

On Thu, 6 Jul 2006, Travis Oliphant wrote:

> Arnd Baecker wrote:
> > Hi
> >>> i made some speed tests using the sin-function and the %-operation to compare
> >>> Numeric, numpy 0.9.8 and numpy 0.9.9.2732.
> >>> As result the latest numpy version seems to be very slow in comparison to the
> >>> two other candidates.
> >>>
>
> Arrgggh.
>
> It looks like a pre-mature return statement was causing scalarmath not
> to load and therefore none of it's speed ups are present  until this SVN.
>
> Thanks for the speed test...

Well, the thanks for this go to Steffen (who wrote the text above and the
test, not me ;-).

Concerning the speed the thanks go to you:

Results (in usec per loop):
                                     sin-array            mod-array
 Numeric                                34                  7
 numpy 0.9.9.2732                       56                 92
 numpy 0.9.9.2732 + math                42
 Numeric + math                          8

*and now*

Results (in usec per loop):
                                     sin-array            mod-array
 Numeric                                33                  7
 numpy 0.9.9.2749                       30                 17.6
 numpy 0.9.9.2749 + math                13
 Numeric + math                          8


Bingo! Only numpy 0.9.9.2749 + math  is slower than
Numeric+math by a factor 1.6 and mod-array is 2.5 times slower
than numpy.
It would be nice if you could have a look at the module operation,
if possible ...

Many thanks,

Arnd

Re: [Numpy-discussion] Speed degression?

[Travis O. <oli...@ee...>]

Arnd Baecker wrote:

>Bingo! Only numpy 0.9.9.2749 + math  is slower than
>Numeric+math by a factor 1.6 and mod-array is 2.5 times slower
>than numpy.
>It would be nice if you could have a look at the module operation,
>if possible ...
>  
>

O.K.  I looked at how Python computed modulo and implemented the same 
thing for both umath (the remainder function) and scalarmath.

The result is a significant speed up for modulo...yeah...

I also placed in hooks so you can replace the scalarmath (for int, 
float, and complex) with  the Python version of math (this works because 
the int, float, and complex scalars are sub-classes of the corresponding 
Python object).  

import numpy.core.scalarmath as ncs

# Replace "scalarmath"  with standard Python arithmetic.
ncs.use_pythonmath(<int | float | complex>) 

# Replace "scalarmath"  with actual scalarmath code.
ncs.use_scalarmath(<int | float | complex>)

This can test the speed of the implementations.


-Travis

Re: [Numpy-discussion] Speed degression?

[Steffen L. <ste...@gm...>]

T24gVGh1cnNkYXkgMDYgSnVseSAyMDA2LCBUcmF2aXMgT2xpcGhhbnQgd3JvdGU6Cj4gVGhpcyBj
YW4gdGVzdCB0aGUgc3BlZWQgb2YgdGhlIGltcGxlbWVudGF0aW9ucy4KCldpdGggdGhlIGxhdGVz
dCBtb2RpZmljYXRpb25zIGkgZ2V0IHRoZSBmb2xsb3dpbmcKClJlc3VsdHMgKGluIHVzZWMgcGVy
IGxvb3ApOgqgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgc2luLWFycmF5IKAg
oCCgIKAgoCCgbW9kLWFycmF5IKAKCQkJICAgICAgICAgICAgICB1c2Vfc2NhbGFybWF0aC91c2Vf
cHl0aG9ubWF0aKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIAoKoE51bWVyaWMg
oCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoDEzNCCgIKAgoCCgIKAgoCCgIKAgoCAxOCCg
IKAgoCCgIAqgIAqgbnVtcHkgMC45LjggoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoCCgIDk3IKAg
oCCgIKAgoCCgIKAgoCCgIDU1CgqgbnVtcHkgMC45LjkuMjc2NCCgIKAgoCCgIKAgoCCgIKAgoCCg
IKAxMTQvNzggoCCgIKAgoCCgIKAgoCCgNTUvMjEKoCCgCqBudW1weSAwLjkuOCArIG1hdGggoCCg
IKAgoCCgIKAgoCCgIKAgoCCgMzgKCqBudW1weSAwLjkuOS4yNzY0ICsgbWF0aCCgIKAgoCCgIKAg
oCCgICAzOC8yMwqgIAqgTnVtZXJpYyArIG1hdGggoCCgIKAgoCCgIKAgoCCgIKAgoCCgIKAgoDIz
CgoKVGhlIG5ld2VzdCB2ZXJzaW9uIHNlZW1zIHRvIGJlIGZhc3RlciB0aGVuIE51bWVyaWMgaW4g
YWxsIHNpbi1hcnJheSB0ZXN0cy4gCkFsc28gdGhlIHNwZWVkIG9mIG1vZC1hcnJheSBoYXMgaW1w
cm92ZWQgYSBsb3QuCgpNYW55IHRoYW5rcywKU3RlZmZlbgoK

Re: [Numpy-discussion] Speed degression?

[Steffen L. <ste...@gm...>]

> I also placed in hooks so you can replace the scalarmath (for int,
> float, and complex) with  the Python version of math (this works because
> the int, float, and complex scalars are sub-classes of the corresponding
> Python object).

Just for completeness some more tests using pythonmath/scalarmath for int, 
float or both (in usec per loop):

                            sin - array     mod - array    xx

(a) - (no import of numpy.core.scalarmath)

numpy 0.9.9.2800              152              76.5
numpy 0.9.9.2800 + math        50.2 


(b) - (use_pythonmath(xx))

numpy 0.9.9.2800              107              60.4       (int)
numpy 0.9.9.2800 + math        32.7 
 
numpy 0.9.9.2800              148              43         (float)
numpy 0.9.9.2800 + math        50.7 

numpy 0.9.9.2800              109              26.5       (int, float)
numpy 0.9.9.2800 + math        32.4 


(c) - (use_scalarmath(xx))

numpy 0.9.9.2800              149              77.1       (int)
numpy 0.9.9.2800 + math        50.7 

numpy 0.9.9.2800              147              74.3       (float)
numpy 0.9.9.2800 + math        50.7 

numpy 0.9.9.2800              148              73.5       (int, float)
numpy 0.9.9.2800 + math        50.8  


Maybe use_pythonmath(int, float, complex) should be set as default?

Many thanks,
Steffen