Thread: [Numpy-discussion] How fast are small arrays currently?

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Has anyone recently benchmarked the speed of numarray vs. Numeric?

Why are numarrays so slow to create?

On Tue, Jan 20, 2004 at 02:42:03PM -0500, Edward C. Jones wrote:
> Has anyone recently benchmarked the speed of numarray vs. Numeric?

Just what I was doing :-)

Check out http://arbutus.mcmaster.ca/dmc/numpy/ for a graph comparing
the two.

Basically, I get on my machine (a 1.3 GHz Athlon running Linux), for an
array of size N (of Float), the time to do a+a is

Numeric:  3.7940e-6 + 2.2556e-8 * N seconds
numarray: 3.7062e-5 + 5.8497e-9 * N

For sin(a), 
Numeric:  1.7824e-6 + 1.1341e-7 * N
numarray: 2.8994e-5 + 9.8985e-8 * N

So the slowness of numarray vs. Numeric for small arrays is because of
an overhead of 3.7e-5 s for numarray, as opposed to 3.8e-6 s for
Numeric. Otherwise, numarray is 4 times faster for large arrays
for addition (and multiplication, which I've also checked).

The crossover is at arrays of about 2000 elements.

If this overhead could be reduced by a factor of 3 or 4, I'd be much
happier with using numarray for small arrays. But for now, it's not
good enough.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

David M. Cooke writes:

> Just what I was doing :-)
> 
> Check out http://arbutus.mcmaster.ca/dmc/numpy/ for a graph comparing
> the two.
> 
> Basically, I get on my machine (a 1.3 GHz Athlon running Linux), for an
> array of size N (of Float), the time to do a+a is
> 
> Numeric:  3.7940e-6 + 2.2556e-8 * N seconds
> numarray: 3.7062e-5 + 5.8497e-9 * N
> 
> For sin(a), 
> Numeric:  1.7824e-6 + 1.1341e-7 * N
> numarray: 2.8994e-5 + 9.8985e-8 * N
> 
> So the slowness of numarray vs. Numeric for small arrays is because of
> an overhead of 3.7e-5 s for numarray, as opposed to 3.8e-6 s for
> Numeric. Otherwise, numarray is 4 times faster for large arrays
> for addition (and multiplication, which I've also checked).
> 
> The crossover is at arrays of about 2000 elements.
> 
> If this overhead could be reduced by a factor of 3 or 4, I'd be much
> happier with using numarray for small arrays. But for now, it's not
> good enough.
> 
How many times do you do the operation for each size? Because of
caching, the first result may be much slower than the rest.
If you didn't could you try computing it by discarding the first
numarray time (or start timing after doing the first iteration)?

Thanks, Perry

On Tue, Jan 20, 2004 at 03:38:34PM -0500, Perry Greenfield wrote:
> David M. Cooke writes:
> 
> > Just what I was doing :-)
> > 
> > Check out http://arbutus.mcmaster.ca/dmc/numpy/ for a graph comparing
> > the two.
> > 
> > Basically, I get on my machine (a 1.3 GHz Athlon running Linux), for an
> > array of size N (of Float), the time to do a+a is
> > 
> > Numeric:  3.7940e-6 + 2.2556e-8 * N seconds
> > numarray: 3.7062e-5 + 5.8497e-9 * N
> > 
> > For sin(a), 
> > Numeric:  1.7824e-6 + 1.1341e-7 * N
> > numarray: 2.8994e-5 + 9.8985e-8 * N
...
> How many times do you do the operation for each size? Because of
> caching, the first result may be much slower than the rest.
> If you didn't could you try computing it by discarding the first
> numarray time (or start timing after doing the first iteration)?

10000 times per size. I'm re-running it like you suggested, but the
difference is small (the new version is up on the above page). For
numarray for addition, it's now
3.8771e-5 + 4.9832e-9 * N

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

>
David M. Cooke writes:
>
> 10000 times per size. I'm re-running it like you suggested, but the
> difference is small (the new version is up on the above page). For
> numarray for addition, it's now
> 3.8771e-5 + 4.9832e-9 * N
>

Well, OK we'll have to look into that. That's different by a factor of
3 or so than what I expected. I'll see if I can find what that is due 
to.

Perry

With a cross-over at ~2000 elements, can we safely say that working with 
video, FITS cubes or other similar imagery would be fastest with numarray 
for summing or dividing 2D arrays? (~920K elements)

Ray
http://rjs.org/astro

On Tuesday 20 January 2004 17:54, Ray Schumacher wrote:
> With a cross-over at ~2000 elements, can we safely say that working with
> video, FITS cubes or other similar imagery would be fastest with numarray
> for summing or dividing 2D arrays? (~920K elements)

My benchmark was for 1-D arrays, but checking 2-D shows the crossover is in 
the same region. I'd say for these types of applications you really want to 
use numarray.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke                      http://arbutus.physics.mcmaster.ca/dmc/
|co...@ph...

On Tuesday, January 20, 2004, at 05:54 PM, Ray Schumacher wrote:

> With a cross-over at ~2000 elements, can we safely say that working 
> with video, FITS cubes or other similar imagery would be fastest with 
> numarray for summing or dividing 2D arrays? (~920K elements)
>
As long as you treat the array as a whole, I'd say that usually 
numarray would be better
suited. That doesn't mean you won't find some instances where it is 
slower for certain
operations. (When you do, let us know).

Perry Greenfield

> -----Original Message-----
> From: num...@li...
> [mailto:num...@li...]On Behalf Of Edward
> C. Jones
> Sent: Tuesday, January 20, 2004 2:42 PM
> To: num...@li...
> Subject: [Numpy-discussion] How fast are small arrays currently?
> 
> 
> Has anyone recently benchmarked the speed of numarray vs. Numeric?
>
We presented some benchmarks at scipy 2003. It depends on many factors
and what functions or operations are being performed so it is hard 
to generalize (one reason I ask for specific cases that need improvement).

But to take ufuncs as examples: the speed for 1 element arrays (about
as small as they get) has:
                                          v0.4 v0.5
Int32 + Int32                               65  3.7
Int32 + Int32 discontiguous                104  7.3
Int32 + Float64                             95  4.9
add.reduce(Int32) NxN swapaxes             111  3.6
add.reduce(Int32, -1) NxN                   98  3.2

What is shown is the (time for numarray operation)/(time for Numeric),
for v0.4 and v0.5. Note that with v0.5, these are typically 3 to 4 times
slower for small arrays with a couple cases some what worse (a factor
of 4.9 and 7.3). Speeds for v0.4 are substantially slower (orders of
magnitude). Note that the speedup is obtained through caching certain
information. The first time you perform a certain operation (say an
Int32/Int16 add), it will be slow. When repeated it will be closer to
that shown benchmark. If you are only going to do one operation on a 
small array, speed presumably doesn't matter much. It is only when you
plan to iterate over many small arrays would it usually be an issue.

Other functions may be much worse (or better). If people let us know 
which things are too slow we can put that on our to do list.

Is a factor of 3 or 4 times slower a killer? What about a factor of
2?

> Why are numarrays so slow to create?
> 
I'll leave it to Todd to give the details of that.

> > Why are numarrays so slow to create?
> > 
There are several portable ways to create numarrays (array(), arange(),
zeros(), ones()) and I'm not really sure which one to address, so I
poked around some.  

I discovered that numarray-0.8 has a problem with array() which causes
very poor performance (~30x slower than Numeric) for arrays created from
a sequence.  The problem is with a private Python function,
_all_arrays(), that scans the sequence to see if it consists only of
arrays;  _all_arrays() works badly for the ordinary case of a sequence
of numbers.  This is fixed now in CVS.

Beyond this flaw in array(),  it's a mixed bag,  with numarray tending
to do well with large arrays and certain use cases, and Numeric doing
well with small arrays and other use cases.

Todd

> I'll leave it to Todd to give the details of that.
> 
> 
> 
> 
> -------------------------------------------------------
> The SF.Net email is sponsored by EclipseCon 2004
> Premiere Conference on Open Tools Development and Integration
> See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
> http://www.eclipsecon.org/osdn
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller 			
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21030
(410) 338 - 4576

Thread: [Numpy-discussion] How fast are small arrays currently?

A package for scientific computing with Python

numpy-discussion