numpy-discussion — Discussion list for all users of Numerical Python

Re: [Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[Travis O. <oli...@ie...>]

Paulo Jose da Silva e Silva wrote:

>Hello,
>
>Travis asked me to benchmark numpy versus matlab in some basic linear
>algebra operations. Here are the resuts for matrices/vectors of
>dimensions 5, 50 and 500:
>  
>

Hi Paulo,

Will you run these again with the latest SVN version of numpy.   I 
couldn't figure out why a copy was being made on transpose (because it 
shouldn't have been).  Then, I dug deep into the PyArray_FromAny code 
and found bad logic in when a copy was needed that was causing an 
inappropriate copy.

I fixed that and now wonder how things will change.  Because presumably, 
the dotblas function should handle the situation now...

-Travis

[Numpy-discussion] Re: [SciPy-dev] numpy/scipy some remarks

[Robert K. <rob...@gm...>]

Fernando Perez wrote:

> Anyway, I won't belabor this point any longer.  I'd just like to hear from 
> others their opinion on this matter, and if a decision is made to go ahead 
> with the overwriting, at least I think the rationale for it should be well 
> justified (and be more than "it's convenient").  The fact that over the last 
> few weeks we've had several surprised questions on this is, to me, an 
> indicator that I'm not the one uncomfortable with this decision.

I haven't followed this discussion in great detail, but I believe the current
situation is this:

1) If you use numpy.dft and numpy.linalg directly, you will always get the numpy
versions no matter what else is installed.

2) If you want to optionally use optimized scipy versions if they are available
and regular numpy versions otherwise, then you use the functions exposed in
numpy.dual. You do so at your own risk.

3) pkgload() exists to support the loading of subpackages. It does not reach
into numpy.dft or numpy.linalg at all. It is not relevant to this issue.

4) There are some places in numpy that use numpy.dual.

I think we can address all of your concerns by changing #4.

-- 
Robert Kern
rob...@gm...

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

[Numpy-discussion] Re: [SciPy-dev] numpy/scipy some remarks

[Travis O. <oli...@ie...>]

Andrew Jaffe wrote:

>I have some related numpy/scipy questions:
>
>  - I recently upgraded to the most recent SVN of numpy, without doing
>likewise for scipy. I found that the numpy.test() calls failed in a
>couple of places -- because *scipy* hadn't been updated with the latest
>dtype updates! (I can't reproduce the errors since I've since updated
>scipy.) I thought the whole point of the numpy/scipy split was to avoid
>'implicit' calls of scipy from numpy, wasn't it?
>  
>
Not entirely.  The issue was namespace clashes between packages.    This 
issue has come up before and is localized to the numpy.dual module.  If 
somebody would like to re-write the module to contain a register 
function that scipy hooks into to register new functions for the ones 
listed there,  then that would be a better solution.  But, I haven't had 
time to do it.

-Travis

Re: [Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[Paulo J. S. S. <pjs...@im...>]

Em Qua, 2006-01-18 às 15:38 +0000, Matthew Brett escreveu:

> 
> This is really excellent, thanks.  Is there any chance we can make
> these and other benchmarks part of the pre-release testing?  Apart
> from testing for bottlenecks, if we could show that we were in the
> ballpark of matlab for speed for each release, this would be very
> helpful for those us trying to persuade our matlab colleagues to
> switch.
> 

Sure. As soon as I add some less synthetic benchmark it can be even more
interesting. I'll try to make the code easily callable from anyone that
has both numpy and matlab on his own machine. As soon as it is polished
enough I'll make it available to this comunity. It may take some weeks
though.

Best,

Paulo

Re: [Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[Paulo J. S. S. <pjs...@im...>]

Ops...

Forgot to send the benchmark code...

(I know it is ugly and very synthetic, I'll improve it with time).

Re: [Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[Fernando P. <Fer...@co...>]

Matthew Brett wrote:
> Hi,
> 
> 
>>Travis asked me to benchmark numpy versus matlab in some basic linear
>>algebra operations. Here are the resuts for matrices/vectors of
>>dimensions 5, 50 and 500:
> 
> 
> This is really excellent, thanks.  Is there any chance we can make
> these and other benchmarks part of the pre-release testing?  Apart
> from testing for bottlenecks, if we could show that we were in the
> ballpark of matlab for speed for each release, this would be very
> helpful for those us trying to persuade our matlab colleagues to
> switch.

+1

It might not be part of test(1), but at test(10) these tests could be 
automatically run, activating each line as each package is found (or not) in 
the system (Numeric, numarray, matlab).  This way, people who have matlab on 
their box can even get a real-time check of how their fresh-off-svn numpy 
fares against matlab that day.

cheers,

f

Re: [Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[Paulo J. S. S. <pjs...@im...>]

Em Qua, 2006-01-18 às 07:56 -0600, Bruce Southey escreveu:
> Hi,
> Thanks for doing this as it helps determine which approach to take
> when coding problems.  Could you add the Numeric and numarray to these
> benchmarks? If for no other reason to show the advantage of the new
> numpy.

I add a new table with the requested benchmarks below.

As you can see, numpy is always faster then numarray but slower than
Numeric in indexing. Numeric also seems to suffer less from the
transpose phenomenon. 
> 
> I am curious in your code because you get very different results for
> matrix class depending on whether x or y is transposed. Do you first
> transpose the x or y first before the multiplication or is the
> multiplication done in place by just switching the indices? 

I think the transpose problem is that the code makes a copy of the array
before calling blas (to make it fit the blas call). Actually if you
change the code from

dot(transpose(A), b)

to 

M = transpose(A).copy()
dot(M, b)

the time spend in the operation doesn't change.

I am also sending my Python benchmark code attached to this message.
Anyone can used as you want (it requires numpy, Numeric and numarray
installed). 

> Also, for x'*y, is the results for Dimension 50 and Dimension 500
> switched?

No the inner product results have different number of calls for all
dimensions. Hence you can't compare time between dimensions.

Paulo

--- New table ---

Obs: I have fixed a error in my old code that made the outer product for
numpy look bad. The way it was coded it was forcing an extra array copy
before the BLAS call.

Tests  x.T*y   x*y.T     A*x     A*B   A.T*x    half    2in2

Dimension: 5
Array   0.94	0.25	0.22	0.27	1.08	0.93	1.10
Matrix  6.93	1.56	0.64	0.75	1.64	3.08	4.48
NumArr  2.87	0.63	0.62	0.68	8.24	7.22 1	1.37
Numeri  1.15	0.33	0.29	0.36	0.68	0.61	0.72
Matlab	1.88	0.44	0.41	0.35	0.37	1.20	0.98

Dimension: 50
Array   9.64	2.03	0.57	18.74	13.75	4.09	4.29
Matrix 82.98	3.70	1.04	19.87	14.58	6.35	7.91
NumArr 29.72	2.58	0.95	18.40	12.88	8.50	12.90
Numeri 11.97	2.21	0.61	17.71	9.98	1.04	3.22
Matlab 16.98	1.94	1.07	17.86	0.73	1.57	1.77

Dimension: 500
Array   1.22	8.97	2.02	165.03	19.99	3.96	4.25
Matrix 18.13	9.20	2.01	164.90	20.29	4.06	4.35
NumArr  3.16	9.02	2.09	164.83	21.59	4.29	5.72
Numeri  1.46	8.99	2.03	165.01	19.22	3.24	4.50
Matlab	2.09	6.07	2.17	169.45	2.10	2.56	3.06

Re: [Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[Matthew B. <mat...@gm...>]

Hi,

> Travis asked me to benchmark numpy versus matlab in some basic linear
> algebra operations. Here are the resuts for matrices/vectors of
> dimensions 5, 50 and 500:

This is really excellent, thanks.  Is there any chance we can make
these and other benchmarks part of the pre-release testing?  Apart
from testing for bottlenecks, if we could show that we were in the
ballpark of matlab for speed for each release, this would be very
helpful for those us trying to persuade our matlab colleagues to
switch.

Best,

Matthew

[Numpy-discussion] array vs. matrix for matlab-compatible stuff

[Sven S. <sve...@gm...>]

Hi,
I've spent a couple of weeks with scipy/numpy and the old-to-new transition; now that the transition 
is over (?) but some confusion is remaining (on my side) I feel the need to ask a basic question 
about matlab compatibility in terms of matrix (linear algebra) programming.

Take "eye" and "identity" for example; given that "eye" supposedly exists to facilitate transiton 
from matlab to numpy/scipy (correct?), I expected eye to be/return a matrix.

However (as you guys know better than I do), it's not a matrix:

 >>> import numpy
 >>> type(numpy.eye(2))
<type 'numpy.ndarray'>
 >>> type(numpy.identity(2))
<type 'numpy.ndarray'>
 >>> type(numpy.mat(numpy.eye(2)))
<class 'numpy.core.defmatrix.matrix'>

Why is that so? There doesn't seem to be any value added of eye over identity, and the downside is 
inconvenience of having to explicitly convert those arrays to matrices (to use them in 
multiplication, inverting them, avoid unexpected broadcasting behavior if I mistakenly write down a 
non-defined matrix product etc.).

Surely I'm missing something here, but what?

Thanks for your help,
Sven

Re: [Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[Bruce S. <bso...@gm...>]

Hi,
Thanks for doing this as it helps determine which approach to take when
coding problems.  Could you add the Numeric and numarray to these
benchmarks? If for no other reason to show the advantage of the new numpy.

I am curious in your code because you get very different results for matrix
class depending on whether x or y is transposed. Do you first transpose the
x or y first before the multiplication or is the multiplication done in
place by just switching the indices?

Also, for x'*y, is the results for Dimension 50 and Dimension 500 switched?

Thanks
Bruce

On 1/18/06, Paulo Jose da Silva e Silva <pjs...@im...> wrote:
>
> Hello,
>
> Travis asked me to benchmark numpy versus matlab in some basic linear
> algebra operations. Here are the resuts for matrices/vectors of
> dimensions 5, 50 and 500:
>
> Operation       x'*y    x*y'    A*x     A*B     A'*x    Half    2in2
>
> Dimension 5
> Array           0.94    0.7     0.22    0.28    1.12    0.98    1.1
> Matrix          7.06    1.57    0.66    0.79    1.6     3.11    4.56
> Matlab          1.88    0.44    0.41    0.35    0.37    1.2     0.98
>
> Dimension 50
> Array           9.74    3.09    0.56    18.12   13.93   4.2     4.33
> Matrix          81.99   3.81    1.04    19.13   14.58   6.3     7.88
> Matlab          16.98   1.94    1.07    17.86   0.73    1.57    1.77
>
> Dimension 500
> Array           1.2     8.97    2.03    166.59  20.34   3.99    4.31
> Matrix          17.95   9.09    2.07    166.62  20.67   4.11    4.45
> Matlab          2.09    6.07    2.17    169.45  2.1     2.56    3.06
>
> Obs: The operation Half is actually A*x using only the lower half of the
> matrix and vector. The operation 2in2 is A*x using only the even
> indexes.
>
> Of course there are many repetitions of the same operation: 100000 for
> dim 5 and 50 and 1000 for dim 500. The inner product is number of
> repetitions is multiplied by dimension (it is very fast).
>
> The software is
>
> numpy svn version 1926
> Matlab 6.5.0.180913a Release 13 (Jun 18 2002)
>
> Both softwares are using the *same* BLAS and LAPACK (ATLAS for sse).
>
> As you can see, numpy array looks very competitive. The matrix class in
> numpy has too much overhead for small dimension though. This overhead is
> very small for medium size arrays. Looking at the results above
> (specially the small dimensions ones, for higher dimensions the main
> computations are being performed by the same BLAS) I believe we can say:
>
> 1) Numpy array is faster on usual operations but outerproduct (I believe
> the reason is that the dot function uses the regular matrix
> multiplication to compute outer-products, instead of using a special
> function. This can "easily" changes). In particular numpy was faster in
> matrix times vector operations, which is the most usual in numerical
> linear algebra.
>
> 2) Any operation that involves transpose suffers a very big penalty in
> numpy. Compare A'*x and A*x, it is 10 times slower. In contrast Matlab
> deals with transpose quite well. Travis is already aware of this and it
> can be probably solved.
>
> 3) When using subarrays, numpy is a slower. The difference seems
> acceptable. Travis, can this be improved?
>
> Best,
>
> Paulo
>
> Obs: Latter on (in a couple of days) I may present less synthetic
> benchmarks (a QR factorization and a Modified Cholesky).
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: Splunk Inc. Do you grep through log
> files
> for problems?  Stop!  Download the new AJAX search engine that makes
> searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D103432&bid=3D230486&dat=
=3D121642
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>

[Numpy-discussion] Numpy x Matlab: some synthetic benchmarks

[Paulo J. da S. e S. <pjs...@im...>]

Hello,

Travis asked me to benchmark numpy versus matlab in some basic linear
algebra operations. Here are the resuts for matrices/vectors of
dimensions 5, 50 and 500:

Operation	x'*y	x*y'	A*x	A*B	A'*x	Half	2in2

Dimension 5							
Array		0.94	0.7	0.22	0.28	1.12	0.98	1.1
Matrix		7.06	1.57	0.66	0.79	1.6	3.11	4.56
Matlab		1.88	0.44	0.41	0.35	0.37	1.2	0.98

Dimension 50							
Array		9.74	3.09	0.56	18.12	13.93	4.2	4.33
Matrix 		81.99	3.81	1.04	19.13	14.58	6.3	7.88
Matlab		16.98	1.94	1.07	17.86	0.73	1.57	1.77

Dimension 500							
Array		1.2	8.97	2.03	166.59	20.34	3.99	4.31
Matrix		17.95	9.09	2.07	166.62	20.67	4.11	4.45
Matlab		2.09	6.07	2.17	169.45	2.1	2.56	3.06

Obs: The operation Half is actually A*x using only the lower half of the
matrix and vector. The operation 2in2 is A*x using only the even
indexes.

Of course there are many repetitions of the same operation: 100000 for
dim 5 and 50 and 1000 for dim 500. The inner product is number of
repetitions is multiplied by dimension (it is very fast).

The software is

numpy svn version 1926
Matlab 6.5.0.180913a Release 13 (Jun 18 2002)

Both softwares are using the *same* BLAS and LAPACK (ATLAS for sse).

As you can see, numpy array looks very competitive. The matrix class in
numpy has too much overhead for small dimension though. This overhead is
very small for medium size arrays. Looking at the results above
(specially the small dimensions ones, for higher dimensions the main
computations are being performed by the same BLAS) I believe we can say:

1) Numpy array is faster on usual operations but outerproduct (I believe
the reason is that the dot function uses the regular matrix
multiplication to compute outer-products, instead of using a special
function. This can "easily" changes). In particular numpy was faster in
matrix times vector operations, which is the most usual in numerical
linear algebra.

2) Any operation that involves transpose suffers a very big penalty in
numpy. Compare A'*x and A*x, it is 10 times slower. In contrast Matlab
deals with transpose quite well. Travis is already aware of this and it
can be probably solved. 

3) When using subarrays, numpy is a slower. The difference seems
acceptable. Travis, can this be improved?

Best,

Paulo

Obs: Latter on (in a couple of days) I may present less synthetic
benchmarks (a QR factorization and a Modified Cholesky).

[Numpy-discussion] Re: Re: Maximum available dimensions in numpy

[Szymon J. <sj...@cs...>]

It's in https://developer.berlios.de/projects/pbnt/.

Cheers,

Szymon

> On Tue, 17 Jan 2006, Szymon Jaroszewicz apparently wrote: 
> > see for example Python Bayesian Network toolkit 
> > http://elliotpbnt.blogspot.com/ funded by Google summer of 
> > code. 
> 
> Where is the code?
> 
> Cheers,
> Alan Isaac
> 
-- 
Szymon Jaroszewicz <sj...@cs...>

Re: [Numpy-discussion] tracing bdist_rpm

[Pearu P. <pe...@sc...>]

On Tue, 17 Jan 2006, Stephen Walton wrote:

> I am still trying, fitfully, to figure out why numpy and scipy setup.py do 
> not respect an alternative fortran compiler with config_fc when bdist_rpm is 
> used as the build command, but do if the normal build command is used.  Is 
> there any easy way to get a trace of what Python is actually being called as 
> a result of "python setup.py config_fc --fcompiler=absoft bdist_rpm"?

This is not scipy/numpy setup.py specific issue. bdist_rpm calls
`python build` and ignores all other commands in command line. See
the code in std distutils/commands/bdist_rpm.py around lines #411, 
def_build variable.. To resolve this issue, 
numpy.distutils.command.bdist_rpm could add a wrapper around 
_make_spec_file that fixes bdist_rpm build command.

Pearu

Re: [Numpy-discussion] arrayobject.h (and friends) install location

[Alexander B. <ale...@gm...>]

On 1/17/06, Travis Oliphant <oli...@ie...> wrote:
> Andrew Straw wrote:
> ...
> This is the important point in this discussion.   There was a reason it
> was changed.  I remember opposing the change, originally.  However, it
> solved a real problem and only required people to give up their
> arbitrary notions of where include files *should* be.  I don't see this
> as a real issue since you can get where the include files are very easily=
.
>
I don't have any "arbitrary notions of where include files *should*
be" and I have no opposition to the current location.  The only thing
I dislike is that the needs of a particular site situation were
addressed by misdiscribing the package structure to distutils rather
than by adding a default site.cfg file to the distribution that users
with different needs can easily edit.

Another concern is that if numpy will ever be considered for inclusion
in the standard "batteries included" python distribution, the same
issue will rise again.

-- sasha

Re: [Numpy-discussion] arrayobject.h (and friends) install location

[Travis O. <oli...@ie...>]

Andrew Straw wrote:

>The main advantage of the current situation for me is that it's easy to
>use with .eggs (for the reasons given). The symlink idea won't break
>this, but the "let's make it like it was" solution would break it (which
>was why it was changed). Can anyone think of a solution that would
>please both the old-way-preferers and the .egg users?
>  
>
This is the important point in this discussion.   There was a reason it 
was changed.  I remember opposing the change, originally.  However, it 
solved a real problem and only required people to give up their 
arbitrary notions of where include files *should* be.  I don't see this 
as a real issue since you can get where the include files are very easily.

But more than that, I don't see how we can change the current behavior 
until a solution is found that will also work for the people who forged 
the current solution in the first place.

With that, however, I think it would be helpful for one of those people 
who use .eggs to clarify why it solves their problem to have the include 
files in their current location.

-Travis

Re: [Numpy-discussion] Interesting timing results

[Sasha <nd...@ma...>]

On 1/17/06, Travis Oliphant <oli...@ie...> wrote:
> Anybody wanting to help out.  This is a good place.  One issue is how to
> handle scalar math division-by-zero, and overflow errors.  Ideally these
> should be handled the same way that ufuncs do.  But, this will
> necessarily cause some slow-down to look up the local and-or
> module-level attribute that can alter the behavior.
>
Let me try to fill in the functions that take and return bools.  Since
this is the only type with a finite domain, it deserves to be handled
separately. Once the two values are interned, the ops can be
implemented as real fast table lookups.

-- sasha

Re: [Numpy-discussion] The Low-down on numpy/scipy?

[Bill B. <wb...@gm...>]

I see the numeric.scipy.org page has been updated.  Looks much less
confusing now.  Great.

One point that I think might still be slightly confusing is that scipy_core
was never intending to replace functionality in scipy at all.  Is that
right?  I wasn't sure for a while there whether "scipy_core is now NumPy"
meant that SciPy functionality had been subsumed into NumPy.

--bb

On 1/17/06, Bill Baxter <wb...@gm...> wrote:
>
> Ok.  Thanks for trying to clear this up.
>
> Top hit on a google for numpy is here: http://numeric.scipy.org/
>
> In the text it says that "the new scipy used to be called scipy core but
> not anymore".  However, right below that in big bold letters it says "New
> Features of SciPy Core".  Then the page goes on to consistently refer to =
it
> as scipy core, despite that apparently no longer being the name.  So it's
> easy to miss that the name has changed.
>
> Also the only file download link on that page is for the sourceforge site=
 http://sourceforge.net/project/showfiles.php?group_id=3D1369&package_id=3D=
175103
>
> Which has downloads for NumPy, but also has a prominent news item
> announcing the release of scipy_core.  The presence of the news release
> there led me to belive that the download for scipy_core would also be
> located there.
>
> So it would be helpful if the numeric.scipy.org page had a link for
> downloading scipy.  The links to the scipy_core presentations just make i=
t
> seem all the more like you should be able to download scipy_core from the=
re.
>
>
> This page: http://new.scipy.org/Wiki/Download could perhaps mention that
> the SciPy they are offering is the package formerly known as scipy_core, =
to
> additionally eliminate confusion.
>
> There was another page I was able to get to previously, perhaps
> www.scipy.org, but it's not responding now.  Anyway, there was one scipy
> page that said basically "information on these pages may be out of date,
> there's something new afoot" but I don't seem to recall any handy link
> taking me directly to the proper downloads.  Or maybe I was still thinkin=
g I
> was looking for something called scipy_core, rather than scipy.
>
> Hope that information can help you make the web pages clearer.
>
> Thanks for your help.
>
> --Bill
>
>
> On 1/17/06, Sasha < nd...@ma...> wrote:
> >
> > On 1/16/06, Travis Oliphant < oli...@ie...> wrote:
> > > For scipy,  go to http://new.scipy.org ...
> >
> > That should be http://new.scipy.org/Wiki
> >
> > -- sasha
> >
>
>
>

Re: [Numpy-discussion] arrayobject.h (and friends) install location

[Christopher B. <Chr...@no...>]

> On Jan 17, 2006, at 9:08 AM, kon...@la... wrote:
> 
>> What good is that symlink if no one can rely on its presence?

That way,  on OS-X (or whatever) when someone has a problem compiling an 
extension, all they have to do is get their sysadmin to put in the 
symlink, rather than re-install the whole numpy package, or have the 
numpy install fail in the first place. A small improvemnet, I suppose.

As a Linux+OS-X user, I've run into a number of these Apple "Think 
Different" issues, but I don't think creating an ugly, non-standard 
system to accommodate the fact that Apple like to do things strangely is 
the right solution. Apple doesn't maintain their python distribution 
well, or really play well with the python community, so many of us end 
up installing a new Python anyway.

Maybe we could even get Apple to fix this. As someone pointed out it 
doesn't really make any sense to let regular users write to 
site-packages, but not to include.

+1 the standard include location.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

[Numpy-discussion] tracing bdist_rpm

[Stephen W. <ste...@cs...>]

I am still trying, fitfully, to figure out why numpy and scipy setup.py 
do not respect an alternative fortran compiler with config_fc when 
bdist_rpm is used as the build command, but do if the normal build 
command is used.  Is there any easy way to get a trace of what Python is 
actually being called as a result of "python setup.py config_fc 
--fcompiler=absoft bdist_rpm"?

Re: [Numpy-discussion] Maximum available dimensions in numpy

[Travis O. <oli...@ie...>]

Tim Hochberg wrote:

>
> I don't see anything wrong with making the maximum dimension size a 
> compile time option though. However, since in the common case the 
> extra dimensions are unlikely to affect performance in any meaningful, 
> I'd recomend that the maximum number of default dimensions stay large 
> by default.  Thus people who need to conserve bytes, which I'd 
> consider the rare case, have the option of reducing the max dimensions 
> while the arrays behave in an unsuprising manner when compiled in the 
> normal manner.
>
> If someone has the extra time, it would be interesting to see some 
> data about how always mallocing the extra dimensions, so there was no 
> maximum dimensions limit,  affects performance. I'd also be interested 
> in seeing cases where the extra dimensions actually affect performance 
> before doing to stuff complicate^H^H^H^H^H^H^H^H fix things.


Right now, the array has dimensions and strides arrays malloced as 
necessary.  So, as far as the array-object itself is concerned there is 
no upper limit.

The dimension limit only comes from the fact that for ease in coding, 
static arrays for dimensions and/or strides are used in quite a few 
places in the code.  If these were all converted to use dynamic memory 
from the heap, then the dimension limit would go away. 

If anybody wants to try and do this, they are certainly welcome...

-Travis

Re: [Numpy-discussion] Interesting timing results

[Travis O. <oli...@ie...>]

Sasha wrote:

>>python -m timeit -s "x=bool(0)" "x or x"
>>    
>>
>10000000 loops, best of 3: 0.111 usec per loop
>  
>
>>python -m timeit -s "from numpy import bool_, logical_or as or_; x=bool_(0)" "or_(x, x)"
>>    
>>
>100000 loops, best of 3: 3.25 usec per loop
>
>Numpy is 32x slower than python -- not very good.
>  
>
This is a known fact.  Currently all array scalar math goes through the 
entire ufunc machinery.  This is clearly sub-optimal.   It is the reason 
for the scalarmath module that I've started in the src directory.   
Eventually, we should be able to have scalar math as fast as Python 
scalars.

Anybody wanting to help out.  This is a good place.  One issue is how to 
handle scalar math division-by-zero, and overflow errors.  Ideally these 
should be handled the same way that ufuncs do.  But, this will 
necessarily cause some slow-down to look up the local and-or 
module-level attribute that can alter the behavior. 

-Travis

>Interestingly,
>
>  
>
>>python -m timeit -s "from numpy import bool_; x=bool_(0)" "x or x"
>>    
>>
>1000000 loops, best of 3: 0.388 usec per loop
>  
>

Again, not too surprising given that logical_or goes through the ufunc 
machinery...

-Travis

Re: [Numpy-discussion] Re: Maximum available dimensions in numpy

[Alan G I. <ai...@am...>]

On Tue, 17 Jan 2006, Szymon Jaroszewicz apparently wrote: 
> see for example Python Bayesian Network toolkit 
> http://elliotpbnt.blogspot.com/ funded by Google summer of 
> code. 

Where is the code?

Cheers,
Alan Isaac

[Numpy-discussion] Re: [Fwd: Re: _byteorder question]

[Travis O. <oli...@ie...>]

Christopher Hanley wrote:

> Hi Travis,
>
> This message is completely unrelated to the fromfile discussion.  I 
> have forwarded your previous message to Todd for clarification of our 
> needs.
>
> I have a question regarding byteorder in numpy.  In pyfits, I have a 
> need to change the "sense" of a byteorder flag on an array without 
> actually performing the byteswap operation.  I cannot find a mechanism 
> in numpy to actually do this.  Does one exist?
>
Yes,  you want to change the array data-type object (which holds 
byte-order information).  Previously NOTSWAPPED was a flag on the array 
itself which could have been toggled similarly to Numarray.  Now, 
however, byte-order information is a property of the data-type object 
itself.

On SVN version of NumPy (where the data-type object is now a data-type 
descriptor --- instead of the typeobject of the array scalar), you use 
either

a.dtype = a.dtype.newbyteorder() 

# swaps byteorder (you can also specify 'big', 'little', or 'native')

or if you want a new view with the new byteorder then,

b = a.view(a.dtype.newbyteorder())


Notice:

a = arange(5)
b = a.view(a.dtype.newbyteorder('swap'))
print a
print b
print a.tostring() == b.tostring()

This results in
[0 1 2 3 4]
[       0 16777216 33554432 50331648 67108864]
True

Notice that the actual data is exactly the same...

For fun, try adding 1 to the b array and notice how the a array is 
changed :-)

-Travis

Re: [Numpy-discussion] numpy.fromfile method modification?

[Christopher H. <ch...@st...>]

Travis Oliphant wrote:
> What do you think about the sizing paramter of Numarray?   Is that 
> important?

The sizing parameter is apparently a recently contributed patch to 
numarray.  We currently have no dependencies on it.  I do not know how 
widely it is used outside of STScI.

Chris

[Numpy-discussion] Interesting timing results

[Sasha <nd...@ma...>]

> python -m timeit -s "x=3Dbool(0)" "x or x"
10000000 loops, best of 3: 0.111 usec per loop
> python -m timeit -s "from numpy import bool_, logical_or as or_; x=3Dbool=
_(0)" "or_(x, x)"
100000 loops, best of 3: 3.25 usec per loop

Numpy is 32x slower than python -- not very good.

Interestingly,

> python -m timeit -s "from numpy import bool_; x=3Dbool_(0)" "x or x"
1000000 loops, best of 3: 0.388 usec per loop

-- sasha