numpy-discussion — Discussion list for all users of Numerical Python

[Numpy-discussion] Some observations or questions about psyco & pyrex

[Perry G. <pe...@st...>]

Both psyco and pyrex have some great aspects. But I think
it is worth a little reflection on what can and can't be
expected of them. I'm basically ignorant of both; I know
a little about them, but haven't used them. if anything I 
say is wrong, please correct me. I'm going to make some
comments based on inferred characteristics of them that 
could well be wrong.

Psyco is very cool and seems the answer to many dreams. 
But consider the cost. From what I can infer, it obtains
its performance enhancements at least in part by constructing
machine code on the fly from the Python code. In other
words it is performing aspects of running on particular
processors that is usually relegated to C compilers by 
Python.

I'd guess that the price is the far greater difficulty of
maintaining such capability across many processor types.
It also likely increases the complexity of the implementation
of Python, perhaps making it much harder to change and
enhance. Even without it handling things that are needed
for array processing, how likely is it that it will be 
accepted as the standard implementation for Python for 
these reasons alone. 

I also am inclined to believe that adding the support for
array processing to a psyco implementation is a significant
undertaking. There are at least two issues that would have 
to be addressed: handling all the numeric types and exception
handling behavior. Then there are aspects important to us
that include handling byteswapped or non-aligned data.
Having the Python VM handle the efficiency aspects of arrays
simplifies aspects of their implementation as compared
to the current implementations of Numeric and numarray it
doesn't eliminate the need to replicate much of it.
Having to deal with the implemenation for several different
processors is likely to outweigh any savings in the implementation.
But maybe I misjudge.

Pyrex's goals are more realistic I believe. But unless I'm 
mistaken, pyrex cannot be a solution to the problems
that Numeric and numarray solve. Writing a something for
pyrex means committing to certain types. It's great for
writing something that you would have written as a C extension,
but it doesn't really solve the problem of implementing
Ufuncs that can handle many different types of arrays,
and particularly combinations of types. But perhaps I
misunderstand here as well. It certainly would be nice if
it could handle some of the aspects of the Numeric/numarray
API automatically.

So I doubt that either really is a solution for masked arrays
in general.

Perry

Re: [Numpy-discussion] Psyco MA?

[Joachim S. <li...@js...>]

* Tim Churches [2003-02-08 01:25]:
> On Sat, 2003-02-08 at 10:57, Joachim Saul wrote:
> > * Tim Churches [2003-02-08 00:07]:
> > > However, I agree 100% about the potential for leveraging Pyrex in
> > > Numarray. Not just in Numarray, but around it, too. The Numarray team
> > > should open serious talks with Greg Ewing about Numarray-enabling Pyrex.
> >
> > What is it that needs to be "enabled"? Pyrex handles Numeric (see
> > Pyrex FAQ), why should it not handle Numarray? AFAIK Pyrex
> > contains no code to specifically support Numeric, and it should
> > therefore be straightforward to use it with Numarray as well.
>
> Hmmm, maybe re-implementing MA in Pyrex is possible right now. Double
> hmmm....

Please check out the Pyrex doc. It's actually very easy right now,
*if* you can live without "sequence operators" such as slicing,
list comprehensions... but this is going to be supported, again
according to the doc.

Here is a exerpt from an extension module that I have built using
Pyrex and Numeric, following the instructions in the Pyrex-FAQ:

cdef extern int decomp(int, double*, double*, double, double, double)

cdef extern from "Numeric/arrayobject.h":
    struct PyArray_Descr:
        int type_num, elsize
        char type
    ctypedef class PyArrayObject [type PyArray_Type]:
        cdef char *data
        cdef int nd
        cdef int *dimensions,*strides
        cdef PyArray_Descr *descr
    object PyArray_FromDims(int, int*, int)
    void import_array()

def _decomp(PyArrayObject z_arr, PyArrayObject r_arr,
            double p, double vs, double sigma):
    cdef double *z, *r
    cdef int n

    n = z_arr.dimensions[0]
    z, r = <double*> z_arr.data, <double*> r_arr.data
    decomp(n, z, r, p, vs, sigma)

This is rather crude code that doesn't check for the type of the
arrays nor their dimension, but it does what I want right now and
if I find the time I'll certainly make it more general. Those
checks are actually performed in yet another Python layer.

As you can see, the above looks like "strongly typed" Python. From
a C-programmers perspective, if find this is extremely cool. If one
leaves the type out, then the argument can be any Python object.

What I like about Pyrex is that you can mix Python and C calls at
your convenience. For example, I may call (C-like)

    arr = PyArray_FromDims(1, &n, PyArray_DOUBLE)

but could have also used a corresponding Python construct like

    from Numeric import zeros
    arr = zeros(n, 'd')

I expect the latter to be slower (not tested), but one can take
Python code "as is" and "compile" it using Pyrex. This already
increases performance and one can then conveniently replace as
much Python code as needed with the corresponding C functions,
which (presumably) will again speed up the code significantly.
The bottle necks are finally moved to external C files and
treated like a C library.

Cheers,
Joachim

Re: [Numpy-discussion] Psyco MA?

[Chris B. <Chr...@no...>]

Tim Hochberg wrote:
> Psyco seems fairly stable these days. However it's one of those things
> that probably needs to get a larger cabal of users to shake the bugs out
> of it. I still only use it to play around with because all things that I
> need speed from I end up doing in Numeric anyway.

Hmmm. It always just seemed too bleeding edge for me to want to drop it
in inplace of my current Python, but maybe I should try...

> For Psyco at least you don't need a multidimensional type. You can get
> good results with flat array, in particular array.array. The number I
> posted earlier showed comparable performance for Numeric and a
> multidimensional array type written all in python and psycoized.

What about non-contiguous arrays? Also, you pointed out yourself that
you are still looking at a factor of two slowdown, it would be nice to
get rid of that.

> >While the Psyco option is the rosy future of Python, Pyrex is here now,
> >and maybe adopting it to handle NumArrays well would be easier than
> >re-writing a bunch of NumArray in C.
> >
> This sounds like you're conflating two different issues. The first issue
> is that Numarray is relatively slow for small arrays.  Pyrex may indeed
> be an easier way to attack this although I wouldn't know, I've only
> looked at it not tried to use it. However, I think that this is
> something that can and should wait. Once use cases of numarray being
> _too_ slow for small arrays start piling up, then it will be time to
> attack the overhead. Premature optimization is the root of all evil and
> all that.

Quite true. I know I have a lot of use cases where I use a LOT of small
arrays. That doesn't mean that performace is a huge problem, we'll see.

I'm talking about other things as well, however. There are a lot of
functions in the current Numeric that are written in a combination of
Python and C. Mostly they are written using the lower level Numeric
functions. This includes concatenate, chop, etc. etc. While speeding up
any individual one of those won't make much difference, speeding them
all up might. If it were much easier to get C-speed functions like this,
we'd have a higher performance package all around.

I've personally re-written byteswap() and chop(). In this case, not to
get them faster, but to get them to use less memory. It would be great
if we could do them all.

> The second issue is how to deal with code that does not vectorize well.
> Here Pyrex again might help if it were made Numarray aware. However,
> isn't this what scipy.weave already does? Again, I haven't used weave,
> but as I understand it, it's another python-c bridge, but one that's
> more geared toward numerics stuff.

Weave is another project that's on my list to check out, so I don't know
why one would choose one over the other.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

RE: [Numpy-discussion] Psyco MA?

[Paul F D. <pa...@pf...>]

Just to confirm the obvious, I don't know the difference between Psyco and
Pyrex and if I ever did it is Friday night and I've lost it. Any two words
that share two letters look the same to me right now. 



> -----Original Message-----
> From: num...@li... 
> [mailto:num...@li...] On 
> Behalf Of Tim Hochberg
> Sent: Friday, February 07, 2003 3:09 PM
> To: Chris Barker
> Cc: Paul F Dubois; num...@li...
> Subject: Re: [Numpy-discussion] Psyco MA?
> 
> 
> Chris Barker wrote:
> 
> >oops, sorry about the blank message.
> >
> >Paul F Dubois wrote:
> >  
> >
> >>{ CC to GvR just to show why I'm +1 on the if-PEP. I liked this in 
> >>another
> >>    
> >>
> >
> >What the heck is the if-PEP ?
> >  
> >
> 
> Pep 308. It's stirring up a bit of a ruckos on CLP as we speak.
> 
> >>Perhaps knowlegeable persons could comment on the feasibility of 
> >>coding MA (masked arrays) in straight Python and then using 
> Psyco on 
> >>it?
> >>    
> >>
> >
> >Is there confusion between Psyco and Pyrex? Psyco runs regular old 
> >Python bytecode, and individually compiles little pieces of it as 
> >needed into machine code. AS I understand it, this should make loops 
> >where the inner part is a pretty simple operation very fast.
> >
> >However, Psyco is pretty new, and I have no idea how robust 
> and stable, 
> >but certainly not cross platform. As it generates machine code, it 
> >needs to be carefully ported to each hardware platform, and it 
> >currently only works on x86.
> >  
> >
> Psyco seems fairly stable these days. However it's one of 
> those things 
> that probably needs to get a larger cabal of users to shake 
> the bugs out 
> of it. I still only use it to play around with because all 
> things that I 
> need speed from I end up doing in Numeric anyway.
> 
> >Pyrex, on the other hand, is a "Python-like" language that 
> is tranlated 
> >into C, and then the C is compiled. It generates pretty darn 
> platform 
> >independent, so it should be able to be used on all platforms.
> >
> >
> >In regard to your question about MA (and any ther similar 
> project): I 
> >think Psyco has the potential to be the next-generation Python VM, 
> >which will have much higher performance, and therefore 
> greatly reduce 
> >the need to write extensions for the sake of performance. I supsect 
> >that it could do its best with large, multi-dimensional arrays of 
> >numbers if there is a Python native object of such a type. Psycho, 
> >however is not ready for general use on all platforms, so in the 
> >forseeable future, there is a need for other ways to get decent 
> >performance. My suggestion follows:
> >
> >  
> >
> >>It could have been written a lot simpler if performance 
> didn't dictate 
> >>trying to leverage off Numeric. In straight Python one can 
> imagine an 
> >>add, for example, that was roughly:
> >>    for k in 0<= k < len(a.data):
> >>       result.mask[k] = a.mask[k] or b.mask[k]
> >>       result.data[k] = a.data[k] if result.mask[k] else 
> a.data[k] + 
> >>b.data[k]
> >>    
> >>
> >
> >This looks like it could be written in Pyrex. If Pyrex were suitably 
> >NumArray aware, then it could work great.
> >
> >What this boils down to, in both the Pyrex and Psyco 
> options, is that 
> >having a multi-dimensional homogenous numeric data type that is 
> >"Native" Python is a great idea! With Pyrex and/or Psyco, Numeric3 
> >(NumArray2 ?) could be implimented by having only the 
> samallest core in 
> >C, and then rest in Python (or Pyrex)
> >  
> >
> For Psyco at least you don't need a multidimensional type. 
> You can get 
> good results with flat array, in particular array.array. The number I 
> posted earlier showed comparable performance for Numeric and a 
> multidimensional array type written all in python and psycoized.
> 
> And since I suspect  that I'm the mysterious person who's name Paul 
> couldn't remember, let me say I suspect the MA would be faster in 
> psycoized python than what your doing now as long as a.data was an 
> instance of array.array. However, there are at least three problems. 
> Psyco doesn't fully support the floating point type('f') right now 
> (although it does support most of the various  integral types in 
> addition to 'd'). I assume that these masked arrays are 
> multidimensional, so someone would have to build the basic 
> multidimensional machinery around array.array to make them 
> work. I have 
> a good start on this, but I'm not sure when I'm going to have time to 
> work on this more. The biggy though is that psyco only works on x86 
> machines.  What we really need to do is to clone Armin.
> 
> >While the Psyco option is the rosy future of Python, Pyrex 
> is here now, 
> >and maybe adopting it to handle NumArrays well would be easier than 
> >re-writing a bunch of NumArray in C.
> >  
> >
> This sounds like you're conflating two different issues. The 
> first issue 
> is that Numarray is relatively slow for small arrays.  Pyrex 
> may indeed 
> be an easier way to attack this although I wouldn't know, I've only 
> looked at it not tried to use it. However, I think that this is 
> something that can and should wait. Once use cases of numarray being 
> _too_ slow for small arrays start piling up, then it will be time to 
> attack the overhead. Premature optimization is the root of 
> all evil and 
> all that.
> 
> The second issue is how to deal with code that does not 
> vectorize well. 
> Here Pyrex again might help if it were made Numarray aware. However, 
> isn't this what scipy.weave already does? Again, I haven't 
> used weave, 
> but as I understand it, it's another python-c bridge, but one that's 
> more geared toward numerics stuff.
> 
> 
> -tim
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld = Something 
> 2 See! http://www.vasoftware.com 
> _______________________________________________
> Numpy-discussion mailing list Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 

Re: [Numpy-discussion] Psyco MA?

[Tim C. <tc...@op...>]

On Sat, 2003-02-08 at 10:57, Joachim Saul wrote:
> * Tim Churches [2003-02-08 00:07]:
> > However, I agree 100% about the potential for leveraging Pyrex in
> > Numarray. Not just in Numarray, but around it, too. The Numarray team
> > should open serious talks with Greg Ewing about Numarray-enabling Pyrex.
> 
> What is it that needs to be "enabled"? Pyrex handles Numeric (see
> Pyrex FAQ), why should it not handle Numarray? AFAIK Pyrex
> contains no code to specifically support Numeric, and it should
> therefore be straightforward to use it with Numarray as well.

Hmmm, maybe re-implementing MA in Pyrex is possible right now. Double
hmmm....

Tim C

Re: [Numpy-discussion] Psyco MA?

[Joachim S. <li...@js...>]

* Tim Churches [2003-02-08 00:07]:
> However, I agree 100% about the potential for leveraging Pyrex in
> Numarray. Not just in Numarray, but around it, too. The Numarray team
> should open serious talks with Greg Ewing about Numarray-enabling Pyrex.

What is it that needs to be "enabled"? Pyrex handles Numeric (see
Pyrex FAQ), why should it not handle Numarray? AFAIK Pyrex
contains no code to specifically support Numeric, and it should
therefore be straightforward to use it with Numarray as well.

Only drawback is currently lack of support for e.g. slicing
operations in Pyrex.

Cheers,
Joachim

Re: [Numpy-discussion] Psyco MA?

[Tim H. <tim...@ie...>]

Chris Barker wrote:

>oops, sorry about the blank message.
>
>Paul F Dubois wrote:
>  
>
>>{ CC to GvR just to show why I'm +1 on the if-PEP. I liked this in another
>>    
>>
>
>What the heck is the if-PEP ?
>  
>

Pep 308. It's stirring up a bit of a ruckos on CLP as we speak.

>>Perhaps knowlegeable persons could comment on the feasibility of coding MA
>>(masked arrays) in straight Python and then using Psyco on it?
>>    
>>
>
>Is there confusion between Psyco and Pyrex? Psyco runs regular old
>Python bytecode, and individually compiles little pieces of it as needed
>into machine code. AS I understand it, this should make loops where the
>inner part is a pretty simple operation very fast. 
>
>However, Psyco is pretty new, and I have no idea how robust and stable,
>but certainly not cross platform. As it generates machine code, it needs
>to be carefully ported to each hardware platform, and it currently only
>works on x86.
>  
>
Psyco seems fairly stable these days. However it's one of those things 
that probably needs to get a larger cabal of users to shake the bugs out 
of it. I still only use it to play around with because all things that I 
need speed from I end up doing in Numeric anyway.

>Pyrex, on the other hand, is a "Python-like" language that is tranlated
>into C, and then the C is compiled. It generates pretty darn platform
>independent, so it should be able to be used on all platforms.
>
>
>In regard to your question about MA (and any ther similar project): I
>think Psyco has the potential to be the next-generation Python VM, which
>will have much higher performance, and therefore greatly reduce the need
>to write extensions for the sake of performance. I supsect that it could
>do its best with large, multi-dimensional arrays of numbers if there is
>a Python native object of such a type. Psycho, however is not ready for
>general use on all platforms, so in the forseeable future, there is a
>need for other ways to get decent performance. My suggestion follows:
>
>  
>
>>It could have been written a lot simpler if performance didn't dictate
>>trying to leverage off Numeric. In straight Python one can imagine an add,
>>for example, that was roughly:
>>    for k in 0<= k < len(a.data):
>>       result.mask[k] = a.mask[k] or b.mask[k]
>>       result.data[k] = a.data[k] if result.mask[k] else a.data[k] +
>>b.data[k]
>>    
>>
>
>This looks like it could be written in Pyrex. If Pyrex were suitably
>NumArray aware, then it could work great.
>
>What this boils down to, in both the Pyrex and Psyco options, is that
>having a multi-dimensional homogenous numeric data type that is "Native"
>Python is a great idea! With Pyrex and/or Psyco, Numeric3 (NumArray2 ?)
>could be implimented by having only the samallest core in C, and then
>rest in Python (or Pyrex)
>  
>
For Psyco at least you don't need a multidimensional type. You can get 
good results with flat array, in particular array.array. The number I 
posted earlier showed comparable performance for Numeric and a 
multidimensional array type written all in python and psycoized.

And since I suspect  that I'm the mysterious person who's name Paul 
couldn't remember, let me say I suspect the MA would be faster in 
psycoized python than what your doing now as long as a.data was an 
instance of array.array. However, there are at least three problems. 
Psyco doesn't fully support the floating point type('f') right now 
(although it does support most of the various  integral types in 
addition to 'd'). I assume that these masked arrays are 
multidimensional, so someone would have to build the basic 
multidimensional machinery around array.array to make them work. I have 
a good start on this, but I'm not sure when I'm going to have time to 
work on this more. The biggy though is that psyco only works on x86 
machines.  What we really need to do is to clone Armin.

>While the Psyco option is the rosy future of Python, Pyrex is here now,
>and maybe adopting it to handle NumArrays well would be easier than
>re-writing a bunch of NumArray in C.
>  
>
This sounds like you're conflating two different issues. The first issue 
is that Numarray is relatively slow for small arrays.  Pyrex may indeed 
be an easier way to attack this although I wouldn't know, I've only 
looked at it not tried to use it. However, I think that this is 
something that can and should wait. Once use cases of numarray being 
_too_ slow for small arrays start piling up, then it will be time to 
attack the overhead. Premature optimization is the root of all evil and 
all that.

The second issue is how to deal with code that does not vectorize well. 
Here Pyrex again might help if it were made Numarray aware. However, 
isn't this what scipy.weave already does? Again, I haven't used weave, 
but as I understand it, it's another python-c bridge, but one that's 
more geared toward numerics stuff.


-tim

Re: [Numpy-discussion] Psyco MA?

[Tim C. <tc...@op...>]

On Sat, 2003-02-08 at 09:08, Chris Barker wrote:
> While the Psyco option is the rosy future of Python, Pyrex is here now,
> and maybe adopting it to handle NumArrays well would be easier than
> re-writing a bunch of NumArray in C.

Well, Psyco is already immediately useful for many problems on Intel
platforms, but I take your point that its real future is as the
next-generation VM for Python.

However, I agree 100% about the potential for leveraging Pyrex in
Numarray. Not just in Numarray, but around it, too. The Numarray team
should open serious talks with Greg Ewing about Numarray-enabling Pyrex.
And New Zealand is a very nice place to visit (seriously, not joking,
even though I am an Australian [reference to trans-Tasman Sea rivalry
between Asutralia and New Zealand there]).

Tim C

Re: [Numpy-discussion] Psyco MA?

[Chris B. <Chr...@no...>]

oops, sorry about the blank message.

Paul F Dubois wrote:
> { CC to GvR just to show why I'm +1 on the if-PEP. I liked this in another

What the heck is the if-PEP ?

> Perhaps knowlegeable persons could comment on the feasibility of coding MA
> (masked arrays) in straight Python and then using Psyco on it?

Is there confusion between Psyco and Pyrex? Psyco runs regular old
Python bytecode, and individually compiles little pieces of it as needed
into machine code. AS I understand it, this should make loops where the
inner part is a pretty simple operation very fast. 

However, Psyco is pretty new, and I have no idea how robust and stable,
but certainly not cross platform. As it generates machine code, it needs
to be carefully ported to each hardware platform, and it currently only
works on x86.


Pyrex, on the other hand, is a "Python-like" language that is tranlated
into C, and then the C is compiled. It generates pretty darn platform
independent, so it should be able to be used on all platforms.


In regard to your question about MA (and any ther similar project): I
think Psyco has the potential to be the next-generation Python VM, which
will have much higher performance, and therefore greatly reduce the need
to write extensions for the sake of performance. I supsect that it could
do its best with large, multi-dimensional arrays of numbers if there is
a Python native object of such a type. Psycho, however is not ready for
general use on all platforms, so in the forseeable future, there is a
need for other ways to get decent performance. My suggestion follows:

> It could have been written a lot simpler if performance didn't dictate
> trying to leverage off Numeric. In straight Python one can imagine an add,
> for example, that was roughly:
>     for k in 0<= k < len(a.data):
>        result.mask[k] = a.mask[k] or b.mask[k]
>        result.data[k] = a.data[k] if result.mask[k] else a.data[k] +
> b.data[k]

This looks like it could be written in Pyrex. If Pyrex were suitably
NumArray aware, then it could work great.

What this boils down to, in both the Pyrex and Psyco options, is that
having a multi-dimensional homogenous numeric data type that is "Native"
Python is a great idea! With Pyrex and/or Psyco, Numeric3 (NumArray2 ?)
could be implimented by having only the samallest core in C, and then
rest in Python (or Pyrex)

While the Psyco option is the rosy future of Python, Pyrex is here now,
and maybe adopting it to handle NumArrays well would be easier than
re-writing a bunch of NumArray in C.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Psyco MA?

[Chris B. <Chr...@no...>]

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

[Numpy-discussion] Psyco MA?

[Paul F D. <pa...@pf...>]

{ CC to GvR just to show why I'm +1 on the if-PEP. I liked this in =
another
language I used to use; Perl ? }

Perhaps knowlegeable persons could comment on the feasibility of coding =
MA
(masked arrays) in straight Python and then using Psyco on it? The =
existing
implementation is in pure python and uses Numeric to represent the two
arrays it holds (the data and sometimes a mask) in each object. A great =
deal
of wasted motion is devoted to preparing Numeric arrays so as to avoid
operations on masked elements.=20

It could have been written a lot simpler if performance didn't dictate
trying to leverage off Numeric. In straight Python one can imagine an =
add,
for example, that was roughly:
    for k in 0<=3D k < len(a.data):
       result.mask[k] =3D a.mask[k] or b.mask[k]
       result.data[k] =3D a.data[k] if result.mask[k] else a.data[k] +
b.data[k]

(using the new if expression PEP just to confuse the populace)

It seems to me that this might be competitive given the numbers someone
posted before. Alas, I can't remember who was the original poster, but =
I'd
guess they might have a good guess.

> -----Original Message-----
> From: num...@li...=20
> [mailto:num...@li...] On=20
> Behalf Of Chris Barker
> Sent: Friday, February 07, 2003 10:34 AM
> To: fa...@op...; num...@li...
> Subject: Re: [Psyco-devel] RE: [Numpy-discussion] Interesting=20
> Psyco/Numeric/Numarray comparison
>=20
>=20
> Francesc Alted wrote:
>=20
> > For this task may be is worth to consider using Pyrex
> > (http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/) for=20
> that. From=20
> > the
> > website:
> >=20
> > """Pyrex lets you write code that mixes Python and C data types any=20
> > way you want, and compiles it into a C extension for Python."""
>=20
> I've been keeping my eye on Pyrex for a while now, but have=20
> not yet had enough of a use for it to justify tryin git out.=20
> I do ahve a question that I ahve not foudn the answer to on=20
> the web, which could make a big difference to how useful it is to me:
>=20
> Is Pyrex aware of Numeric Arrays?
>=20
> I imagine it could use them just fine, using the generic=20
> Python sequence get item stuff, but that would be a whole lot=20
> lower performance than if it understood the Numeric API and=20
> could access the data array directly. Also, how does it deal=20
> with multiple dimension indexing ( array[3,6,2] ) which the=20
> standard python sequence types do not support?
>=20
> As I think about this, I think your suggestion is fabulous.=20
> Pyrex (or a
> Pyrex-like) language would be a fabulous way to write code=20
> for NumArray, if it really made use of the NumArray API.
>=20
> Thanks for your input,
>=20
> -Chris
>=20
> --=20
> Christopher Barker, Ph.D.
> Oceanographer
>                                     	=09
> NOAA/OR&R/HAZMAT         (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>=20
> Chr...@no...
>=20
>=20
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld =3D Something=20
> 2 See! http://www.vasoftware.com=20
> _______________________________________________
> Numpy-discussion mailing list Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>=20

Re: [Psyco-devel] RE: [Numpy-discussion] Interesting Psyco/Numeric/Numarray comparison

[Chris B. <Chr...@no...>]

Francesc Alted wrote:

> For this task may be is worth to consider using Pyrex
> (http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/) for that. From the
> website:
> 
> """Pyrex lets you write code that mixes Python and C data types any way you
> want, and compiles it into a C extension for Python."""

I've been keeping my eye on Pyrex for a while now, but have not yet had
enough of a use for it to justify tryin git out. I do ahve a question
that I ahve not foudn the answer to on the web, which could make a big
difference to how useful it is to me:

Is Pyrex aware of Numeric Arrays?

I imagine it could use them just fine, using the generic Python sequence
get item stuff, but that would be a whole lot lower performance than if
it understood the Numeric API and could access the data array directly.
Also, how does it deal with multiple dimension indexing ( array[3,6,2] )
which the standard python sequence types do not support?

As I think about this, I think your suggestion is fabulous. Pyrex (or a
Pyrex-like) language would be a fabulous way to write code for NumArray,
if it really made use of the NumArray API.

Thanks for your input,

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

[Numpy-discussion] Some bugs in Numeric fixed today in CVS

[Paul D. <pa...@pf...>]

[ 614808 ] Inconsistent use of tabs and spaces
           Fixed as suggested by Jimmy Retzlaff
               LinearAlgebra.py 
               Matrix.py 
               RNG/__init__.py 
               RNG/Statistics.py 

[ 621032 ] needless work in multiarraymodule.c 
           Fixes suggested by Greg Smith applied.
           Also recoded OBJECT_DotProduct to eliminate a warning error.

[ 630584 ] generalized_inverse of complex array 
           Fix suggested by Greg Smith applied.

[ 652061 ] PyArray_As2D doesn't check pointer.
           Fix suggested by Andrea Riciputi applied.

[ 655512 ] inverse_real_fft incorrect many sizes
           Fix given by mbriest applied.

Re: [Psyco-devel] RE: [Numpy-discussion] Interesting Psyco/Numeric/Numarray comparison

[Francesc A. <fa...@op...>]

A Dimecres 05 Febrer 2003 17:53, Tim Hochberg va escriure:
> However, for small arrays it seems that you're likely to
> be fighting the function call overhead of Python unless you go
> completely, or nearly completely, to C. But that would be a shame as it
> would make modifying/extending numarray that much harder.

For this task may be is worth to consider using Pyrex
(http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/) for that. From the
website:

"""Pyrex lets you write code that mixes Python and C data types any way y=
ou
want, and compiles it into a C extension for Python."""

i.e. if you have code in Python and want to accelerate it, it's quite mor=
e
easy moving it to Pyrex rather than C, as Pyrex has Python syntax. In
addition, it lets you call C routines very easily (just declaring them) a=
nd
provide transparent access to variables, functions and objects in Python
namespace. Apart from the standard Python loop statement, Pyrex introduce=
s a
new kind of for-loop (in the form "for i from 0 <=3D i < n:") for iterati=
ng
over ranges of integers at C speed, that can, for sure, be very handy whe=
n
optimizing many numarray loops.

Another advantage is that Pyrex compiles their own code to C and you can
distribute this C code in your package, without necessity to force the fi=
nal
Pyrex extension user to install Pyrex (because it's just a kind of
compiler). I've been using it for more than six months and it's pretty
stable and works very well (at least for UNIX machines; I don't have
experience on Windows or OSX platforms).

Just my two cents,

--=20
Francesc Alted

Re: [Psyco-devel] RE: [Numpy-discussion] Interesting Psyco/Numeric/Numarray comparison

[Tim H. <tim...@ie...>]

Perry Greenfield wrote:

>The "psymeric" results are indeed interesting. However, I'd like to
>make some remarks about numarray benchmarks. At this stage, most of
>the focus has been on large, contiguous array performance (and as
>can be seen that is where numarray does best). There are a number
>of other improvements that can and will be made to numarray performance
>so some of the other benchmarks are bound to improve (how much is
>uncertain). For example, the current behavior with strided arrays
>results in looping over subblocks of the array, and that looping is
>done on relatively small blocks in Python. We haven't done any tuning
>yet to see what the optimum size of block should be (it may be machine
>dependent as well), and it is likely that the loop will eventually be
>moved into C. Small array performance should improve quite a bit, we
>are looking into how to do that now and should have a better idea
>soon of whether we can beat Numeric's performance or not.
>  
>
I fully expect numarray to beat Numeric for large arrays eventually just 
based on the fact the psymeric tends to be slightly faster Numeric now 
for many cases. However, for small arrays it seems that you're likely to 
be fighting the function call overhead of Python unless you go 
completely, or nearly completely, to C. But that would be a shame as it 
would make modifying/extending numarray that much harder.

>But "psymeric" approach raises an obvious question (implied I guess, but
>not explicitly stated). With Psyco, is there a need for Numeric or
>numarray at all? I haven't thought this through in great detail, but at
>least one issue seems tough to address in this approach, and that is
>handling numeric types not supported by Python (e.g., Int8, Int16 UInt16,
>Float32, etc.).  Are you offering the possiblity of the "pysmeric"
>approach as being the right way to go,
>  
>
I think there are too many open questions at this point to be a serious 
contender. It's interesting enough and the payoff would be big enough 
that I think it's worth throwing out some of the questions and see if 
anything interesting pops out.

> and if so, how would you handle
>this issue?
>  
>

The types issue may not be a problem. Python's array.array supports a 
full set of types 
(http://www.python.org/doc/current/lib/module-array.html). However, 
psyco does not currently support fast operations on types 'f', 'I' and 
'L'. I don't know if this is a technical problem, or something that's 
likely to be resolved in time. The 'f' (Float32) case is critical, the 
others less so.

Armin, if you're reading this perhaps you'd like to comment?

>On the other hand, there are lots of algorithms that cannot be handled
>well with array manipulations.
>
This is where the Psyco approach would shine. One occasionally runs into 
cases where some part of the computation just cannot be done naturaly 
with array operations. A common case is the equivalent of this bit of C 
code: "A[i] = (C[i]<TOL) ? B[i]+5 : C[i]-5". This can be done using 
take, but it requires a bunch of extra memory (3 arrays worth) and 
calculations. In principle at least this could be done using psymeric in 
a more natural way without the extra memory and calculations.

> It would seem that psyco would be a natural
>alternative in such cases (as long as one is content to use Float64 or
>Int32), but it isn't obivious that these require arrays as anything but
>data structures (e.g. places to obtain and store scalars).
>  
>
That's not been my experience. When I've run into awkward cases like 
this it's been in situations where nearly all of my computations could 
be vectorized.

Anyway, here are what I see as the issues with this type of approach:

* Types: I believe that this should not be a problem

* Interfacing with C/Fortran:  This seems necessary for any Numeric 
wannabe. It seems that it must be possible, but it may require a bit of 
C-code, so it may not be possible to get completely away from C.

* Speed: It's not clear to me at this point whether psymeric would get 
any faster than it currently is. It's pretty fast now, but the factor of 
two difference between it and numarray for contiguous arrays (a common 
case) is nothing to sneeze at.

Cross-platform: This is the reall killer. Psyco only runs on x86 
machines. I don't know if or when that's likely to change. Not being 
cross platform seems nix this from being a serious contender as a 
Numeric replacement for the time being.


-tim

RE: [Numpy-discussion] Interesting Psyco/Numeric/Numarray comparison

[Perry G. <pe...@st...>]

Tim Hochberg writes:

> I was inspired by Armin's latest Psyco version to try and see how well
> one could do with NumPy/NumArray implemented in Psycotic Python. I wrote
> a bare bones, pure Python, Numeric array class loosely based on Jnumeric
> (which in turn was loosely based on Numeric). The buffer is just
> Python's array.array. At the moment, all that one can do to the arrays
> is add and index them and the code is still a bit of a mess. I plan to
> clean things up over the next week in my copius free time <0.999 wink>
> and at that point it should be easy add the remaining operations.
>
> I benchmarked this code, which I'm calling Psymeric for the moment,
> against NumPy and Numarray to see how it did. I used a variety of array
> sizes, but mostly relatively large arrays of shape (500,100) and of type
> Float64 and Int32 (mixed and with consistent types) as well as scalar
> values. Looking at the benchmark data one comes to three main conclusions:
>    * For small arrays NumPy always wins. Both Numarray and Psymeric have
> much larger overhead.
>    * For large, contiguouse arrays, Numarray is about twice as fast as
> either of the other two.
>    * For large, noncontiguous arrays, Psymeric and NumPy are ~20% faster
> than Numarray
> The impressive thing is that Psymeric is generally slightly faster than
> NumPy when adding two arrays. It's slightly slower (~10%) when adding an
> array and a scalar although I suspect that could be fixed by some
> special casing a la Numarray. Adding two (500,100) arrays of type
> Float64 together results in the following timings:
>                    psymeric      numpy       numarray
> contiguous  0.0034 s      0.0038 s    0.0019 s
> stride-2      0.0020 s      0.0023 s    0.0033 s
>
> I'm not sure if this is important, but it is an impressive demonstration
> of Psyco! More later when I get the code a bit more cleaned up.
>
> -tim
> 0.002355
>
> 0.002355
>
The "psymeric" results are indeed interesting. However, I'd like to
make some remarks about numarray benchmarks. At this stage, most of
the focus has been on large, contiguous array performance (and as
can be seen that is where numarray does best). There are a number
of other improvements that can and will be made to numarray performance
so some of the other benchmarks are bound to improve (how much is
uncertain). For example, the current behavior with strided arrays
results in looping over subblocks of the array, and that looping is
done on relatively small blocks in Python. We haven't done any tuning
yet to see what the optimum size of block should be (it may be machine
dependent as well), and it is likely that the loop will eventually be
moved into C. Small array performance should improve quite a bit, we
are looking into how to do that now and should have a better idea
soon of whether we can beat Numeric's performance or not.

But "psymeric" approach raises an obvious question (implied I guess, but
not explicitly stated). With Psyco, is there a need for Numeric or
numarray at all? I haven't thought this through in great detail, but at
least one issue seems tough to address in this approach, and that is
handling numeric types not supported by Python (e.g., Int8, Int16 UInt16,
Float32, etc.).  Are you offering the possiblity of the "pysmeric"
approach as being the right way to go, and if so, how would you handle
this issue?

On the other hand, there are lots of algorithms that cannot be handled
well with array manipulations. It would seem that psyco would be a natural
alternative in such cases (as long as one is content to use Float64 or
Int32), but it isn't obivious that these require arrays as anything but
data structures (e.g. places to obtain and store scalars).

Perry Greenfield

Re: [Numpy-discussion] more than 2-D numarrays in recarray?

[Francesc A. <fa...@op...>]

A Dimarts 04 Febrer 2003 16:40, Perry Greenfield va escriure:
> We'd like to work with you about how that should be best implemented.
> Basically the issue is how we save the shape information for that field=
=2E
> I don't think it would be hard to implement.

Ok, great!

Well, my proposals for extended recarray syntax are:

1.- Extend the actual formats to read something like:
['(1,)i1', '(3,4)i4', '(16,)a', '(2,3,4)i2']

Pro's:
    - It's the straightforward extension of the actual format
    - Should be easy to implement
    - Note that the charcodes has been substituted by a slightly more ver=
bose
      version ('i2' instead of 's', for example)
    - Short and simple
   =20
Con's:
    - It is still string-code based
    - Implicit field order

2.- Make use of the syntax I'm suggesting in past messages:

class Particle(IsRecord):
    name        =3D Col(CharType, (16,), dflt=3D"", pos=3D3)  # 16-charac=
ter String
    ADCcount    =3D Col(Int8, (1,), dflt=3D0, pos=3D1)    # signed byte
    TDCcount    =3D Col(Int32, (3,4), dflt=3D0, pos=3D2)    # signed inte=
ger
    grid_i      =3D Col(Int16, (2,3,4), dflt=3D0, pos=3D4)    # signed sh=
ort integer

Pro's:
    - It gets rid of charcodes or string codes
    - The map between name and type is visually clear
    - Explicit field order
    - The columns can be defined as __slots__ in the class constructor
      making impossible to assign (through __setattr__ for example) value=
s to
      non-existing columns.
    - Is elegant (IMO)

Con's:
    - Requires more typing to define
    - Not as concise as 1) (but a short representation can be made inside
      IsRecord!)
    - Difficult to define dynamically
=09 =20
3.- Similar than 2), but with a dictionary like:

Particle =3D {
    "name"      : Col(CharType, (16,), dflt=3D"", pos=3D3),  # 16-charact=
er String
    "ADCcount"  : Col(Int8, (1,), dflt=3D0, pos=3D1),    # signed byte
    "TDCcount"  : Col(Int32, (3,4), dflt=3D0, pos=3D2),    # signed integ=
er
    "grid_i"    : Col(Int16, (2,3,4), dflt=3D0, pos=3D4),    # signed sho=
rt=20
integer
    }

Pro's:
    - It gets rid of charcodes or string codes
    - The map between name and type is visually clear
    - Explicit field order
    - Easy to build dynamically
Con's
    - No possibility to define __slots__
    - Not as elegant as 2), but it looks fine.

4.- List-based approach:

Particle =3D [
    Col(Int8, (1,), dflt=3D0),    # signed byte
    Col(Int32, (3,4), dflt=3D0),    # signed integer
    Col(CharType, (16,), dflt=3D""),  # 16-character String
    Col(Int16, (2,3,4), dflt=3D0),    # signed short integer
    ]

Pro's:
    - Costs less to type (less verbose)
    - Easy to build dynamically
Con's
    - Implicit field order
    - Map between field names and contents not visually clear
   =20
Note: In the previous discussion explicit order has been considered bette=
r
than implicit, following the Python mantra, and although some people may
think that this don't apply well here, I do (but, again, this is purely
subjective).
   =20
Of course, a combination of 2 alternatives can be the best. My current
experience tells me that a combination of 2 and 3 may be very good. In th=
at
way, a user can define their recarrays as classes, but if he needs to def=
ine
them dynamically, the recarray constructor can accept also a dictionary l=
ike
3 (but, obviously, the same applies to case 4).

In the end, the recarray instance should have a variable that points to t=
his
definition class, where metadata is keeped, but a shortcut in the form
1) can also be constructed for convenience.

IMO integrating options 2 and 3 (even 4) are not difficult to implement a=
nd
in fact, such a combination is already present in PyTables CVS version. I
even might provide a recarray version with 2 & 3 integrated for developer=
s
evaluation.

Comments?,

--=20
Francesc Alted

[Numpy-discussion] Interesting Psyco/Numeric/Numarray comparison

[Tim H. <tim...@ie...>]

I was inspired by Armin's latest Psyco version to try and see how well 
one could do with NumPy/NumArray implemented in Psycotic Python. I wrote 
a bare bones, pure Python, Numeric array class loosely based on Jnumeric 
(which in turn was loosely based on Numeric). The buffer is just 
Python's array.array. At the moment, all that one can do to the arrays 
is add and index them and the code is still a bit of a mess. I plan to 
clean things up over the next week in my copius free time <0.999 wink> 
and at that point it should be easy add the remaining operations.

I benchmarked this code, which I'm calling Psymeric for the moment, 
against NumPy and Numarray to see how it did. I used a variety of array 
sizes, but mostly relatively large arrays of shape (500,100) and of type 
Float64 and Int32 (mixed and with consistent types) as well as scalar 
values. Looking at the benchmark data one comes to three main conclusions:
   * For small arrays NumPy always wins. Both Numarray and Psymeric have 
much larger overhead.
   * For large, contiguouse arrays, Numarray is about twice as fast as 
either of the other two.
   * For large, noncontiguous arrays, Psymeric and NumPy are ~20% faster 
than Numarray
The impressive thing is that Psymeric is generally slightly faster than 
NumPy when adding two arrays. It's slightly slower (~10%) when adding an 
array and a scalar although I suspect that could be fixed by some 
special casing a la Numarray. Adding two (500,100) arrays of type 
Float64 together results in the following timings:
                   psymeric      numpy       numarray
contiguous  0.0034 s      0.0038 s    0.0019 s
stride-2      0.0020 s      0.0023 s    0.0033 s

I'm not sure if this is important, but it is an impressive demonstration 
of Psyco! More later when I get the code a bit more cleaned up.

-tim
0.002355

0.002355



 

RE: [Numpy-discussion] more than 2-D numarrays in recarray?

[Perry G. <pe...@st...>]

> 
> A Dimarts 04 Febrer 2003 13:19, Todd Miller va escriure:
> > I see two problems with multi-d numarray fields, both
> > solvable:
> >
> > 1.  Multidimensional numarrays must be described in the recarray spec.
> >
> > 2. Either numarray or recarray must be able to handle a (slightly) more
> > complicated case of recomputing array strides from shape and
> > (bytestride,record-length).
> >
> > I didn't design or implement recarray so there may be other problems as
> > well.
> 
> I had a look at the code and it seems like you are right.
> 
> > I don't think it's a priority now.  What do you need them for?
> 
> Well, I've adopted the recarray object (actually a slightly 
> modified version
> of it) to be a fundamental building block in next release of PyTables. If
> arbitrary dimensionality were implemented, the resulting tables would be
> more general. Moreover, I'm thinking about implementing unlimited 
> (just one
> axis) array dimension support and having a degenerated recarray with just
> one column as a multimensional numarray object would easy quite a lot the
> implementation.
> 
> Of course, I could implement my own recarray version with that 
> support, but
> I just don't want to diverge so much from the reference implementation.
> 
> -- 
> Francesc Alted
> 
As Todd says, the initial implementation was to support only 1-d cases.
There is no fundamental reason why it shouldn't support the general case.
We'd like to work with you about how that should be best implemented.
Basically the issue is how we save the shape information for that field.
I don't think it would be hard to implement.

Perry

Re: [Numpy-discussion] more than 2-D numarrays in recarray?

[Francesc A. <fa...@op...>]

A Dimarts 04 Febrer 2003 13:19, Todd Miller va escriure:
> I see two problems with multi-d numarray fields, both
> solvable:
>
> 1.  Multidimensional numarrays must be described in the recarray spec.
>
> 2. Either numarray or recarray must be able to handle a (slightly) more
> complicated case of recomputing array strides from shape and
> (bytestride,record-length).
>
> I didn't design or implement recarray so there may be other problems as
> well.

I had a look at the code and it seems like you are right.

> I don't think it's a priority now.  What do you need them for?

Well, I've adopted the recarray object (actually a slightly modified vers=
ion
of it) to be a fundamental building block in next release of PyTables. If
arbitrary dimensionality were implemented, the resulting tables would be
more general. Moreover, I'm thinking about implementing unlimited (just o=
ne
axis) array dimension support and having a degenerated recarray with just
one column as a multimensional numarray object would easy quite a lot the
implementation.

Of course, I could implement my own recarray version with that support, b=
ut
I just don't want to diverge so much from the reference implementation.

--=20
Francesc Alted

Re: [Numpy-discussion] more than 2-D numarrays in recarray?

[Todd M. <jm...@st...>]

Francesc Alted wrote:

>Hi,
>
>It seems that recarray doesn't support more than 1-D numarray arrays as
>fields. Is that a fundamental limitation? 
>
I don't think it is fundamental,  merely a question of what is needed 
and works easily.  I see two problems with multi-d numarray fields, both 
solvable:

1.  Multidimensional numarrays must be described in the recarray spec.

2. Either numarray or recarray must be able to handle a (slightly) more 
complicated case of recomputing array strides from shape and 
(bytestride,record-length).

I didn't design or implement recarray so there may be other problems as 
well.

>If not, do you plan to support
>arbitrary dimensions in the future?.
>
I don't think it's a priority now.  What do you need them for?

>
>Thanks,
>
>  
>
Regards,
Todd

[Numpy-discussion] more than 2-D numarrays in recarray?

[Francesc A. <fa...@op...>]

Hi,

It seems that recarray doesn't support more than 1-D numarray arrays as
fields. Is that a fundamental limitation? If not, do you plan to support
arbitrary dimensions in the future?.

Thanks,

--=20
Francesc Alted

Re: [Numpy-discussion] fastest way to make two vectors into anarray

[Rob H. <ro...@ho...>]

Chris Barker wrote:
> 
> X = transpose([x]+[y])
> 
> 
> well, I learned a little bit more about Numeric today.
> 

I've been skipping through a lot of messages today because I was getting 
behind on mailing list traffic, but I missed one thing in the discussion 
so far (sorry if it was marked already):

    transpose doesn't actually do any work.

Actually, transpose only sets the "strides" counts differently, and this 
is blazingly fast. What is NOT fast is using the transposed array later! 
The problem is that many routines actually require a contiguous array, 
and will make a temporary local contiguous copy. This may happen 
multiple times if the lifetime of the transposed array is long. Even 
routines that do not require a contiguous array and can actually use the 
strides may run significantly slower because the CPU cache is trashed a 
lot by the high strides.

Moral: you can't test this code by looping a 1000 times through it, you 
actually should take into account the time it takes to make a contiguous 
array immediately after the transpose call.

Regards,

Rob Hooft
-- 
Rob W.W. Hooft  ||  ro...@ho...  ||  http://www.hooft.net/people/rob/

Re: [Numpy-discussion] fastest way to make two vectors into anarray

[Chris B. <Chr...@no...>]

John Hunter wrote:
>     John> I have two equal length 1D arrays of 256-4096 complex or
>     John> floating point numbers which I need to put into a
>     John> shape=(len(x),2) array.

> I tested all the suggested methods and the transpose with [x] and [y]
> was the clear winner, with an 8 fold speed up over my original code.
> The concatenate method was between 2-3 times faster.

I was a little surprised by this, as I figured that the transpose method
made an extra copy of the data (array() makes one copy, transpose()
another. So I looked at the source for concatenate:

def concatenate(a, axis=0):
    """concatenate(a, axis=0) joins the tuple of sequences in a into a
single
    NumPy array.
    """
    if axis == 0:
        return multiarray.concatenate(a)
    else:
        new_list = []
        for m in a:
            new_list.append(swapaxes(m, axis, 0))
    return swapaxes(multiarray.concatenate(new_list), axis, 0)

So, if you are concantenating along anything other than the zero-th
axis, you end up doing something similar to the transpose method. Seeign
this, I trioed something else:

def test_concat2(x,y):
    x.shape = (1,-1)
    y.shape = (1,-1)
    X = transpose( concatenate( (x, y) ) )
    x.shape = (-1,)
    y.shape = (-1,)

This then uses the native concatenate, but requires an extra copy in teh
transpose.

Here's a somewhat cleaner version, though you get more copies:

def test_concat3(x,y):
    "Thanks to Chris Barker and Bryan Cole"
    X = transpose( concatenate( ( reshape(x,(1,-1)), reshape(y,(1,-1)) )
) )

Here are the test results:

testing on vectors of length:  4096

test_concat 0.286280035973
test_transpose 0.100033998489
test_naive 0.805399060249
test_concat3 0.109319090843
test_concat2 0.136469960213

All the transpose methods are essentially a tie. Would it be that hard
for concatenate to do it's thing for any axis in C? It does seem like
this is a fairly basic operation, and shouldn't require more than one
copy.

By the way, I realised that the transpose method had an extra call.
transpose() can take an approprriate python sequence, so this works just
fine:

def test_transpose2(x,y):
    X = transpose([x]+[y])

However, it doesn't really save you the copy, as I'm retty sure
transpose makes a copy internally anyway. Test results:
testing on vectors of length:  4096

test_transpose 0.104995965958
test_transpose2 0.103582024574

I think the winner is:

X = transpose([x]+[y])


well, I learned a little bit more about Numeric today.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                    		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] numarray 0.4 on osX/darwin

[Todd M. <jm...@st...>]

Sebastian Haase wrote:

>Hi everybody,
>I tried a 'python2.2 setup.py install'
>of numarray  on a Mac running os-X (10.1; I have also Fink installed)
>I starts crunching until:
>/usr/bin/ld: Undefined symbols:
>_fclearexcept
>_fetestexcept
>
>Anyone out there, who uses numarray on osX ?
>
>I'm thankful for any pointer...
>
>Sebastian Haase
>  
>
Hi Sebastian,

I am very much a Mac-Amateur,  but I have run numarray under osX by 
first installing a local UNIX version of Python using the source 
tarball.  The steps were roughly as follows:

1. Obtain and unpack the Python source tarball in you home directory. 
 cd there.

2. Configure Python using:  ./configure --prefix=$HOME  

3. Edit the Makefile for the following:

61c61
 > LDFLAGS=
---
< LDFLAGS=      -framework System -framework CoreServices -framework 
Foundation

This was the only (reasonable) way I could figure out how to tunnel link 
time options down through the distutils in the proper command line 
order.  I'm not really sure this is a minimal set of frameworks,  but it 
did at least work.

4. Build and install python:  make ; make install

5.  Obtain and unpack the numarray source tarball.  cd there.

6.  Build and install numarray:  python setupall.py install

7.  Put $HOME/bin on your PATH and rehash.


Todd

>
>
>
>-------------------------------------------------------
>This SF.NET email is sponsored by:
>SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
>http://www.vasoftware.com
>_______________________________________________
>Numpy-discussion mailing list
>Num...@li...
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>