Menu
▾
▴
numpy-discussion
|
From: Tim H. <tim...@ie...> - 2003-02-04 16:51:39
|
I was inspired by Armin's latest Psyco version to try and see how well
one could do with NumPy/NumArray implemented in Psycotic Python. I wrote
a bare bones, pure Python, Numeric array class loosely based on Jnumeric
(which in turn was loosely based on Numeric). The buffer is just
Python's array.array. At the moment, all that one can do to the arrays
is add and index them and the code is still a bit of a mess. I plan to
clean things up over the next week in my copius free time <0.999 wink>
and at that point it should be easy add the remaining operations.
I benchmarked this code, which I'm calling Psymeric for the moment,
against NumPy and Numarray to see how it did. I used a variety of array
sizes, but mostly relatively large arrays of shape (500,100) and of type
Float64 and Int32 (mixed and with consistent types) as well as scalar
values. Looking at the benchmark data one comes to three main conclusions:
* For small arrays NumPy always wins. Both Numarray and Psymeric have
much larger overhead.
* For large, contiguouse arrays, Numarray is about twice as fast as
either of the other two.
* For large, noncontiguous arrays, Psymeric and NumPy are ~20% faster
than Numarray
The impressive thing is that Psymeric is generally slightly faster than
NumPy when adding two arrays. It's slightly slower (~10%) when adding an
array and a scalar although I suspect that could be fixed by some
special casing a la Numarray. Adding two (500,100) arrays of type
Float64 together results in the following timings:
psymeric numpy numarray
contiguous 0.0034 s 0.0038 s 0.0019 s
stride-2 0.0020 s 0.0023 s 0.0033 s
I'm not sure if this is important, but it is an impressive demonstration
of Psyco! More later when I get the code a bit more cleaned up.
-tim
0.002355
0.002355
|
|
From: Perry G. <pe...@st...> - 2003-02-05 15:05:44
|
Tim Hochberg writes: > I was inspired by Armin's latest Psyco version to try and see how well > one could do with NumPy/NumArray implemented in Psycotic Python. I wrote > a bare bones, pure Python, Numeric array class loosely based on Jnumeric > (which in turn was loosely based on Numeric). The buffer is just > Python's array.array. At the moment, all that one can do to the arrays > is add and index them and the code is still a bit of a mess. I plan to > clean things up over the next week in my copius free time <0.999 wink> > and at that point it should be easy add the remaining operations. > > I benchmarked this code, which I'm calling Psymeric for the moment, > against NumPy and Numarray to see how it did. I used a variety of array > sizes, but mostly relatively large arrays of shape (500,100) and of type > Float64 and Int32 (mixed and with consistent types) as well as scalar > values. Looking at the benchmark data one comes to three main conclusions: > * For small arrays NumPy always wins. Both Numarray and Psymeric have > much larger overhead. > * For large, contiguouse arrays, Numarray is about twice as fast as > either of the other two. > * For large, noncontiguous arrays, Psymeric and NumPy are ~20% faster > than Numarray > The impressive thing is that Psymeric is generally slightly faster than > NumPy when adding two arrays. It's slightly slower (~10%) when adding an > array and a scalar although I suspect that could be fixed by some > special casing a la Numarray. Adding two (500,100) arrays of type > Float64 together results in the following timings: > psymeric numpy numarray > contiguous 0.0034 s 0.0038 s 0.0019 s > stride-2 0.0020 s 0.0023 s 0.0033 s > > I'm not sure if this is important, but it is an impressive demonstration > of Psyco! More later when I get the code a bit more cleaned up. > > -tim > 0.002355 > > 0.002355 > The "psymeric" results are indeed interesting. However, I'd like to make some remarks about numarray benchmarks. At this stage, most of the focus has been on large, contiguous array performance (and as can be seen that is where numarray does best). There are a number of other improvements that can and will be made to numarray performance so some of the other benchmarks are bound to improve (how much is uncertain). For example, the current behavior with strided arrays results in looping over subblocks of the array, and that looping is done on relatively small blocks in Python. We haven't done any tuning yet to see what the optimum size of block should be (it may be machine dependent as well), and it is likely that the loop will eventually be moved into C. Small array performance should improve quite a bit, we are looking into how to do that now and should have a better idea soon of whether we can beat Numeric's performance or not. But "psymeric" approach raises an obvious question (implied I guess, but not explicitly stated). With Psyco, is there a need for Numeric or numarray at all? I haven't thought this through in great detail, but at least one issue seems tough to address in this approach, and that is handling numeric types not supported by Python (e.g., Int8, Int16 UInt16, Float32, etc.). Are you offering the possiblity of the "pysmeric" approach as being the right way to go, and if so, how would you handle this issue? On the other hand, there are lots of algorithms that cannot be handled well with array manipulations. It would seem that psyco would be a natural alternative in such cases (as long as one is content to use Float64 or Int32), but it isn't obivious that these require arrays as anything but data structures (e.g. places to obtain and store scalars). Perry Greenfield |
|
From: Tim H. <tim...@ie...> - 2003-02-05 16:53:35
|
Perry Greenfield wrote: >The "psymeric" results are indeed interesting. However, I'd like to >make some remarks about numarray benchmarks. At this stage, most of >the focus has been on large, contiguous array performance (and as >can be seen that is where numarray does best). There are a number >of other improvements that can and will be made to numarray performance >so some of the other benchmarks are bound to improve (how much is >uncertain). For example, the current behavior with strided arrays >results in looping over subblocks of the array, and that looping is >done on relatively small blocks in Python. We haven't done any tuning >yet to see what the optimum size of block should be (it may be machine >dependent as well), and it is likely that the loop will eventually be >moved into C. Small array performance should improve quite a bit, we >are looking into how to do that now and should have a better idea >soon of whether we can beat Numeric's performance or not. > > I fully expect numarray to beat Numeric for large arrays eventually just based on the fact the psymeric tends to be slightly faster Numeric now for many cases. However, for small arrays it seems that you're likely to be fighting the function call overhead of Python unless you go completely, or nearly completely, to C. But that would be a shame as it would make modifying/extending numarray that much harder. >But "psymeric" approach raises an obvious question (implied I guess, but >not explicitly stated). With Psyco, is there a need for Numeric or >numarray at all? I haven't thought this through in great detail, but at >least one issue seems tough to address in this approach, and that is >handling numeric types not supported by Python (e.g., Int8, Int16 UInt16, >Float32, etc.). Are you offering the possiblity of the "pysmeric" >approach as being the right way to go, > > I think there are too many open questions at this point to be a serious contender. It's interesting enough and the payoff would be big enough that I think it's worth throwing out some of the questions and see if anything interesting pops out. > and if so, how would you handle >this issue? > > The types issue may not be a problem. Python's array.array supports a full set of types (http://www.python.org/doc/current/lib/module-array.html). However, psyco does not currently support fast operations on types 'f', 'I' and 'L'. I don't know if this is a technical problem, or something that's likely to be resolved in time. The 'f' (Float32) case is critical, the others less so. Armin, if you're reading this perhaps you'd like to comment? >On the other hand, there are lots of algorithms that cannot be handled >well with array manipulations. > This is where the Psyco approach would shine. One occasionally runs into cases where some part of the computation just cannot be done naturaly with array operations. A common case is the equivalent of this bit of C code: "A[i] = (C[i]<TOL) ? B[i]+5 : C[i]-5". This can be done using take, but it requires a bunch of extra memory (3 arrays worth) and calculations. In principle at least this could be done using psymeric in a more natural way without the extra memory and calculations. > It would seem that psyco would be a natural >alternative in such cases (as long as one is content to use Float64 or >Int32), but it isn't obivious that these require arrays as anything but >data structures (e.g. places to obtain and store scalars). > > That's not been my experience. When I've run into awkward cases like this it's been in situations where nearly all of my computations could be vectorized. Anyway, here are what I see as the issues with this type of approach: * Types: I believe that this should not be a problem * Interfacing with C/Fortran: This seems necessary for any Numeric wannabe. It seems that it must be possible, but it may require a bit of C-code, so it may not be possible to get completely away from C. * Speed: It's not clear to me at this point whether psymeric would get any faster than it currently is. It's pretty fast now, but the factor of two difference between it and numarray for contiguous arrays (a common case) is nothing to sneeze at. Cross-platform: This is the reall killer. Psyco only runs on x86 machines. I don't know if or when that's likely to change. Not being cross platform seems nix this from being a serious contender as a Numeric replacement for the time being. -tim |
|
From: Francesc A. <fa...@op...> - 2003-02-07 17:45:05
|
A Dimecres 05 Febrer 2003 17:53, Tim Hochberg va escriure: > However, for small arrays it seems that you're likely to > be fighting the function call overhead of Python unless you go > completely, or nearly completely, to C. But that would be a shame as it > would make modifying/extending numarray that much harder. For this task may be is worth to consider using Pyrex (http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/) for that. From the website: """Pyrex lets you write code that mixes Python and C data types any way y= ou want, and compiles it into a C extension for Python.""" i.e. if you have code in Python and want to accelerate it, it's quite mor= e easy moving it to Pyrex rather than C, as Pyrex has Python syntax. In addition, it lets you call C routines very easily (just declaring them) a= nd provide transparent access to variables, functions and objects in Python namespace. Apart from the standard Python loop statement, Pyrex introduce= s a new kind of for-loop (in the form "for i from 0 <=3D i < n:") for iterati= ng over ranges of integers at C speed, that can, for sure, be very handy whe= n optimizing many numarray loops. Another advantage is that Pyrex compiles their own code to C and you can distribute this C code in your package, without necessity to force the fi= nal Pyrex extension user to install Pyrex (because it's just a kind of compiler). I've been using it for more than six months and it's pretty stable and works very well (at least for UNIX machines; I don't have experience on Windows or OSX platforms). Just my two cents, --=20 Francesc Alted |
|
From: Chris B. <Chr...@no...> - 2003-02-07 19:05:46
|
Francesc Alted wrote: > For this task may be is worth to consider using Pyrex > (http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/) for that. From the > website: > > """Pyrex lets you write code that mixes Python and C data types any way you > want, and compiles it into a C extension for Python.""" I've been keeping my eye on Pyrex for a while now, but have not yet had enough of a use for it to justify tryin git out. I do ahve a question that I ahve not foudn the answer to on the web, which could make a big difference to how useful it is to me: Is Pyrex aware of Numeric Arrays? I imagine it could use them just fine, using the generic Python sequence get item stuff, but that would be a whole lot lower performance than if it understood the Numeric API and could access the data array directly. Also, how does it deal with multiple dimension indexing ( array[3,6,2] ) which the standard python sequence types do not support? As I think about this, I think your suggestion is fabulous. Pyrex (or a Pyrex-like) language would be a fabulous way to write code for NumArray, if it really made use of the NumArray API. Thanks for your input, -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chr...@no... |
|
From: Paul F D. <pa...@pf...> - 2003-02-07 21:47:58
|
{ CC to GvR just to show why I'm +1 on the if-PEP. I liked this in =
another
language I used to use; Perl ? }
Perhaps knowlegeable persons could comment on the feasibility of coding =
MA
(masked arrays) in straight Python and then using Psyco on it? The =
existing
implementation is in pure python and uses Numeric to represent the two
arrays it holds (the data and sometimes a mask) in each object. A great =
deal
of wasted motion is devoted to preparing Numeric arrays so as to avoid
operations on masked elements.=20
It could have been written a lot simpler if performance didn't dictate
trying to leverage off Numeric. In straight Python one can imagine an =
add,
for example, that was roughly:
for k in 0<=3D k < len(a.data):
result.mask[k] =3D a.mask[k] or b.mask[k]
result.data[k] =3D a.data[k] if result.mask[k] else a.data[k] +
b.data[k]
(using the new if expression PEP just to confuse the populace)
It seems to me that this might be competitive given the numbers someone
posted before. Alas, I can't remember who was the original poster, but =
I'd
guess they might have a good guess.
> -----Original Message-----
> From: num...@li...=20
> [mailto:num...@li...] On=20
> Behalf Of Chris Barker
> Sent: Friday, February 07, 2003 10:34 AM
> To: fa...@op...; num...@li...
> Subject: Re: [Psyco-devel] RE: [Numpy-discussion] Interesting=20
> Psyco/Numeric/Numarray comparison
>=20
>=20
> Francesc Alted wrote:
>=20
> > For this task may be is worth to consider using Pyrex
> > (http://www.cosc.canterbury.ac.nz/~greg/python/Pyrex/) for=20
> that. From=20
> > the
> > website:
> >=20
> > """Pyrex lets you write code that mixes Python and C data types any=20
> > way you want, and compiles it into a C extension for Python."""
>=20
> I've been keeping my eye on Pyrex for a while now, but have=20
> not yet had enough of a use for it to justify tryin git out.=20
> I do ahve a question that I ahve not foudn the answer to on=20
> the web, which could make a big difference to how useful it is to me:
>=20
> Is Pyrex aware of Numeric Arrays?
>=20
> I imagine it could use them just fine, using the generic=20
> Python sequence get item stuff, but that would be a whole lot=20
> lower performance than if it understood the Numeric API and=20
> could access the data array directly. Also, how does it deal=20
> with multiple dimension indexing ( array[3,6,2] ) which the=20
> standard python sequence types do not support?
>=20
> As I think about this, I think your suggestion is fabulous.=20
> Pyrex (or a
> Pyrex-like) language would be a fabulous way to write code=20
> for NumArray, if it really made use of the NumArray API.
>=20
> Thanks for your input,
>=20
> -Chris
>=20
> --=20
> Christopher Barker, Ph.D.
> Oceanographer
> =09
> NOAA/OR&R/HAZMAT (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>=20
> Chr...@no...
>=20
>=20
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld =3D Something=20
> 2 See! http://www.vasoftware.com=20
> _______________________________________________
> Numpy-discussion mailing list Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>=20
|
|
From: Chris B. <Chr...@no...> - 2003-02-07 22:24:20
|
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Chris B. <Chr...@no...> - 2003-02-07 22:40:55
|
oops, sorry about the blank message.
Paul F Dubois wrote:
> { CC to GvR just to show why I'm +1 on the if-PEP. I liked this in another
What the heck is the if-PEP ?
> Perhaps knowlegeable persons could comment on the feasibility of coding MA
> (masked arrays) in straight Python and then using Psyco on it?
Is there confusion between Psyco and Pyrex? Psyco runs regular old
Python bytecode, and individually compiles little pieces of it as needed
into machine code. AS I understand it, this should make loops where the
inner part is a pretty simple operation very fast.
However, Psyco is pretty new, and I have no idea how robust and stable,
but certainly not cross platform. As it generates machine code, it needs
to be carefully ported to each hardware platform, and it currently only
works on x86.
Pyrex, on the other hand, is a "Python-like" language that is tranlated
into C, and then the C is compiled. It generates pretty darn platform
independent, so it should be able to be used on all platforms.
In regard to your question about MA (and any ther similar project): I
think Psyco has the potential to be the next-generation Python VM, which
will have much higher performance, and therefore greatly reduce the need
to write extensions for the sake of performance. I supsect that it could
do its best with large, multi-dimensional arrays of numbers if there is
a Python native object of such a type. Psycho, however is not ready for
general use on all platforms, so in the forseeable future, there is a
need for other ways to get decent performance. My suggestion follows:
> It could have been written a lot simpler if performance didn't dictate
> trying to leverage off Numeric. In straight Python one can imagine an add,
> for example, that was roughly:
> for k in 0<= k < len(a.data):
> result.mask[k] = a.mask[k] or b.mask[k]
> result.data[k] = a.data[k] if result.mask[k] else a.data[k] +
> b.data[k]
This looks like it could be written in Pyrex. If Pyrex were suitably
NumArray aware, then it could work great.
What this boils down to, in both the Pyrex and Psyco options, is that
having a multi-dimensional homogenous numeric data type that is "Native"
Python is a great idea! With Pyrex and/or Psyco, Numeric3 (NumArray2 ?)
could be implimented by having only the samallest core in C, and then
rest in Python (or Pyrex)
While the Psyco option is the rosy future of Python, Pyrex is here now,
and maybe adopting it to handle NumArrays well would be easier than
re-writing a bunch of NumArray in C.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Tim C. <tc...@op...> - 2003-02-07 23:07:03
|
On Sat, 2003-02-08 at 09:08, Chris Barker wrote: > While the Psyco option is the rosy future of Python, Pyrex is here now, > and maybe adopting it to handle NumArrays well would be easier than > re-writing a bunch of NumArray in C. Well, Psyco is already immediately useful for many problems on Intel platforms, but I take your point that its real future is as the next-generation VM for Python. However, I agree 100% about the potential for leveraging Pyrex in Numarray. Not just in Numarray, but around it, too. The Numarray team should open serious talks with Greg Ewing about Numarray-enabling Pyrex. And New Zealand is a very nice place to visit (seriously, not joking, even though I am an Australian [reference to trans-Tasman Sea rivalry between Asutralia and New Zealand there]). Tim C |
|
From: Joachim S. <li...@js...> - 2003-02-07 23:57:50
|
* Tim Churches [2003-02-08 00:07]: > However, I agree 100% about the potential for leveraging Pyrex in > Numarray. Not just in Numarray, but around it, too. The Numarray team > should open serious talks with Greg Ewing about Numarray-enabling Pyrex. What is it that needs to be "enabled"? Pyrex handles Numeric (see Pyrex FAQ), why should it not handle Numarray? AFAIK Pyrex contains no code to specifically support Numeric, and it should therefore be straightforward to use it with Numarray as well. Only drawback is currently lack of support for e.g. slicing operations in Pyrex. Cheers, Joachim |
|
From: Tim C. <tc...@op...> - 2003-02-08 00:24:40
|
On Sat, 2003-02-08 at 10:57, Joachim Saul wrote: > * Tim Churches [2003-02-08 00:07]: > > However, I agree 100% about the potential for leveraging Pyrex in > > Numarray. Not just in Numarray, but around it, too. The Numarray team > > should open serious talks with Greg Ewing about Numarray-enabling Pyrex. > > What is it that needs to be "enabled"? Pyrex handles Numeric (see > Pyrex FAQ), why should it not handle Numarray? AFAIK Pyrex > contains no code to specifically support Numeric, and it should > therefore be straightforward to use it with Numarray as well. Hmmm, maybe re-implementing MA in Pyrex is possible right now. Double hmmm.... Tim C |
|
From: Joachim S. <li...@js...> - 2003-02-08 10:55:13
|
* Tim Churches [2003-02-08 01:25]:
> On Sat, 2003-02-08 at 10:57, Joachim Saul wrote:
> > * Tim Churches [2003-02-08 00:07]:
> > > However, I agree 100% about the potential for leveraging Pyrex in
> > > Numarray. Not just in Numarray, but around it, too. The Numarray team
> > > should open serious talks with Greg Ewing about Numarray-enabling Pyrex.
> >
> > What is it that needs to be "enabled"? Pyrex handles Numeric (see
> > Pyrex FAQ), why should it not handle Numarray? AFAIK Pyrex
> > contains no code to specifically support Numeric, and it should
> > therefore be straightforward to use it with Numarray as well.
>
> Hmmm, maybe re-implementing MA in Pyrex is possible right now. Double
> hmmm....
Please check out the Pyrex doc. It's actually very easy right now,
*if* you can live without "sequence operators" such as slicing,
list comprehensions... but this is going to be supported, again
according to the doc.
Here is a exerpt from an extension module that I have built using
Pyrex and Numeric, following the instructions in the Pyrex-FAQ:
cdef extern int decomp(int, double*, double*, double, double, double)
cdef extern from "Numeric/arrayobject.h":
struct PyArray_Descr:
int type_num, elsize
char type
ctypedef class PyArrayObject [type PyArray_Type]:
cdef char *data
cdef int nd
cdef int *dimensions,*strides
cdef PyArray_Descr *descr
object PyArray_FromDims(int, int*, int)
void import_array()
def _decomp(PyArrayObject z_arr, PyArrayObject r_arr,
double p, double vs, double sigma):
cdef double *z, *r
cdef int n
n = z_arr.dimensions[0]
z, r = <double*> z_arr.data, <double*> r_arr.data
decomp(n, z, r, p, vs, sigma)
This is rather crude code that doesn't check for the type of the
arrays nor their dimension, but it does what I want right now and
if I find the time I'll certainly make it more general. Those
checks are actually performed in yet another Python layer.
As you can see, the above looks like "strongly typed" Python. From
a C-programmers perspective, if find this is extremely cool. If one
leaves the type out, then the argument can be any Python object.
What I like about Pyrex is that you can mix Python and C calls at
your convenience. For example, I may call (C-like)
arr = PyArray_FromDims(1, &n, PyArray_DOUBLE)
but could have also used a corresponding Python construct like
from Numeric import zeros
arr = zeros(n, 'd')
I expect the latter to be slower (not tested), but one can take
Python code "as is" and "compile" it using Pyrex. This already
increases performance and one can then conveniently replace as
much Python code as needed with the corresponding C functions,
which (presumably) will again speed up the code significantly.
The bottle necks are finally moved to external C files and
treated like a C library.
Cheers,
Joachim
|
|
From: Perry G. <pe...@st...> - 2003-02-08 21:51:40
|
Both psyco and pyrex have some great aspects. But I think it is worth a little reflection on what can and can't be expected of them. I'm basically ignorant of both; I know a little about them, but haven't used them. if anything I say is wrong, please correct me. I'm going to make some comments based on inferred characteristics of them that could well be wrong. Psyco is very cool and seems the answer to many dreams. But consider the cost. From what I can infer, it obtains its performance enhancements at least in part by constructing machine code on the fly from the Python code. In other words it is performing aspects of running on particular processors that is usually relegated to C compilers by Python. I'd guess that the price is the far greater difficulty of maintaining such capability across many processor types. It also likely increases the complexity of the implementation of Python, perhaps making it much harder to change and enhance. Even without it handling things that are needed for array processing, how likely is it that it will be accepted as the standard implementation for Python for these reasons alone. I also am inclined to believe that adding the support for array processing to a psyco implementation is a significant undertaking. There are at least two issues that would have to be addressed: handling all the numeric types and exception handling behavior. Then there are aspects important to us that include handling byteswapped or non-aligned data. Having the Python VM handle the efficiency aspects of arrays simplifies aspects of their implementation as compared to the current implementations of Numeric and numarray it doesn't eliminate the need to replicate much of it. Having to deal with the implemenation for several different processors is likely to outweigh any savings in the implementation. But maybe I misjudge. Pyrex's goals are more realistic I believe. But unless I'm mistaken, pyrex cannot be a solution to the problems that Numeric and numarray solve. Writing a something for pyrex means committing to certain types. It's great for writing something that you would have written as a C extension, but it doesn't really solve the problem of implementing Ufuncs that can handle many different types of arrays, and particularly combinations of types. But perhaps I misunderstand here as well. It certainly would be nice if it could handle some of the aspects of the Numeric/numarray API automatically. So I doubt that either really is a solution for masked arrays in general. Perry |
|
From: John E. <ja...@zh...> - 2003-02-08 23:10:14
|
Perry Greenfield wrote: > Both psyco and pyrex have some great aspects. But I think > it is worth a little reflection on what can and can't be > expected of them. I'm basically ignorant of both; I know > a little about them, but haven't used them. if anything I > say is wrong, please correct me. I'm going to make some > comments based on inferred characteristics of them that > could well be wrong. I'd like to suggest to anyone interested in these ideas that they take a look a the pypython/minimal-python mailing list: http://codespeak.net/mailman/listinfo/pypy-dev > Psyco is very cool and seems the answer to many dreams. > But consider the cost. From what I can infer, it obtains > its performance enhancements at least in part by constructing > machine code on the fly from the Python code. In other > words it is performing aspects of running on particular > processors that is usually relegated to C compilers by > Python. > > I'd guess that the price is the far greater difficulty of > maintaining such capability across many processor types. > It also likely increases the complexity of the implementation > of Python, perhaps making it much harder to change and > enhance. Even without it handling things that are needed > for array processing, how likely is it that it will be > accepted as the standard implementation for Python for > these reasons alone. The hope is that quite the opposite of just about every one of these points will be true. That once Python is reimplemented in Python, with psycho as a backend jit-like compiler, it will decrease the complexity of the implementation. Making it much easier to change and enhance. I tend to be quite optimistic about the potential for pypython and psycho. I think the added work of the platform dependent psycho modules will be offset by the rest of the system being written in Python. -- John Eikenberry [ja...@zh... - http://zhar.net] ______________________________________________________________ "Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away." -- Antoine de Saint-Exupery |
|
From: Tim H. <tim...@ie...> - 2003-02-07 23:09:00
|
Chris Barker wrote:
>oops, sorry about the blank message.
>
>Paul F Dubois wrote:
>
>
>>{ CC to GvR just to show why I'm +1 on the if-PEP. I liked this in another
>>
>>
>
>What the heck is the if-PEP ?
>
>
Pep 308. It's stirring up a bit of a ruckos on CLP as we speak.
>>Perhaps knowlegeable persons could comment on the feasibility of coding MA
>>(masked arrays) in straight Python and then using Psyco on it?
>>
>>
>
>Is there confusion between Psyco and Pyrex? Psyco runs regular old
>Python bytecode, and individually compiles little pieces of it as needed
>into machine code. AS I understand it, this should make loops where the
>inner part is a pretty simple operation very fast.
>
>However, Psyco is pretty new, and I have no idea how robust and stable,
>but certainly not cross platform. As it generates machine code, it needs
>to be carefully ported to each hardware platform, and it currently only
>works on x86.
>
>
Psyco seems fairly stable these days. However it's one of those things
that probably needs to get a larger cabal of users to shake the bugs out
of it. I still only use it to play around with because all things that I
need speed from I end up doing in Numeric anyway.
>Pyrex, on the other hand, is a "Python-like" language that is tranlated
>into C, and then the C is compiled. It generates pretty darn platform
>independent, so it should be able to be used on all platforms.
>
>
>In regard to your question about MA (and any ther similar project): I
>think Psyco has the potential to be the next-generation Python VM, which
>will have much higher performance, and therefore greatly reduce the need
>to write extensions for the sake of performance. I supsect that it could
>do its best with large, multi-dimensional arrays of numbers if there is
>a Python native object of such a type. Psycho, however is not ready for
>general use on all platforms, so in the forseeable future, there is a
>need for other ways to get decent performance. My suggestion follows:
>
>
>
>>It could have been written a lot simpler if performance didn't dictate
>>trying to leverage off Numeric. In straight Python one can imagine an add,
>>for example, that was roughly:
>> for k in 0<= k < len(a.data):
>> result.mask[k] = a.mask[k] or b.mask[k]
>> result.data[k] = a.data[k] if result.mask[k] else a.data[k] +
>>b.data[k]
>>
>>
>
>This looks like it could be written in Pyrex. If Pyrex were suitably
>NumArray aware, then it could work great.
>
>What this boils down to, in both the Pyrex and Psyco options, is that
>having a multi-dimensional homogenous numeric data type that is "Native"
>Python is a great idea! With Pyrex and/or Psyco, Numeric3 (NumArray2 ?)
>could be implimented by having only the samallest core in C, and then
>rest in Python (or Pyrex)
>
>
For Psyco at least you don't need a multidimensional type. You can get
good results with flat array, in particular array.array. The number I
posted earlier showed comparable performance for Numeric and a
multidimensional array type written all in python and psycoized.
And since I suspect that I'm the mysterious person who's name Paul
couldn't remember, let me say I suspect the MA would be faster in
psycoized python than what your doing now as long as a.data was an
instance of array.array. However, there are at least three problems.
Psyco doesn't fully support the floating point type('f') right now
(although it does support most of the various integral types in
addition to 'd'). I assume that these masked arrays are
multidimensional, so someone would have to build the basic
multidimensional machinery around array.array to make them work. I have
a good start on this, but I'm not sure when I'm going to have time to
work on this more. The biggy though is that psyco only works on x86
machines. What we really need to do is to clone Armin.
>While the Psyco option is the rosy future of Python, Pyrex is here now,
>and maybe adopting it to handle NumArrays well would be easier than
>re-writing a bunch of NumArray in C.
>
>
This sounds like you're conflating two different issues. The first issue
is that Numarray is relatively slow for small arrays. Pyrex may indeed
be an easier way to attack this although I wouldn't know, I've only
looked at it not tried to use it. However, I think that this is
something that can and should wait. Once use cases of numarray being
_too_ slow for small arrays start piling up, then it will be time to
attack the overhead. Premature optimization is the root of all evil and
all that.
The second issue is how to deal with code that does not vectorize well.
Here Pyrex again might help if it were made Numarray aware. However,
isn't this what scipy.weave already does? Again, I haven't used weave,
but as I understand it, it's another python-c bridge, but one that's
more geared toward numerics stuff.
-tim
|
|
From: Paul F D. <pa...@pf...> - 2003-02-08 00:28:56
|
Just to confirm the obvious, I don't know the difference between Psyco and
Pyrex and if I ever did it is Friday night and I've lost it. Any two words
that share two letters look the same to me right now.
> -----Original Message-----
> From: num...@li...
> [mailto:num...@li...] On
> Behalf Of Tim Hochberg
> Sent: Friday, February 07, 2003 3:09 PM
> To: Chris Barker
> Cc: Paul F Dubois; num...@li...
> Subject: Re: [Numpy-discussion] Psyco MA?
>
>
> Chris Barker wrote:
>
> >oops, sorry about the blank message.
> >
> >Paul F Dubois wrote:
> >
> >
> >>{ CC to GvR just to show why I'm +1 on the if-PEP. I liked this in
> >>another
> >>
> >>
> >
> >What the heck is the if-PEP ?
> >
> >
>
> Pep 308. It's stirring up a bit of a ruckos on CLP as we speak.
>
> >>Perhaps knowlegeable persons could comment on the feasibility of
> >>coding MA (masked arrays) in straight Python and then using
> Psyco on
> >>it?
> >>
> >>
> >
> >Is there confusion between Psyco and Pyrex? Psyco runs regular old
> >Python bytecode, and individually compiles little pieces of it as
> >needed into machine code. AS I understand it, this should make loops
> >where the inner part is a pretty simple operation very fast.
> >
> >However, Psyco is pretty new, and I have no idea how robust
> and stable,
> >but certainly not cross platform. As it generates machine code, it
> >needs to be carefully ported to each hardware platform, and it
> >currently only works on x86.
> >
> >
> Psyco seems fairly stable these days. However it's one of
> those things
> that probably needs to get a larger cabal of users to shake
> the bugs out
> of it. I still only use it to play around with because all
> things that I
> need speed from I end up doing in Numeric anyway.
>
> >Pyrex, on the other hand, is a "Python-like" language that
> is tranlated
> >into C, and then the C is compiled. It generates pretty darn
> platform
> >independent, so it should be able to be used on all platforms.
> >
> >
> >In regard to your question about MA (and any ther similar
> project): I
> >think Psyco has the potential to be the next-generation Python VM,
> >which will have much higher performance, and therefore
> greatly reduce
> >the need to write extensions for the sake of performance. I supsect
> >that it could do its best with large, multi-dimensional arrays of
> >numbers if there is a Python native object of such a type. Psycho,
> >however is not ready for general use on all platforms, so in the
> >forseeable future, there is a need for other ways to get decent
> >performance. My suggestion follows:
> >
> >
> >
> >>It could have been written a lot simpler if performance
> didn't dictate
> >>trying to leverage off Numeric. In straight Python one can
> imagine an
> >>add, for example, that was roughly:
> >> for k in 0<= k < len(a.data):
> >> result.mask[k] = a.mask[k] or b.mask[k]
> >> result.data[k] = a.data[k] if result.mask[k] else
> a.data[k] +
> >>b.data[k]
> >>
> >>
> >
> >This looks like it could be written in Pyrex. If Pyrex were suitably
> >NumArray aware, then it could work great.
> >
> >What this boils down to, in both the Pyrex and Psyco
> options, is that
> >having a multi-dimensional homogenous numeric data type that is
> >"Native" Python is a great idea! With Pyrex and/or Psyco, Numeric3
> >(NumArray2 ?) could be implimented by having only the
> samallest core in
> >C, and then rest in Python (or Pyrex)
> >
> >
> For Psyco at least you don't need a multidimensional type.
> You can get
> good results with flat array, in particular array.array. The number I
> posted earlier showed comparable performance for Numeric and a
> multidimensional array type written all in python and psycoized.
>
> And since I suspect that I'm the mysterious person who's name Paul
> couldn't remember, let me say I suspect the MA would be faster in
> psycoized python than what your doing now as long as a.data was an
> instance of array.array. However, there are at least three problems.
> Psyco doesn't fully support the floating point type('f') right now
> (although it does support most of the various integral types in
> addition to 'd'). I assume that these masked arrays are
> multidimensional, so someone would have to build the basic
> multidimensional machinery around array.array to make them
> work. I have
> a good start on this, but I'm not sure when I'm going to have time to
> work on this more. The biggy though is that psyco only works on x86
> machines. What we really need to do is to clone Armin.
>
> >While the Psyco option is the rosy future of Python, Pyrex
> is here now,
> >and maybe adopting it to handle NumArrays well would be easier than
> >re-writing a bunch of NumArray in C.
> >
> >
> This sounds like you're conflating two different issues. The
> first issue
> is that Numarray is relatively slow for small arrays. Pyrex
> may indeed
> be an easier way to attack this although I wouldn't know, I've only
> looked at it not tried to use it. However, I think that this is
> something that can and should wait. Once use cases of numarray being
> _too_ slow for small arrays start piling up, then it will be time to
> attack the overhead. Premature optimization is the root of
> all evil and
> all that.
>
> The second issue is how to deal with code that does not
> vectorize well.
> Here Pyrex again might help if it were made Numarray aware. However,
> isn't this what scipy.weave already does? Again, I haven't
> used weave,
> but as I understand it, it's another python-c bridge, but one that's
> more geared toward numerics stuff.
>
>
> -tim
>
>
>
>
>
>
>
>
>
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld = Something
> 2 See! http://www.vasoftware.com
> _______________________________________________
> Numpy-discussion mailing list Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>
|
|
From: Chris B. <Chr...@no...> - 2003-02-08 01:09:52
|
Tim Hochberg wrote:
> Psyco seems fairly stable these days. However it's one of those things
> that probably needs to get a larger cabal of users to shake the bugs out
> of it. I still only use it to play around with because all things that I
> need speed from I end up doing in Numeric anyway.
Hmmm. It always just seemed too bleeding edge for me to want to drop it
in inplace of my current Python, but maybe I should try...
> For Psyco at least you don't need a multidimensional type. You can get
> good results with flat array, in particular array.array. The number I
> posted earlier showed comparable performance for Numeric and a
> multidimensional array type written all in python and psycoized.
What about non-contiguous arrays? Also, you pointed out yourself that
you are still looking at a factor of two slowdown, it would be nice to
get rid of that.
> >While the Psyco option is the rosy future of Python, Pyrex is here now,
> >and maybe adopting it to handle NumArrays well would be easier than
> >re-writing a bunch of NumArray in C.
> >
> This sounds like you're conflating two different issues. The first issue
> is that Numarray is relatively slow for small arrays. Pyrex may indeed
> be an easier way to attack this although I wouldn't know, I've only
> looked at it not tried to use it. However, I think that this is
> something that can and should wait. Once use cases of numarray being
> _too_ slow for small arrays start piling up, then it will be time to
> attack the overhead. Premature optimization is the root of all evil and
> all that.
Quite true. I know I have a lot of use cases where I use a LOT of small
arrays. That doesn't mean that performace is a huge problem, we'll see.
I'm talking about other things as well, however. There are a lot of
functions in the current Numeric that are written in a combination of
Python and C. Mostly they are written using the lower level Numeric
functions. This includes concatenate, chop, etc. etc. While speeding up
any individual one of those won't make much difference, speeding them
all up might. If it were much easier to get C-speed functions like this,
we'd have a higher performance package all around.
I've personally re-written byteswap() and chop(). In this case, not to
get them faster, but to get them to use less memory. It would be great
if we could do them all.
> The second issue is how to deal with code that does not vectorize well.
> Here Pyrex again might help if it were made Numarray aware. However,
> isn't this what scipy.weave already does? Again, I haven't used weave,
> but as I understand it, it's another python-c bridge, but one that's
> more geared toward numerics stuff.
Weave is another project that's on my list to check out, so I don't know
why one would choose one over the other.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Tim H. <tim...@ie...> - 2003-02-10 16:52:07
|
Chris Barker wrote:
>Tim Hochberg wrote:
>
>
>>Psyco seems fairly stable these days. However it's one of those things
>>that probably needs to get a larger cabal of users to shake the bugs out
>>of it. I still only use it to play around with because all things that I
>>need speed from I end up doing in Numeric anyway.
>>
>>
>
>Hmmm. It always just seemed too bleeding edge for me to want to drop it
>in inplace of my current Python, but maybe I should try...
>
>
I think Psyco was a reworked interpreter at some point, but it isn't any
longer. Now it's just an extension module. You typically use it like this:
def some_function_that_needs_to_be_fast(...):
....
psyco.bind(some_function_that_needs_to_be_fast)
Of course, it's still possible to bomb the interpreter with Psyco and
it's a huge memory hog if you bind a lot of functions. On the other hand
in the course of playing with psymeric I found one way to crash the
interpreter with Psyco, one way with Numeric, and one way to cause
Numarray to fail, although this did not crash the interpreter. So if I
was keeping a tally of evil bugs, they'd all be tied right now....
>>For Psyco at least you don't need a multidimensional type. You can get
>>good results with flat array, in particular array.array. The number I
>>posted earlier showed comparable performance for Numeric and a
>>multidimensional array type written all in python and psycoized.
>>
>>
>
>What about non-contiguous arrays? Also, you pointed out yourself that
>you are still looking at a factor of two slowdown, it would be nice to
>get rid of that.
>
>
Non contiguous arrays are easy to build on top of contiguous arrays,
psymeric works with noncontiguous arrays now. If you'd like, I can send
you some code.
The factor of two slowdown is an issue. A bigger issue is that only x86
platforms are supported. Also there is not support for things like
byteswapped and nonalligned arrays. There also might be problems getting
the exception handling right. If this approach were to be done "right"
for heavy duty number cruncher types, it would require a more capable,
c-based, core buffer object, with most other things written in python
and psycoized. This begins to sounds a lot like what you would get if
you put a lot of psyco.bind calls into the python parts of Numarray now.
On the other hand, it's possible some interesting stuff will come out of
the PyPy project that will make this thing possible in pure Python. I'm
watching that project wit interest.
I did some more tuning of the Psymeric code to reduce overhead and this
is what the speed situation is now. This is complicated to compare,
since the relative speeds depend on both the array type and shaps but
one can get a general feel for things by looking at two things: the
overhead, that is the time it takes to operate on very small arrays, and
the asymptotic time/element for large arrays. These numbers differ
substantially for contiguous and noncontiguous arrays but there relative
values are fairly constant across types. That gives four numbers:
Overhead (c) Overhead (nc)
TimePerElement (c) TimePerElement (nc)
NumPy 10 us 10
us 85 ps 95 ps
NumArray 200 us 530 us
45 ps 135 ps
Psymeric 50 us 65
us 80 ps 80 ps
The times shown above are for Float64s and are pretty approximate, and
they happen to be a particularly favorable array shape for Psymeric. I
have seen pymeric as much as 50% slower than NumPy for large arrays of
certain shapes.
The overhead for NumArray is suprisingly large. After doing this
experiment I'm certainly more sympathetic to Konrad wanting less
overhead for NumArray before he adopts it.
-tim
|
|
From: Perry G. <pe...@st...> - 2003-02-11 20:15:22
|
Tim Hochberg writes: > Overhead (c) Overhead (nc) > TimePerElement (c) TimePerElement (nc) > NumPy 10 us 10 > us 85 ps 95 ps > NumArray 200 us 530 us > 45 ps 135 ps > Psymeric 50 us 65 > us 80 ps 80 ps > > > The times shown above are for Float64s and are pretty approximate, and > they happen to be a particularly favorable array shape for Psymeric. I > have seen pymeric as much as 50% slower than NumPy for large arrays of > certain shapes. > > The overhead for NumArray is surprisingly large. After doing this > experiment I'm certainly more sympathetic to Konrad wanting less > overhead for NumArray before he adopts it. > Wow! Do you really mean picoseconds? I never suspected that either Numeric or numarray were that fast. ;-) Anyway, this issue is timely [Err...]. As it turns out we started looking at ways of improving small array performance a couple weeks ago and are coming closer to trying out an approach that should reduce the overhead significantly. But I have some questions about your benchmarks. Could you show me the code that is used to generate the above timings? In particular I'm interested in the kinds of arrays that are being operated on. It turns out that that the numarray overhead depends on more than just contiguity and it isn't obvious to me which case you are testing. For example, Todd's benchmarks indicate that numarray's overhead is about a factor of 5 larger than numpy when the input arrays are contiguous and of the same type. On the other hand, if the array is not contiguous or requires a type conversion, the overhead is much larger. (Also, these cases require blocking loops over large arrays; we have done nothing yet to optimize the block size or the speed of that loop.) If you are doing the benchmark on contiguous, same type arrays, I'd like to get a copy of the benchmark program to try to see where the disagreement arises. The very preliminary indications are that we should be able to make numarray overheads approximately 3 times higher for all ufunc cases. That's still slower, but not by a factor of 20 as shown above. How much work it would take to reduce it further is unclear (the main bottleneck at that point appears to be how long it takes to create new output arrays) We are still mainly in the analysis and design phase of how to improve performance for small arrays and block looping. We believe that this first step will not require moving very much of the existing Python code into C (but some will be). Hopefully we will have some working code in a couple weeks. Thanks, Perry |
|
From: Tim H. <tim...@ie...> - 2003-02-11 21:04:09
|
Perry Greenfield wrote:
>Tim Hochberg writes:
>
>
>> Overhead (c) Overhead (nc)
>>TimePerElement (c) TimePerElement (nc)
>>NumPy 10 us 10
>>us 85 ps 95 ps
>>NumArray 200 us 530 us
>>45 ps 135 ps
>>Psymeric 50 us 65
>>us 80 ps 80 ps
>>
>>
>>The times shown above are for Float64s and are pretty approximate, and
>>they happen to be a particularly favorable array shape for Psymeric. I
>>have seen pymeric as much as 50% slower than NumPy for large arrays of
>>certain shapes.
>>
>>The overhead for NumArray is surprisingly large. After doing this
>>experiment I'm certainly more sympathetic to Konrad wanting less
>>overhead for NumArray before he adopts it.
>>
>>
>>
>Wow! Do you really mean picoseconds? I never suspected that
>either Numeric or numarray were that fast. ;-)
>
>
My bad, I meant ns. What's a little factor of 10^3 among friends.
>Anyway, this issue is timely [Err...]. As it turns out we started
>
>
>looking at ways of improving small array performance a couple weeks
>ago and are coming closer to trying out an approach that should
>reduce the overhead significantly.
>
>But I have some questions about your benchmarks. Could you show me
>the code that is used to generate the above timings? In particular
>I'm interested in the kinds of arrays that are being operated on.
>It turns out that that the numarray overhead depends on more than
>just contiguity and it isn't obvious to me which case you are testing.
>
>
I'll send you psymeric, including all the tests by private email to
avoid cluttering up the list. (Don't worry, it's not huge -- only 750
lines of Python at this point). You can let me know if you find any
horrible issues with it.
>For example, Todd's benchmarks indicate that numarray's overhead is
>about a factor of 5 larger than numpy when the input arrays are
>contiguous and of the same type. On the other hand, if the array
>is not contiguous or requires a type conversion, the overhead is
>much larger. (Also, these cases require blocking loops over large
>arrays; we have done nothing yet to optimize the block size or
>the speed of that loop.) If you are doing the benchmark on
>contiguous, same type arrays, I'd like to get a copy of the benchmark
>program to try to see where the disagreement arises.
>
>
Basically, I'm operating on two, random contiguous, 3x3, Float64
arrays.In the noncontiguous case the arrays are indexed using [::2,::2]
and [1::2,::2] so these arrays are 2x2 and 1x2. Hmmm, that wasn't
intentional, I'm measuring axis stretching as well. However using
[::2.::2] for both axes doesn't change things a whole lot. The core
timing part looks like this:
t0 = clock()
if op == '+': c = a + b
elif op == '-': c = a - b
elif op == '*': c = a * b
elif op == '/': c = a / b
elif op == '==': c = a==b
else:
raise ValueError("unknown op %s" % op)
t1 = clock()
This is done N times, the first M values are thrown away and the
remaining values are averaged. Currently N is 3 and M is 1, so not a lot
averaging is taking place.
>The very preliminary indications are that we should be able to make
>numarray overheads approximately 3 times higher for all ufunc cases.
>That's still slower, but not by a factor of 20 as shown above. How
>much work it would take to reduce it further is unclear (the main
>bottleneck at that point appears to be how long it takes to create
>new output arrays)
>
>
That's good. I think it's important to get people like Konrad on board
and that will require dropping the overhead.
>We are still mainly in the analysis and design phase of how to
>improve performance for small arrays and block looping. We believe
>that this first step will not require moving very much of the
>existing Python code into C (but some will be). Hopefully we
>will have some working code in a couple weeks.
>
I hope it goes well.
-tim
|
|
From: Francesc A. <fa...@op...> - 2003-02-12 13:03:17
|
Hi, Some days ago, I've also done some benchmarks on this issue, and I think that could be good to share my results.=20 I'm basically reproducing the figures of Tim, although that with a difference still bigger in favour of Numeric for small matrix (2x2). The benchmarks are made with a combination of Python and Pyrex (in order to t= est also some functions in Numeric and numarray C API). The figures I'm getting are, roughly: Matrix multiplication: In Python: matrixmultiply (double(2,2) x double(2,)) in Numeric: 70 us matrixmultiply (double(2,2) x double(2,)) in numarray: 4800 us In Pyrex: numarray multiply in Pyrex, using NA_InputArray: 620 us numarray multiply in Pyrex, using PyObject_AsWriteBuffer: 146 us zeros: In Python: double(2,) in Numeric: 58 us double(2,) in numarray: 3100 us In Pyrex (using PyArray_FromDims): double(2,) with Numeric: 26 us double(2,) with numarray: 730 us As, you can see, in pure Python, numarray has a factor of 50 (for zeros) = and up to 70 (for matrix multiply) times more overhead than Numeric. Increasi= ng the matrix to a 200x20 the overhead difference falls down to factor of 16 (for matrix multiply) and 50 (for zeros) always in favor of Numeric. With Pyrex (i.e. making the C calls), the differences are not so big, but there is still a difference. In particular, when assuming a contiguous matrix and calling PyObject_AsWriteBuffer directly upon the object._data memory buffer, the factor falls down to 2. Increasing the matrix to 200x2= 0, the overhead for zeros (using PyArray_FromDims) is the same for numarray than Numeric (around 700 us), while multiply in Pyrex can't beat the matrixmultiply in Numeric (Numeric is still 2 times faster). Hope that helps. I also can send you my testbeds if you are interested in= =2E --=20 Francesc Alted |
|
From: Francesc A. <fa...@op...> - 2003-02-11 07:23:18
|
A Divendres 07 Febrer 2003 19:33, Chris Barker va escriure:
>
> Is Pyrex aware of Numeric Arrays?
Joachim Saul already answered that, it is.
More exactly, Pyrex is not aware of any special object outside the Python
standard types, but with a bit of cleverness and patience, you can map an=
y
object you want to Pyrex.
The Numeric array object map just happens to be documented in the FAQ, bu=
t I
managed to access numarray objects as well. Here is the recipe:
First, define some enum types and headers:
# Structs and functions from numarray
cdef extern from "numarray/numarray.h":
ctypedef enum NumRequirements:
NUM_CONTIGUOUS
NUM_NOTSWAPPED
NUM_ALIGNED
NUM_WRITABLE
NUM_C_ARRAY
NUM_UNCONVERTED
ctypedef enum NumarrayByteOrder:
NUM_LITTLE_ENDIAN
NUM_BIG_ENDIAN
cdef enum:
UNCONVERTED
C_ARRAY
ctypedef enum NumarrayType:
tAny
tBool=09
tInt8
tUInt8
tInt16
tUInt16
tInt32
tUInt32
tInt64
tUInt64
tFloat32
tFloat64
tComplex32
tComplex64
tObject
tDefault
tLong
=20
# Declaration for the PyArrayObject
=20
struct PyArray_Descr:
int type_num, elsize
char type
=20
ctypedef class PyArrayObject [type PyArray_Type]:
# Compatibility with Numeric
cdef char *data
cdef int nd
cdef int *dimensions, *strides
cdef object base
cdef PyArray_Descr *descr
cdef int flags
# New attributes for numarray objects
cdef object _data # object must meet buffer API */
cdef object _shadows # ill-behaved original array. */
cdef int nstrides # elements in strides array */
cdef long byteoffset # offset into buffer where array data begin=
s */
cdef long bytestride # basic seperation of elements in bytes */
cdef long itemsize # length of 1 element in bytes */
cdef char byteorder # NUM_BIG_ENDIAN, NUM_LITTLE_ENDIAN */
cdef char _aligned # test override flag */
cdef char _contiguous # test override flag */
void import_array()
=20
# The Numeric API requires this function to be called before
# using any Numeric facilities in an extension module.
import_array()
Then, declare the API routines you want to use:
cdef extern from "numarray/libnumarray.h":
PyArrayObject NA_InputArray (object, NumarrayType, int)
PyArrayObject NA_OutputArray (object, NumarrayType, int)
PyArrayObject NA_IoArray (object, NumarrayType, int)
PyArrayObject NA_Empty(int nd, int *d, NumarrayType type)
object PyArray_FromDims(int nd, int *d, NumarrayType type)
define now a couple of maps between C enum types and Python numarrar type
classes:
# Conversion tables from/to classes to the numarray enum types
toenum =3D {numarray.Int8:tInt8, numarray.UInt8:tUInt8,
numarray.Int16:tInt16, numarray.UInt16:tUInt16,
numarray.Int32:tInt32, numarray.UInt32:tUInt32,
numarray.Float32:tFloat32, numarray.Float64:tFloat64,
}
toclass =3D {}
for (key, value) in toenum.items():
toclass[value] =3D key
ok. you are on the way. We can finally define our user funtion; for examp=
le,
I will show here a function to multiply a matrix by a vector (C double
precision):
def multMatVec(object a, object b, object c):
cdef PyArrayObject carra, carrb, carrc
cdef double *da, *db, *dc
cdef int i, j
=20
carra =3D NA_InputArray(a, toenum[a._type], C_ARRAY)
carrb =3D NA_InputArray(b, toenum[b._type], C_ARRAY)
carrc =3D NA_InputArray(c, toenum[c._type], C_ARRAY)
da =3D <double *>carra.data
db =3D <double *>carrb.data
dc =3D <double *>carrc.data
dim1 =3D carra.dimensions[0]
dim2 =3D carra.dimensions[1]
for i from 0<=3D i < dim1:
dc[i] =3D 0.
for j from 0<=3D j < dim2:
dc[i] =3D dc[i] + da[i*dim2+j] * db[j]
=20
return carrc
where NA_InputArray is a high-level numarray API that ensures that the
object retrieved is a well-behaved array, and not mis-aligned, discontigu=
ous
or whatever.=20
Maybe at first glance such a procedure would seem obscure, but it is not.=
I
find it to be quite elegant.
Look at the "for i from 0<=3D i < dim1:" construction. We could have used=
the
more pythonic form: "for i in range(dim1):", but by using the former, the
Pyrex compiler is able to produce a loop in plain C, so achieving C-speed=
on
this piece of code. Of course, you must be aware to not introduce Python
objects inside the loop, or all the potential speed-up improvement will
vanish. But, with a bit of practice, this is easy to avoid.
For me Pyrex is like having Python but with the speed of C. This is why I=
'm
so enthusiastic with it.
>
> I imagine it could use them just fine, using the generic Python sequenc=
e
> get item stuff, but that would be a whole lot lower performance than if
> it understood the Numeric API and could access the data array directly.
> Also, how does it deal with multiple dimension indexing ( array[3,6,2] =
)
> which the standard python sequence types do not support?
In general, you can access sequence objects like in Python (and I've just
checked that extended slicing *is* supported, I don't know why Joachim wa=
s
saying that not; perhaps he was meaning Pyrex C-arrays?), but at Python
speed. So, if you need speed, always use pointers to your data and use a =
bit
of pointer arithmetic to access the element you want (look at the example=
).
Of course, you can also define C arrays if you know the boundaries in
compilation time and let the compiler do the computations to access your
desired element, but you will need first to copy the data from your buffe=
rs
to the C-array, and perhaps this is a bit inconvenient in some situations=
=2E
> As I think about this, I think your suggestion is fabulous. Pyrex (or a
> Pyrex-like) language would be a fabulous way to write code for NumArray=
,
> if it really made use of the NumArray API.
There can be drawbacks, like the one stated by Perry related with how to
construct general Ufuncs that can handle many different combinations of
arrays and types, although I don't understand that very well because Nume=
ric
and numarray crews already achieved to do that in C, so why it cannot be
possible with Pyrex?. Mmm, perhaps there is some pre-processor involved?.
Cheers,
--=20
Francesc Alted
|
|
From: Chris B. <Chr...@no...> - 2003-02-11 18:35:07
|
Francesc Alted wrote:
> First, define some enum types and headers:
Could all this be put into Pyrex? (when NumArray becomes more stable
anyway) It's well beyond me to understand it.
> I will show here a function to multiply a matrix by a vector (C double
> precision):
>
> def multMatVec(object a, object b, object c):
> cdef PyArrayObject carra, carrb, carrc
> cdef double *da, *db, *dc
> cdef int i, j
>
> carra = NA_InputArray(a, toenum[a._type], C_ARRAY)
> carrb = NA_InputArray(b, toenum[b._type], C_ARRAY)
> carrc = NA_InputArray(c, toenum[c._type], C_ARRAY)
> da = <double *>carra.data
> db = <double *>carrb.data
> dc = <double *>carrc.data
> dim1 = carra.dimensions[0]
> dim2 = carra.dimensions[1]
> for i from 0<= i < dim1:
> dc[i] = 0.
> for j from 0<= j < dim2:
> dc[i] = dc[i] + da[i*dim2+j] * db[j]
>
> return carrc
> For me Pyrex is like having Python but with the speed of C. This is why I'm
> so enthusiastic with it.
That actually looks more like C than Python to me. As soon as I am doing
pointer arithmetic, I don't feel like I'm writng Python. Would it be all
that much more code in C?
> speed. So, if you need speed, always use pointers to your data and use a bit
> of pointer arithmetic to access the element you want (look at the example).
Is there really no way to get this to work?
> Of course, you can also define C arrays if you know the boundaries in
> compilation time and let the compiler do the computations to access your
> desired element, but you will need first to copy the data from your buffers
> to the C-array, and perhaps this is a bit inconvenient in some situations.
Couldn't you access the data array of the NumArray directly? I do this
all the time with Numeric.
> Why you are saying that slicing is not supported?. I've checked them (as
> python expressions, of course) and work well. May be you are referring to
> cdef'd c-typed arrays in Pyrex?. I think this should be a dangerous thing
> that can make the pointer arithmetic to slow down because of the additional
> required checks in the slice range.
Well, there would need to be two value checks per slice. That would be
significant for small slices, but not for large ones, I'd love to have
it. It just doesn't feel like Python without slicing, and it doesn't
feel like NumPy without multi-dimensional slicing.
> There can be drawbacks, like the one stated by Perry related with how to
> construct general Ufuncs that can handle many different combinations of
> arrays and types, although I don't understand that very well because Numeric
> and numarray crews already achieved to do that in C, so why it cannot be
> possible with Pyrex?. Mmm, perhaps there is some pre-processor involved?.
I was curious about this comment as well. I have only had success with
writing any of my Numeric based extensions for pre-determined types. If
I had to support additional types (and/or discontiguous and/or rank-N
arrays), I ended up with a whole pile of case and/or if statements. Also
kind of slow and inefficient code.
It seems the only way to do this right is with C++ and templates (eg.
Blitz++), but there are good reasons not to go that route.
Would it really be any harder to use Pyrex than C for this kind of
thing? Also, would it be possible to take a Pyrex type approach and have
it do someting template-like: you wright the generic code in Pyrex, it
generates all the type-specific C code for you.
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
NOAA/OR&R/HAZMAT (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chr...@no...
|
|
From: Francesc A. <fa...@op...> - 2003-02-11 19:33:46
|
A Dimarts 11 Febrer 2003 18:54, Chris Barker va escriure:
> >
> > def multMatVec(object a, object b, object c):
> > cdef PyArrayObject carra, carrb, carrc
> > cdef double *da, *db, *dc
> > cdef int i, j
> >
> > carra =3D NA_InputArray(a, toenum[a._type], C_ARRAY)
> > carrb =3D NA_InputArray(b, toenum[b._type], C_ARRAY)
> > carrc =3D NA_InputArray(c, toenum[c._type], C_ARRAY)
> > da =3D <double *>carra.data
> > db =3D <double *>carrb.data
> > dc =3D <double *>carrc.data
> > dim1 =3D carra.dimensions[0]
> > dim2 =3D carra.dimensions[1]
> > for i from 0<=3D i < dim1:
> > dc[i] =3D 0.
> > for j from 0<=3D j < dim2:
> > dc[i] =3D dc[i] + da[i*dim2+j] * db[j]
> >
> > return carrc
> >
> >
> > For me Pyrex is like having Python but with the speed of C. This is w=
hy
> > I'm so enthusiastic with it.
>
> That actually looks more like C than Python to me. As soon as I am doin=
g
> pointer arithmetic, I don't feel like I'm writng Python. Would it be al=
l
> that much more code in C?
Doing that in C implies writing the "glue" code. In the past example,
multMatVec is a function *directly* accessible in Python, without any
additional declaration.
Moreover, you can do in Pyrex the same things you do in python, so you co=
uld
have written the last piece of code as:
def multMatVec(object a, object b, object c):
for i in range(a.shape[0]):
c[i] =3D 0.
for j in range(a.shape[1]):
dc[i] =3D dc[i] + da[i][j] * db[j]
return c
but, of course, you get only Python speed.
So, the moral is that C speed is only accessible in Pyrex if you use C li=
ke
types and constructions, it just don't come for free. I just find this wa=
y
to code to be more elegant than using Swig, or other approaches. But I'm
most probably biased because Pyrex is my first (and unique) serious tool =
for
doing Python extensions.
>
> > speed. So, if you need speed, always use pointers to your data and us=
e a
> > bit of pointer arithmetic to access the element you want (look at the
> > example).
>
> Is there really no way to get this to work?
>
> > Of course, you can also define C arrays if you know the boundaries in
> > compilation time and let the compiler do the computations to access y=
our
> > desired element, but you will need first to copy the data from your
> > buffers to the C-array, and perhaps this is a bit inconvenient in som=
e
> > situations.
>
> Couldn't you access the data array of the NumArray directly? I do this
> all the time with Numeric.
Yeah, you can, and both examples shown here (in Numeric and numarray), yo=
u
are accessing directly the array data buffer, with no copies (whenever yo=
ur
original array is well-.behaved, of course).
>
> > Why you are saying that slicing is not supported?. I've checked them =
(as
> > python expressions, of course) and work well. May be you are referrin=
g to
> > cdef'd c-typed arrays in Pyrex?. I think this should be a dangerous t=
hing
> > that can make the pointer arithmetic to slow down because of the
> > additional required checks in the slice range.
>
> Well, there would need to be two value checks per slice. That would be
> significant for small slices, but not for large ones, I'd love to have
> it. It just doesn't feel like Python without slicing, and it doesn't
> feel like NumPy without multi-dimensional slicing.
>
Again, right now, you can use slicing in Pyrex if you are dealing with
Python objects, but from the moment you access to the lower-level
Numeric/numarray buffer and assign to a Pyrex C-pointer, you can't do tha=
t
anymore. That's the price to pay for speed.
About implementing slicing in Pyrex C-pointer arithmetics, well, it can b=
e
worth to ask Greg Ewing, the Pyrex author. I'll send him this particular
question and forward his answer (if any) to the list.
> > There can be drawbacks, like the one stated by Perry related with how=
to
> > construct general Ufuncs that can handle many different combinations =
of
> > arrays and types, although I don't understand that very well because
> > Numeric and numarray crews already achieved to do that in C, so why i=
t
> > cannot be possible with Pyrex?. Mmm, perhaps there is some pre-proces=
sor
> > involved?.
>
> I was curious about this comment as well. I have only had success with
> writing any of my Numeric based extensions for pre-determined types. If
> I had to support additional types (and/or discontiguous and/or rank-N
> arrays), I ended up with a whole pile of case and/or if statements. Als=
o
> kind of slow and inefficient code.
>
> It seems the only way to do this right is with C++ and templates (eg.
> Blitz++), but there are good reasons not to go that route.
>
> Would it really be any harder to use Pyrex than C for this kind of
> thing? Also, would it be possible to take a Pyrex type approach and hav=
e
> it do someting template-like: you wright the generic code in Pyrex, it
> generates all the type-specific C code for you.
Well, this is another good question for Greg. I'll try to ask him, althou=
gh
as I don't have experience on that kind of issues, chances are that my
question might result a complete nonsense :).
Cheers,
--=20
Francesc Alted
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X