numpy-discussion — Discussion list for all users of Numerical Python

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Andrew P. L. Jr. <bs...@ma...>]

On Jun 29, 2004, at 4:25 PM, Chris Barker wrote:

> Todd Miller wrote:
>> Yes.  What I most want to do is a 50000 point drawlines, similar to 
>> what
>> you profiled.  Friends are fine too.
>
> Actually, it was someone else that did the profiling, but here is a 
> sample, about as simple as I could make it.
>
> It draws an N point line and M points. At the moment, it is using 
> Numeric for the points, and numarray for the lines. Numeric is MUCH 
> faster (which is the whole point of this discussion). Otherwise, it 
> takes about the same amount of time to draw the lines as the points.
>
> Another note: if use the tolist() method in the numarray first, it's 
> much faster also:
>
> dc.DrawLines(LinesPoints.tolist())
>

If you are going to move lots of lines and points, I would recommend 
pushing this stuff through PyOpenGL with array objects and vertex 
objects.  Letting OpenGL handle the transformations, clipping, movement 
and iteration in hardware stomps all over even the best written C code. 
  Most UI toolkits have some form of OpenGL widget.

For lots of the code I have written, even Mesa (the open-souce software 
OpenGL renderer) was fast enough, and not having to write all of the 
display transformation code by hand was a huge win even when the speed 
was somewhat lagging.

-a

[Numpy-discussion] bug in numarray.maximum.reduce ?

[Sebastian H. <ha...@ms...>]

Hi,

Is this a bug?:
>>> # (import numarray as na ; 'd' is a 3 dimensional array)
>>> d.type()
Float32
>>> d[80, 136, 122]
80.3997039795
>>> na.maximum.reduce(d[:,136, 122])
85.8426361084
>>> na.maximum.reduce(d) [136, 122]
37.3658103943
>>> 
>>> 
>>> na.maximum.reduce(d,0)[136, 122]
37.3658103943
>>> na.maximum.reduce(d,1)[136, 122]
Traceback (most recent call last):
  File "<input>", line 1, in ?
IndexError: Index out of range

I was using  na.maximum.reduce(d)  to get a "pixelwise" maximum along Z (axis 
0). But as seen above it does not get it right.  I then tried to reproduce 
this with some simple arrays, but here it works just fine:
>>> a = na.arange(4*4*4)
>>> a.shape=(4,4,4)
>>> na.maximum.reduce(a)
[[48 49 50 51]
 [52 53 54 55]
 [56 57 58 59]
 [60 61 62 63]]
>>> a = na.arange(4*4*4).astype(na.Float32)
>>> a.shape=(4,4,4)
>>> na.maximum.reduce(a)
[[ 48.  49.  50.  51.]
 [ 52.  53.  54.  55.]
 [ 56.  57.  58.  59.]
 [ 60.  61.  62.  63.]]
>>> 

Any hint ?

Regards,
Sebastian Haase

Re: [Numpy-discussion] bug ? in Records arrays in numarray

[Sebastian H. <ha...@ms...>]

On Tuesday 29 June 2004 04:49 pm, Todd Miller wrote:
> On Tue, 2004-06-29 at 17:52, Sebastian Haase wrote:
> > OK,
> > I'm still trying to get a handle on these record arrays - because I think
> > they are pretty cool, if I could get them to work...
> > Following the code from yesterday (see that posting below) I discovered
> > this: main.ring4ext[0][0]   is not the same as   main.ring4ext[0,0]
> > is this intended ??
> >
> > >>> main.ring4ext[0][0]
> >
> > (2308, 76, 272, 1088481152.0, 104.18000030517578, 1994.949951171875)
> >
> > >>> main.ring4ext[0,0]
> >
> > (array([2308, 2309]), array([76, 76]), array([272, 269]), array([ 
> > 1.08848115e +09,   1.08848115e+09], type=Float32), array([ 104.18000031, 
> > 104.45999908], type=Float32), array([ 1994.94995117,  1994.95996094],
> > type=Float32))
> >
> > >>> main.ring4ext.shape    # yesterday I had this different !!! (20,1)
> >
> > (20, 2)
> >
> > Any comments are appreciated,
>
> I talked to JC Hsu, the numarray.records author, and he explained that
> we're probably looking at a limitation of numarray.records: it doesn't
> yet handle multi-dimensional arrays of records.  JC indicated he had
> replied to Sebastian,  but for the benefit of everyone else, that's the
> deal.
>
> Regards,
> Todd
>

Thanks,
but it actually seems to work well now, as long as I use myRecArray[i][j] 
instead of myRecArray[i,j].
So, it looks like there is not missing much.
I'll keep your "official statement" in mind when I explore this further...

Thanks again,
Sebastian Haase

Re: [Numpy-discussion] bug ? in Records arrays in numarray

[Todd M. <jm...@st...>]

On Tue, 2004-06-29 at 17:52, Sebastian Haase wrote:
> OK,
> I'm still trying to get a handle on these record arrays - because I think they 
> are pretty cool, if I could get them to work...
> Following the code from yesterday (see that posting below) I discovered this:
> main.ring4ext[0][0]   is not the same as   main.ring4ext[0,0]
> is this intended ??
> 
> >>> main.ring4ext[0][0]
> (2308, 76, 272, 1088481152.0, 104.18000030517578, 1994.949951171875)
> >>> main.ring4ext[0,0]
> (array([2308, 2309]), array([76, 76]), array([272, 269]), array([  1.08848115e
> +09,   1.08848115e+09], type=Float32), array([ 104.18000031,  104.45999908], 
> type=Float32), array([ 1994.94995117,  1994.95996094], type=Float32))
> >>> main.ring4ext.shape    # yesterday I had this different !!! (20,1)
> (20, 2)
> 
> Any comments are appreciated,

I talked to JC Hsu, the numarray.records author, and he explained that
we're probably looking at a limitation of numarray.records: it doesn't
yet handle multi-dimensional arrays of records.  JC indicated he had
replied to Sebastian,  but for the benefit of everyone else, that's the
deal.

Regards,
Todd

> Thanks
> Sebastian
> 
> 
> 
> 
> On Monday 28 June 2004 05:00 pm, Sebastian Haase wrote:
> > Hi,
> > I have two record arrays. I was trying to assign one item from one recarray
> > to
> >
> > the other:
> > >>> omx.zext[0] = main.ring4ext[0,0]
> >
> > Traceback (most recent call last):
> >   File "<input>", line 1, in ?
> >   File "X:/PrWin\numarray\records.py", line 744, in _setitem
> >     self.field(self._names[i])[row] = value.field(self._names[i])
> >   File "C:\Python22\Lib\site-packages\numarray\numarraycore.py", line 619,
> > in __tonumtype__
> > ValueError: Can't use non-rank-0 array as a scalar.
> >
> > >>> `omx.zext[0]`
> >
> > '<numarray.records.Record instance at 0x042AFC78>'
> >
> > >>> `main.ring4ext[0,0]`
> >
> > '<numarray.records.Record instance at 0x042AFC78>'
> >
> > >>> q = omx.zext[0]
> > >>> w = main.ring4ext[0,0]
> > >>> q
> >
> > (2097152107, -595656700, 91141, 1.0634608642000868e+037, -1.14841804241843e
> > +018, 1.2771574333702815e-040)
> >
> > >>> w
> >
> > (array([428]), array([75]), array([124]), array([  1.08846451e+09],
> > type=Float32), array([ 99.25], type=Float32), array([ 1996.82995605],
> > type=Float32))
> >
> > >>> omx.zext._formats
> >
> > ['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
> >
> > >>> main.ring4ext._formats
> >
> > ['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
> >
> >
> > I can't track down why one (q) contains scalars and the other (w) constains
> > arrays (is array([428]) a 'rank-0 array'? ) !
> > This is how I generate them:
> > main.ring4ext = rec.RecArray(main._extHdrRingBuffer, "i4,i4,i4,f4,f4,f4",
> >   shape=(ts,nc), names=("num","min","max","time","mean","darkVal"),
> > aligned=1) omx.zext      = rec.array(formats="i4,i4,i4,f4,f4,f4",
> >  
> > names=("num","min","max","time","mean","darkVal"),shape=tuple(shapeZ),align
> >ed=1 )
> >
> > [[BTW: (shapeZ is a list, if I say: ...,shape=shapeZ    I get
> > NameError, "Illegal shape %s" % `shape`   from "records.py" in line 462  
> > -- usually type(shape) == types.ListType should be OK, right ?)
> > ]]
> >
> >
> > Any ideas?
> >
> > Thanks,
> > Sebastian Haase
> >
> >
> >
> > -------------------------------------------------------
> > This SF.Net email sponsored by Black Hat Briefings & Training.
> > Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
> > digital self defense, top technical experts, no vendor pitches,
> > unmatched networking opportunities. Visit www.blackhat.com
> > _______________________________________________
> > Numpy-discussion mailing list
> > Num...@li...
> > https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
> digital self defense, top technical experts, no vendor pitches, 
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Chris B. <Chr...@no...>]

Todd Miller wrote:
> Yes.  What I most want to do is a 50000 point drawlines, similar to what
> you profiled.  Friends are fine too.

Actually, it was someone else that did the profiling, but here is a 
sample, about as simple as I could make it.

It draws an N point line and M points. At the moment, it is using 
Numeric for the points, and numarray for the lines. Numeric is MUCH 
faster (which is the whole point of this discussion). Otherwise, it 
takes about the same amount of time to draw the lines as the points.

Another note: if use the tolist() method in the numarray first, it's 
much faster also:

dc.DrawLines(LinesPoints.tolist())

Obviously, the tolist method is much faster than wxPython's sequence 
methods, as would be expected.

I'm going to send a note to Robin Dunn about this as well, and see what 
he thinks.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Todd M. <jm...@st...>]

On Tue, 2004-06-29 at 16:19, Tim Hochberg wrote:
> I'd bet a case of beer (or cash equivalent) that one of the main 
> bottlenecks is the path 
> PySequence_GetItem->_ndarray_item->_universalIndexing->_simpleIndexing->_simpleIndexingCore.  

I won't take the bet but if this works out, you get the beer.  If it
doesn't, well, I don't drink anymore anyway.

> The path through _universalIndexing in particular, if I deciphered it 
> correctly,  looks very slow. I don't think it needs to be that way 
> though, _universalIndexing could probably be sped up, but more promising 
> I think _ndarray_item could be made to call _simpleIndexingCore without 
> all that much work. It appears that this would save the creation of 
> several intermediate objects and it also looks like a couple of calls 
> back to python! I'm not familiar with this code though, so I could 
> easily be missing something that makes calling _simpleIndexingCore 
> harder than it looks.

This looks very promising.  I'll take a look tomorrow.

Regards,
Todd

Re: [Numpy-discussion] bug ? in Records arrays in numarray

[Sebastian H. <ha...@ms...>]

OK,
I'm still trying to get a handle on these record arrays - because I think they 
are pretty cool, if I could get them to work...
Following the code from yesterday (see that posting below) I discovered this:
main.ring4ext[0][0]   is not the same as   main.ring4ext[0,0]
is this intended ??

>>> main.ring4ext[0][0]
(2308, 76, 272, 1088481152.0, 104.18000030517578, 1994.949951171875)
>>> main.ring4ext[0,0]
(array([2308, 2309]), array([76, 76]), array([272, 269]), array([  1.08848115e
+09,   1.08848115e+09], type=Float32), array([ 104.18000031,  104.45999908], 
type=Float32), array([ 1994.94995117,  1994.95996094], type=Float32))
>>> main.ring4ext.shape    # yesterday I had this different !!! (20,1)
(20, 2)

Any comments are appreciated,
Thanks
Sebastian




On Monday 28 June 2004 05:00 pm, Sebastian Haase wrote:
> Hi,
> I have two record arrays. I was trying to assign one item from one recarray
> to
>
> the other:
> >>> omx.zext[0] = main.ring4ext[0,0]
>
> Traceback (most recent call last):
>   File "<input>", line 1, in ?
>   File "X:/PrWin\numarray\records.py", line 744, in _setitem
>     self.field(self._names[i])[row] = value.field(self._names[i])
>   File "C:\Python22\Lib\site-packages\numarray\numarraycore.py", line 619,
> in __tonumtype__
> ValueError: Can't use non-rank-0 array as a scalar.
>
> >>> `omx.zext[0]`
>
> '<numarray.records.Record instance at 0x042AFC78>'
>
> >>> `main.ring4ext[0,0]`
>
> '<numarray.records.Record instance at 0x042AFC78>'
>
> >>> q = omx.zext[0]
> >>> w = main.ring4ext[0,0]
> >>> q
>
> (2097152107, -595656700, 91141, 1.0634608642000868e+037, -1.14841804241843e
> +018, 1.2771574333702815e-040)
>
> >>> w
>
> (array([428]), array([75]), array([124]), array([  1.08846451e+09],
> type=Float32), array([ 99.25], type=Float32), array([ 1996.82995605],
> type=Float32))
>
> >>> omx.zext._formats
>
> ['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
>
> >>> main.ring4ext._formats
>
> ['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
>
>
> I can't track down why one (q) contains scalars and the other (w) constains
> arrays (is array([428]) a 'rank-0 array'? ) !
> This is how I generate them:
> main.ring4ext = rec.RecArray(main._extHdrRingBuffer, "i4,i4,i4,f4,f4,f4",
>   shape=(ts,nc), names=("num","min","max","time","mean","darkVal"),
> aligned=1) omx.zext      = rec.array(formats="i4,i4,i4,f4,f4,f4",
>  
> names=("num","min","max","time","mean","darkVal"),shape=tuple(shapeZ),align
>ed=1 )
>
> [[BTW: (shapeZ is a list, if I say: ...,shape=shapeZ    I get
> NameError, "Illegal shape %s" % `shape`   from "records.py" in line 462  
> -- usually type(shape) == types.ListType should be OK, right ?)
> ]]
>
>
> Any ideas?
>
> Thanks,
> Sebastian Haase
>
>
>
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
> digital self defense, top technical experts, no vendor pitches,
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion

Re: [Numpy-discussion] Numarray header PEP

[Todd M. <jm...@st...>]

Awesome!  Thanks Gerard.  This certainly sounds like the way to do it.  
I'll take a closer look tomorrow.

Regards,
Todd

On Tue, 2004-06-29 at 17:32, ger...@gr... wrote:
> On 29 Jun 2004 15:09:43 -0400, Todd Miller wrote
> > On Tue, 2004-06-29 at 13:44, Gerard Vermeulen wrote:
> > > > 
> > > > The PEP is attached.  It is formatted using the docutils package which
> > > > can be used to generate HTML or PDF.  Comments and corrections would be
> > > > appreciated.
> > > > 
> > > PyQwt is a Python extension which can be conditionally compiled against
> > > Numeric and/or numarray (both, one of them or none).
> > 
> > Well that's cool!  I'll have to keep the PyQwt guys in mind as potential
> > first users.
> > 
> > > Your PEP excludes importing of Numeric and numarray in the same C-extension.
> > 
> > This is true but I don't understand your solution so I'll blather on
> > below.
> > 
> > > All you need to do is to hide the macros PyArray_Present(), PyArray_isArray()
> > > and import_array() into a few functions with numarray specific names, so
> > > that the following becomes possible:
> > > 
> > > #include <Numeric/meta.h>
> > > /* defines the functions (no macros)
> > >    int Numeric_Present();
> > >    int Numeric_isArray();
> > >    void import_numeric();
> > >    to hide the Numeric C-API stuff in a small Numeric/meta.c file.
> > > */
> > > #include <numarray/meta.h>
> > > /* defines the functions (no macros)
> > >    int numarray_Present();
> > >    int numarray_isArray();
> > >    void import_numarray();
> > >    to hide the numarray C-API stuff in a small numarray/meta.c file.
> > > */
> > >
> > 
> > I may have come up with the wrong scheme for the Present() and
> > isArray().  With my scheme, they *have* to be macros because the API
> > functions are unsafe:  when numarray or Numeric is not present, the API
> > function table pointers are NULL so calling through them results in
> > either a fatal error or a segfault.
> >
> The macros can be hidden from the module file scope by wrapping them
> in a function (see attached demo)
> > 
> > There is an additional problem that the "same functions" need to be
> > called through different API pointers depending on whether numarray 
> > or Numeric is being supported.  Redefinition of typedefs and enumerations
> > 
> > (or perhaps conditional compilation short-circuited re-definitions) may
> > also present a problem with compiling (or worse, running).
> >
> Tested and works. 
> >
> > I certainly like the idea of supporting both in the same extension
> > module,  but don't see how to get there, other than with separate
> > compilation units.  With seperate .c files,  I'm not aware of a problem
> > other than lots of global symbols.  I haven't demoed that yet so I am
> > interested if someone has made it work.
> > 
> Yes, you cannot mix the numarray API and Numeric API in the same .c
> file, but nevertheless you can hide the macros in small functions so
> that the macros don't pollute.
> 
> Here you have a litte extension module 'pep' which tries to import Numeric
> and numarray. It has a method 'whatsThere' that prints the names of
> the Numerical/numarray extension modules it has imported.
> 
> It has also a method 'whatsThis' that prints if an object is a Numeric
> array, a numarray array or something else:
> 
> Python 2.3.3 (#2, Feb 17 2004, 11:45:40)
> [GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import pep
> >>> pep.whatsThere()
> Numeric!
> Numarray!
> >>> import Numeric
> >>> a = Numeric.arange(10)
> >>> pep.whatsThis(a)
> Numeric array
> >>> import numarray
> >>> a = numarray.arange(10)
> >>> pep.whatsThis(a)
> Numarray array
> >>> a = 10
> >>> pep.whatsThis(a)
> Something else
> >>>
> 
> The MANIFEST reads:
> pepmodule.c
> setup.py
> Numeric/meta.c
> Numeric/meta.h
> numarray/meta.c
> numarray/meta.h
> 
> My initial idea was to add the meta.* files to the Numeric/numarray include
> files, but I did not try yet to solve the problem of the
> PyArray_API PY_ARRAY_UNIQUE_SYMBOL defines in the same way (I am sure
> it can be done.
> 
> Regards -- Gerard
-- 

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Todd M. <jm...@st...>]

On Tue, 2004-06-29 at 17:05, Chris Barker wrote:
> Tim Hochberg wrote:
> > Todd Miller wrote:
> >> Me too.  Can you (or somebody) post the application code which does the
> >> drawlines?  I can definitely instrument the bottleneck C-code,  but I
> >> don't have time to ascend the wxPython learning curve.
> 
> > Let me second that. With a little nudge from Chris Barker, I managed to 
> > get wxPython to compile here and I have some changes that may speed up 
> > drawlines, but it'd probably be best if we were all using the same 
> > benchmark.
> 
> Well, if I can't code, at least I can nudge!
> 
> Perhaps I can also help get Todd what he's asking for, except that I'm 
> not sure what you mean by "application code". Do you mean a small 
> wxPython application that uses DC.DrawLines (and friends)? 

Yes.  What I most want to do is a 50000 point drawlines, similar to what
you profiled.  Friends are fine too.

> If so, yes, I 
> can do that. What version of wxPython should I target?
> CVS head?
> 2.5.1? (The latest "released" version)

I'd prefer 2.5.1 unless Tim says otherwise.

> there are some slight incompatibilities between versions, particularly 
> with DC calls, unfortunately.

I'm hoping this won't affect the profile.

Regards,
Todd

Re: [Numpy-discussion] Numarray header PEP

[<ger...@gr...>]

On 29 Jun 2004 15:09:43 -0400, Todd Miller wrote
> On Tue, 2004-06-29 at 13:44, Gerard Vermeulen wrote:
> > > 
> > > The PEP is attached.  It is formatted using the docutils package which
> > > can be used to generate HTML or PDF.  Comments and corrections would be
> > > appreciated.
> > > 
> > PyQwt is a Python extension which can be conditionally compiled against
> > Numeric and/or numarray (both, one of them or none).
> 
> Well that's cool!  I'll have to keep the PyQwt guys in mind as potential
> first users.
> 
> > Your PEP excludes importing of Numeric and numarray in the same C-extension.
> 
> This is true but I don't understand your solution so I'll blather on
> below.
> 
> > All you need to do is to hide the macros PyArray_Present(), PyArray_isArray()
> > and import_array() into a few functions with numarray specific names, so
> > that the following becomes possible:
> > 
> > #include <Numeric/meta.h>
> > /* defines the functions (no macros)
> >    int Numeric_Present();
> >    int Numeric_isArray();
> >    void import_numeric();
> >    to hide the Numeric C-API stuff in a small Numeric/meta.c file.
> > */
> > #include <numarray/meta.h>
> > /* defines the functions (no macros)
> >    int numarray_Present();
> >    int numarray_isArray();
> >    void import_numarray();
> >    to hide the numarray C-API stuff in a small numarray/meta.c file.
> > */
> >
> 
> I may have come up with the wrong scheme for the Present() and
> isArray().  With my scheme, they *have* to be macros because the API
> functions are unsafe:  when numarray or Numeric is not present, the API
> function table pointers are NULL so calling through them results in
> either a fatal error or a segfault.
>
The macros can be hidden from the module file scope by wrapping them
in a function (see attached demo)
> 
> There is an additional problem that the "same functions" need to be
> called through different API pointers depending on whether numarray 
> or Numeric is being supported.  Redefinition of typedefs and enumerations
> 
> (or perhaps conditional compilation short-circuited re-definitions) may
> also present a problem with compiling (or worse, running).
>
Tested and works. 
>
> I certainly like the idea of supporting both in the same extension
> module,  but don't see how to get there, other than with separate
> compilation units.  With seperate .c files,  I'm not aware of a problem
> other than lots of global symbols.  I haven't demoed that yet so I am
> interested if someone has made it work.
> 
Yes, you cannot mix the numarray API and Numeric API in the same .c
file, but nevertheless you can hide the macros in small functions so
that the macros don't pollute.

Here you have a litte extension module 'pep' which tries to import Numeric
and numarray. It has a method 'whatsThere' that prints the names of
the Numerical/numarray extension modules it has imported.

It has also a method 'whatsThis' that prints if an object is a Numeric
array, a numarray array or something else:

Python 2.3.3 (#2, Feb 17 2004, 11:45:40)
[GCC 3.3.2 (Mandrake Linux 10.0 3.3.2-6mdk)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pep
>>> pep.whatsThere()
Numeric!
Numarray!
>>> import Numeric
>>> a = Numeric.arange(10)
>>> pep.whatsThis(a)
Numeric array
>>> import numarray
>>> a = numarray.arange(10)
>>> pep.whatsThis(a)
Numarray array
>>> a = 10
>>> pep.whatsThis(a)
Something else
>>>

The MANIFEST reads:
pepmodule.c
setup.py
Numeric/meta.c
Numeric/meta.h
numarray/meta.c
numarray/meta.h

My initial idea was to add the meta.* files to the Numeric/numarray include
files, but I did not try yet to solve the problem of the
PyArray_API PY_ARRAY_UNIQUE_SYMBOL defines in the same way (I am sure
it can be done.

Regards -- Gerard

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Chris B. <Chr...@no...>]

Tim Hochberg wrote:
> Todd Miller wrote:
>> Me too.  Can you (or somebody) post the application code which does the
>> drawlines?  I can definitely instrument the bottleneck C-code,  but I
>> don't have time to ascend the wxPython learning curve.

> Let me second that. With a little nudge from Chris Barker, I managed to 
> get wxPython to compile here and I have some changes that may speed up 
> drawlines, but it'd probably be best if we were all using the same 
> benchmark.

Well, if I can't code, at least I can nudge!

Perhaps I can also help get Todd what he's asking for, except that I'm 
not sure what you mean by "application code". Do you mean a small 
wxPython application that uses DC.DrawLines (and friends)? If so, yes, I 
can do that. What version of wxPython should I target?

CVS head?
2.5.1? (The latest "released" version)

there are some slight incompatibilities between versions, particularly 
with DC calls, unfortunately.

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Tim H. <tim...@co...>]

I'd bet a case of beer (or cash equivalent) that one of the main 
bottlenecks is the path 
PySequence_GetItem->_ndarray_item->_universalIndexing->_simpleIndexing->_simpleIndexingCore.  
The path through _universalIndexing in particular, if I deciphered it 
correctly,  looks very slow. I don't think it needs to be that way 
though, _universalIndexing could probably be sped up, but more promising 
I think _ndarray_item could be made to call _simpleIndexingCore without 
all that much work. It appears that this would save the creation of 
several intermediate objects and it also looks like a couple of calls 
back to python! I'm not familiar with this code though, so I could 
easily be missing something that makes calling _simpleIndexingCore 
harder than it looks.

-tim

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Tim H. <tim...@co...>]

Todd Miller wrote:

>On Mon, 2004-06-28 at 19:38, Sebastian Haase wrote:
>
>  
>
>>>Is 10x a measured number or a gut feel?
>>>      
>>>
>>[SNIP]
>>
>>so this looks to me like a factor of 10x.
>>    
>>
>
>Me too.  Can you (or somebody) post the application code which does the
>drawlines?  I can definitely instrument the bottleneck C-code,  but I
>don't have time to ascend the wxPython learning curve.
>  
>
Let me second that. With a little nudge from Chris Barker, I managed to 
get wxPython to compile here and I have some changes that may speed up 
drawlines, but it'd probably be best if we were all using the same 
benchmark.

-tim

Re: [Numpy-discussion] Numarray header PEP

[Todd M. <jm...@st...>]

On Tue, 2004-06-29 at 13:44, Gerard Vermeulen wrote:
> > 
> > The PEP is attached.  It is formatted using the docutils package which
> > can be used to generate HTML or PDF.  Comments and corrections would be
> > appreciated.
> > 
> PyQwt is a Python extension which can be conditionally compiled against
> Numeric and/or numarray (both, one of them or none).

Well that's cool!  I'll have to keep the PyQwt guys in mind as potential
first users.

> Your PEP excludes importing of Numeric and numarray in the same C-extension.

This is true but I don't understand your solution so I'll blather on
below.

> All you need to do is to hide the macros PyArray_Present(), PyArray_isArray()
> and import_array() into a few functions with numarray specific names, so
> that the following becomes possible:
> 
> #include <Numeric/meta.h>
> /* defines the functions (no macros)
>    int Numeric_Present();
>    int Numeric_isArray();
>    void import_numeric();
>    to hide the Numeric C-API stuff in a small Numeric/meta.c file.
> */
> #include <numarray/meta.h>
> /* defines the functions (no macros)
>    int numarray_Present();
>    int numarray_isArray();
>    void import_numarray();
>    to hide the numarray C-API stuff in a small numarray/meta.c file.
> */
> 

I may have come up with the wrong scheme for the Present() and
isArray().  With my scheme, they *have* to be macros because the API
functions are unsafe:  when numarray or Numeric is not present, the API
function table pointers are NULL so calling through them results in
either a fatal error or a segfault.

There is an additional problem that the "same functions" need to be
called through different API pointers depending on whether numarray or
Numeric is being supported.  Redefinition of typedefs and enumerations
(or perhaps conditional compilation short-circuited re-definitions) may
also present a problem with compiling (or worse, running).  

I certainly like the idea of supporting both in the same extension
module,  but don't see how to get there, other than with separate
compilation units.  With seperate .c files,  I'm not aware of a problem
other than lots of global symbols.  I haven't demoed that yet so I am
interested if someone has made it work.

Thanks very much for commenting on the PEP.

Regards,
Todd

> PyObject *
> some_array_returning_function(PyObject *m, PyObject *args)
> {
>     int param;
>     PyObject *result;
>     
>     if (!PyArg_ParseTuple(args, "i", &param))
> 	return NULL;
>     
>     if (Numeric_Present()) {
> 	result = numeric_returning_function(param);
>     } else if (Numarray_Present()) {
> 	result = numarray_returning_function(param);
>     } else {
> 	result = list_returning_function(param);
>     }
>     return result;
> }
> 
> PyObject *
> some_array_accepting_function(PyObject *m, PyObject *args)
> {
>     PyObject *sequence, *result;
>     
>     if (!PyArg_ParseTuple(args, "O", &sequence))
> 	return NULL;
>     
>     if (Numeric_isArray(sequence)) {
> 	result = numeric_input_function(sequence);
>     } else if (Numarray_isArray(sequence)) {
> 	result = numarray_input_function(sequence);
>     } else {
> 	result = sequence_input_function(sequence);
>     }
>     return result;
> }
> 
> /*
>   the code for the numeric_whatever_functions and for the
>   numarray_whatever_functions should be source files.
>  */
> 
> static void
> initfoo(void)
> {
>     PyObject *m = Py_InitModule3(
> 	"foo",
> 	_foo_functions,
> 	_foo__doc__);
>     if (m == NULL) return;
>     import_numeric();
>     import_numarray();
> }       
> 
> Regards -- Gerard
> 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
> digital self defense, top technical experts, no vendor pitches, 
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21218
(410) 338-4576

RE: [Numpy-discussion] Can not import image module

[Nadav H. <na...@vi...>]

Same (RH9.0 + Python 2.3.4). It is OK on my home PC (Gentoo 1.4 +Python =
2.3.4). I'll try your advice when I'll back at work.

 Thanks,
  Nadav.


-----Original Message-----
From:	Todd Miller [mailto:jm...@st...]
Sent:	Tue 29-Jun-04 20:36
To:	Nadav Horesh
Cc:	numpy-discussion
Subject:	Re: [Numpy-discussion] Can not import image module
On Tue, 2004-06-29 at 07:14, Nadav Horesh wrote:
> >>> from numarray import image

I tried this and it worked fine for me under RH9.0 and Python-2.3.4 and
also under Windows 2000 pro.  I suggest deleting your entire numarray
directory from site-packages and then re-installing.  Also, what's your
platform and shell?

Regards,
Todd

> Traceback (most recent call last):
>   File "<pyshell#4>", line 1, in -toplevel-
>     from numarray import image
>   File =
"/usr/local/lib/python2.3/site-packages/numarray/image/__init__.py", =
line 2, in -toplevel-
>     from combine import *
>   File =
"/usr/local/lib/python2.3/site-packages/numarray/image/combine.py", line =
65, in -toplevel-
>     _median =3D _combine.median
> AttributeError: 'module' object has no attribute 'median'
>=20
>=20
>=20
> numarray version: 1.0. I got this error also with version 0.9
>=20
>   Nadav.
>=20
>=20
>=20
>=20
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 -=20
> digital self defense, top technical experts, no vendor pitches,=20
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
--=20
Todd Miller
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21218
(410) 338-4576

Re: [Numpy-discussion] Numarray header PEP

[Gerard V. <ger...@gr...>]

> 
> The PEP is attached.  It is formatted using the docutils package which
> can be used to generate HTML or PDF.  Comments and corrections would be
> appreciated.
> 
PyQwt is a Python extension which can be conditionally compiled against
Numeric and/or numarray (both, one of them or none).

Your PEP excludes importing of Numeric and numarray in the same C-extension.

All you need to do is to hide the macros PyArray_Present(), PyArray_isArray()
and import_array() into a few functions with numarray specific names, so
that the following becomes possible:

#include <Numeric/meta.h>
/* defines the functions (no macros)
   int Numeric_Present();
   int Numeric_isArray();
   void import_numeric();
   to hide the Numeric C-API stuff in a small Numeric/meta.c file.
*/
#include <numarray/meta.h>
/* defines the functions (no macros)
   int numarray_Present();
   int numarray_isArray();
   void import_numarray();
   to hide the numarray C-API stuff in a small numarray/meta.c file.
*/

PyObject *
some_array_returning_function(PyObject *m, PyObject *args)
{
    int param;
    PyObject *result;
    
    if (!PyArg_ParseTuple(args, "i", &param))
	return NULL;
    
    if (Numeric_Present()) {
	result = numeric_returning_function(param);
    } else if (Numarray_Present()) {
	result = numarray_returning_function(param);
    } else {
	result = list_returning_function(param);
    }
    return result;
}

PyObject *
some_array_accepting_function(PyObject *m, PyObject *args)
{
    PyObject *sequence, *result;
    
    if (!PyArg_ParseTuple(args, "O", &sequence))
	return NULL;
    
    if (Numeric_isArray(sequence)) {
	result = numeric_input_function(sequence);
    } else if (Numarray_isArray(sequence)) {
	result = numarray_input_function(sequence);
    } else {
	result = sequence_input_function(sequence);
    }
    return result;
}

/*
  the code for the numeric_whatever_functions and for the
  numarray_whatever_functions should be source files.
 */

static void
initfoo(void)
{
    PyObject *m = Py_InitModule3(
	"foo",
	_foo_functions,
	_foo__doc__);
    if (m == NULL) return;
    import_numeric();
    import_numarray();
}       

Regards -- Gerard

Re: [Numpy-discussion] Can not import image module

[Todd M. <jm...@st...>]

On Tue, 2004-06-29 at 07:14, Nadav Horesh wrote:
> >>> from numarray import image

I tried this and it worked fine for me under RH9.0 and Python-2.3.4 and
also under Windows 2000 pro.  I suggest deleting your entire numarray
directory from site-packages and then re-installing.  Also, what's your
platform and shell?

Regards,
Todd

> Traceback (most recent call last):
>   File "<pyshell#4>", line 1, in -toplevel-
>     from numarray import image
>   File "/usr/local/lib/python2.3/site-packages/numarray/image/__init__.py", line 2, in -toplevel-
>     from combine import *
>   File "/usr/local/lib/python2.3/site-packages/numarray/image/combine.py", line 65, in -toplevel-
>     _median = _combine.median
> AttributeError: 'module' object has no attribute 'median'
> 
> 
> 
> numarray version: 1.0. I got this error also with version 0.9
> 
>   Nadav.
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 - 
> digital self defense, top technical experts, no vendor pitches, 
> unmatched networking opportunities. Visit www.blackhat.com
> _______________________________________________
> Numpy-discussion mailing list
> Num...@li...
> https://lists.sourceforge.net/lists/listinfo/numpy-discussion
-- 
Todd Miller
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21218
(410) 338-4576

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Chris B. <Chr...@no...>]

Tim Hochberg wrote:
>>> but I believe that wxPoints now obey the PySequence protocol, so I 
>>> think that the whole wxPySwigInstance_Check branch could be removed. 
>>> To get that into wxPython you'd probably have to convince Robin that 
>>> it wouldn't hurt the speed of list of wxPoints unduly.
>>>
>>> Wait... If the above doesn't work, I think I do have a way that might 
>>> work for speeding the check for a wxPoint. Before the loop starts, 
>>> get a pointer to wx.core.Point (the class for wxPoints these days) 
>>> and call it wxPoint_Type. Then just use for the check:
>>>        o->ob_type == &wxPoint_Type
>>> Worth a try anyway.
>>>
>>> Unfortunately, I don't have any time to try any of this out right now.
>>>
>>> Chris, are you feeling bored?

Do you mean me? if so:
A) I'm not bored.
B) This is all kind of beyond me anyway, and
C) I'm planning on implementing my own custom DC.DrawLotsOfStuff code, 
because I have some specialized needs that probably don't really belong 
in wxPython. My stuff will take Numeric arrays as input (This is for my 
FloatCanvas, if anyone cares). I'm still using Numeric, as numarray is a 
LOT slower when used in FloatCanvas, probably because I do a lot with 
small arrays, and maybe because of what we're talking about here as well.

However, This may turn out to be important to me some day, so who knows? 
I'll keep this note around.

>> What's the chance of adding direct support for numarray to wxPython? 
>> Our PEP reduces the burden on a package to at worst adding 3 include
>> files for numarray plus the specialized package code.   With those
>> files,  the package can be compiled by users without numarray and also
>> run without numarray, but would receive a real boost for people willing
>> to install numarray since the sequence protocol could be bypassed.

If the PEP is accepted, and those include files are part of the standard 
Python distro, I suspect Robin would be quite happy to add direct 
support, at least if someone else writes the code.

Whether he'd be open to including those files in the wxPython 
distribution itself, I don't know. Perhaps I'll drop him a line.

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Tim H. <tim...@co...>]

Todd Miller wrote:

>On Mon, 2004-06-28 at 20:45, Tim Hochberg wrote:
>  
>
>>Todd Miller wrote:
>>
>>    
>>
[SNIP]

>>This reminds me, when profiling bits and pieces of my code I've often 
>>noticed that __del__ chews up a large chunk of time. Is there any 
>>prospect of this being knocked down at all, or is it inherent in the 
>>structure of numarray?
>>    
>>
>
>__del__ is IMHO the elegant way to do numarray's shadowing of
>"misbehaved arrays".  misbehaved arrays are ones which don't meet the
>requirements of a particular C-function, but generally that means
>noncontiguous, byte-swapped, misaligned, or of the wrong type;  it also
>can mean some other sequence type like a list or tuple.  I think using
>the destructor is "necessary" for maintaining Numeric compatibility in C
>because you can generally count on arrays being DECREF'd,  but obviously
>you couldn't count on some new API call being called.  
>  
>
OK, that makes sense. In a general sense at least, I'll have to dig into 
the source to figure out the details.

>__del__ used to be implemented in C as tp_dealloc,  but I was running
>into segfaults which I tracked down to the order in which a new style
>class instance is torn down.  The purpose of __del__ is to copy the
>contents of a well behaved working array (the shadow) back onto the
>original mis-behaved array.  The problem was that, because of the
>numarray class hierarchy, critical pieces of the shadow (the instance
>dictionary) had already been torn down before the tp_dealloc was
>called.  The only way I could think of to fix it was to move the
>destructor farther down in the class hierarchy, i.e. from
>_numarray.tp_dealloc to NumArray.__del__ in Python.
>  
>
It seems that one could stash a reference to the instance dict somewhere 
(in PyArrayObject perhaps) to prevent the instance dict from being torn 
down when the rest of the subclass goes away. It would require another 
field in PyArrayObject , but that's big enough allready that no one 
would notice. I could easily be way off base here though. If I end up 
with too much spare time at some point, I may try to look into this.


>If anyone can think of a way to get rid of __del__, I'm all for it.
>
>  
>
[SNIP]

>>
>>I don't see any easy/obvious ways to speed up wxPySwigInstance_Check, 
>>    
>>
>
>Why no CheckExact, even if it's hand coded?  Maybe the setup is tedious?
>  
>
I don't know although I suspect the setup being tedious might be part of 
it. I also believe that until recently, wxPython used old style classes 
and you couldn't use CheckExact. What I propose below is essentially a 
hand coded version of check exact for wxPoints.

>  
>
>>but I believe that wxPoints now obey the PySequence protocol, so I think 
>>that the whole wxPySwigInstance_Check branch could be removed. To get 
>>that into wxPython you'd probably have to convince Robin that it 
>>wouldn't hurt the speed of list of wxPoints unduly.
>>
>>Wait... If the above doesn't work, I think I do have a way that might 
>>work for speeding the check for a wxPoint. Before the loop starts, get a 
>>pointer to wx.core.Point (the class for wxPoints these days) and call it 
>>wxPoint_Type. Then just use for the check:
>>        o->ob_type == &wxPoint_Type
>>Worth a try anyway.
>>
>>Unfortunately, I don't have any time to try any of this out right now.
>>
>>Chris, are you feeling bored?
>>
>>-tim
>>    
>>
>
>What's the chance of adding direct support for numarray to wxPython? 
>Our PEP reduces the burden on a package to at worst adding 3 include
>files for numarray plus the specialized package code.   With those
>files,  the package can be compiled by users without numarray and also
>run without numarray, but would receive a real boost for people willing
>to install numarray since the sequence protocol could be bypassed.
>  
>
No idea, sorry. I haven't been keeping up with wxPython development lately.

-tim

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Todd M. <jm...@st...>]

On Mon, 2004-06-28 at 19:38, Sebastian Haase wrote:

> > Is 10x a measured number or a gut feel?
> 
> I put some time.clock() statements into the wxPyPlot code
> I got this: (the times are differences: T_after-T_before)
> one run with numarray:
> <__main__.PolyLine instance at 0x868d414> time= 1.06
> <__main__.PolyMarker instance at 0x878e9c4> time= 1.37
> a second run with numarray:
> <__main__.PolyLine instance at 0x875da1c> time= 0.85
> <__main__.PolyMarker instance at 0x86da034> time= 1.04
> first run with Numeric:
> <__main__.PolyLine instance at 0x858babc> time= 0.07
> <__main__.PolyMarker instance at 0x858bc4c> time= 0.14
> a second one:
> <__main__.PolyLine instance at 0x858cd7c> time= 0.08
> <__main__.PolyMarker instance at 0x8585d8c> time= 0.17
> This seems to be consistent with the profiling I did before:
> I get this w/ numarray:
>    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>         1    1.140    1.140    1.320    1.320 gdi.py:554(DrawLines)
>         1    1.250    1.250    1.520    1.520 gdi.py:792(_DrawRectangleList)
>     50230    0.450    0.000    0.450    0.000 numarraycore.py:501(__del__)
> and this with Numeric:
>         1    0.080    0.080    0.080    0.080 gdi.py:554(DrawLines)
>         1    0.090    0.090    0.090    0.090 gdi.py:792(_DrawRectangleList)
> 
> so this looks to me like a factor of 10x.

Me too.  Can you (or somebody) post the application code which does the
drawlines?  I can definitely instrument the bottleneck C-code,  but I
don't have time to ascend the wxPython learning curve.

Todd

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Todd M. <jm...@st...>]

On Mon, 2004-06-28 at 20:45, Tim Hochberg wrote:
> Todd Miller wrote:
> 
> >On Mon, 2004-06-28 at 17:14, Sebastian Haase wrote:
> >  
> >
> >> [SNIP]
> >>
> >>My original question was just this: Does anyone know why numarray is maybe 10 
> >>times slower that Numeric with that particular code segment 
> >>(PySequence_GetItem) ?
> >>    
> >>
> >
> >Well, the short answer is probably: no.
> >
> >Looking at the numarray sequence protocol benchmarks in
> >Examples/bench.py, and looking at what wxPython is probably doing
> >(fetching a 1x2 element array from an Nx2 and then fetching 2 numerical
> >values from that)... I can't fully nail it down.  My benchmarks show
> >that numarray is 4x slower for fetching the two element array but only
> >1.1x slower for the two numbers;  that makes me expect at most 4x
> >slower.  
> >
> >Noticing the 50k __del__ calls in your profile,  I eliminated __del__
> >(breaking numarray) to see if that was the problem;  the ratios changed
> >to 2.5x slower and 0.9x slower (actually faster) respectively.
> >  
> >
> This reminds me, when profiling bits and pieces of my code I've often 
> noticed that __del__ chews up a large chunk of time. Is there any 
> prospect of this being knocked down at all, or is it inherent in the 
> structure of numarray?

__del__ is IMHO the elegant way to do numarray's shadowing of
"misbehaved arrays".  misbehaved arrays are ones which don't meet the
requirements of a particular C-function, but generally that means
noncontiguous, byte-swapped, misaligned, or of the wrong type;  it also
can mean some other sequence type like a list or tuple.  I think using
the destructor is "necessary" for maintaining Numeric compatibility in C
because you can generally count on arrays being DECREF'd,  but obviously
you couldn't count on some new API call being called.  

__del__ used to be implemented in C as tp_dealloc,  but I was running
into segfaults which I tracked down to the order in which a new style
class instance is torn down.  The purpose of __del__ is to copy the
contents of a well behaved working array (the shadow) back onto the
original mis-behaved array.  The problem was that, because of the
numarray class hierarchy, critical pieces of the shadow (the instance
dictionary) had already been torn down before the tp_dealloc was
called.  The only way I could think of to fix it was to move the
destructor farther down in the class hierarchy, i.e. from
_numarray.tp_dealloc to NumArray.__del__ in Python.

If anyone can think of a way to get rid of __del__, I'm all for it.

> >The large number of "Check" routines preceding the numarray path (I
> >count 7 looking at my copy of wxPython) has me a little concerned.  I
> >think those checks are  more expensive for numarray because it is a new
> >style class.  
> >
> If that's really a significant slowdown, the culprit's are likely 
> PyTuple_Check, PyList_Check and wxPySwigInstance_Check.  
> PySequence_Check appears to just be pointer compares and shouldn't 
> invoke any new style class machinery. PySequence_Length calls sq_length, 
> but appears also to not involve new class machinery. Of these, I think 
> PyTuple_Check and PyList_Check could be replaced with PyTuple_CheckExact 
> and PyList_CheckExact. This would slow down people using subclasses of 
> tuple/list, but speed everyone else up since the latter pair of 
> functions are just pointer compares. I think the former group is a very 
> small minority, possibly nonexistent, minority, so this would probably 
> be acceptable.
> 
> I don't see any easy/obvious ways to speed up wxPySwigInstance_Check, 

Why no CheckExact, even if it's hand coded?  Maybe the setup is tedious?

> but I believe that wxPoints now obey the PySequence protocol, so I think 
> that the whole wxPySwigInstance_Check branch could be removed. To get 
> that into wxPython you'd probably have to convince Robin that it 
> wouldn't hurt the speed of list of wxPoints unduly.
> 
> Wait... If the above doesn't work, I think I do have a way that might 
> work for speeding the check for a wxPoint. Before the loop starts, get a 
> pointer to wx.core.Point (the class for wxPoints these days) and call it 
> wxPoint_Type. Then just use for the check:
>         o->ob_type == &wxPoint_Type
> Worth a try anyway.
> 
> Unfortunately, I don't have any time to try any of this out right now.
> 
> Chris, are you feeling bored?
> 
> -tim

What's the chance of adding direct support for numarray to wxPython? 
Our PEP reduces the burden on a package to at worst adding 3 include
files for numarray plus the specialized package code.   With those
files,  the package can be compiled by users without numarray and also
run without numarray, but would receive a real boost for people willing
to install numarray since the sequence protocol could be bypassed.

Regards,
Todd

[Numpy-discussion] generalized SVD

[Curzio B. <cur...@un...>]

Hi.

Has anyone a suggestion on where to find a GSVD implementation which I 
can use with numarray? I've seen GSVD is part of LAPACK, so maybe one 
option could be to use a LAPACK library implementing the required 
functions...

thanks in advance. curzio.

[Numpy-discussion] Can not import image module

[Nadav H. <na...@vi...>]

>>> from numarray import image

Traceback (most recent call last):
  File "<pyshell#4>", line 1, in -toplevel-
    from numarray import image
  File =
"/usr/local/lib/python2.3/site-packages/numarray/image/__init__.py", =
line 2, in -toplevel-
    from combine import *
  File =
"/usr/local/lib/python2.3/site-packages/numarray/image/combine.py", line =
65, in -toplevel-
    _median =3D _combine.median
AttributeError: 'module' object has no attribute 'median'



numarray version: 1.0. I got this error also with version 0.9

  Nadav.

Re: [off topic] Re: [Numpy-discussion] numarray speed - PySequence_GetItem

[Tim H. <tim...@co...>]

Todd Miller wrote:

>On Mon, 2004-06-28 at 17:14, Sebastian Haase wrote:
>  
>
>> [SNIP]
>>
>>My original question was just this: Does anyone know why numarray is maybe 10 
>>times slower that Numeric with that particular code segment 
>>(PySequence_GetItem) ?
>>    
>>
>
>Well, the short answer is probably: no.
>
>Looking at the numarray sequence protocol benchmarks in
>Examples/bench.py, and looking at what wxPython is probably doing
>(fetching a 1x2 element array from an Nx2 and then fetching 2 numerical
>values from that)... I can't fully nail it down.  My benchmarks show
>that numarray is 4x slower for fetching the two element array but only
>1.1x slower for the two numbers;  that makes me expect at most 4x
>slower.  
>
>Noticing the 50k __del__ calls in your profile,  I eliminated __del__
>(breaking numarray) to see if that was the problem;  the ratios changed
>to 2.5x slower and 0.9x slower (actually faster) respectively.
>  
>
This reminds me, when profiling bits and pieces of my code I've often 
noticed that __del__ chews up a large chunk of time. Is there any 
prospect of this being knocked down at all, or is it inherent in the 
structure of numarray?

>The large number of "Check" routines preceding the numarray path (I
>count 7 looking at my copy of wxPython) has me a little concerned.  I
>think those checks are  more expensive for numarray because it is a new
>style class.  
>
If that's really a significant slowdown, the culprit's are likely 
PyTuple_Check, PyList_Check and wxPySwigInstance_Check.  
PySequence_Check appears to just be pointer compares and shouldn't 
invoke any new style class machinery. PySequence_Length calls sq_length, 
but appears also to not involve new class machinery. Of these, I think 
PyTuple_Check and PyList_Check could be replaced with PyTuple_CheckExact 
and PyList_CheckExact. This would slow down people using subclasses of 
tuple/list, but speed everyone else up since the latter pair of 
functions are just pointer compares. I think the former group is a very 
small minority, possibly nonexistent, minority, so this would probably 
be acceptable.

I don't see any easy/obvious ways to speed up wxPySwigInstance_Check, 
but I believe that wxPoints now obey the PySequence protocol, so I think 
that the whole wxPySwigInstance_Check branch could be removed. To get 
that into wxPython you'd probably have to convince Robin that it 
wouldn't hurt the speed of list of wxPoints unduly.

Wait... If the above doesn't work, I think I do have a way that might 
work for speeding the check for a wxPoint. Before the loop starts, get a 
pointer to wx.core.Point (the class for wxPoints these days) and call it 
wxPoint_Type. Then just use for the check:
        o->ob_type == &wxPoint_Type
Worth a try anyway.

Unfortunately, I don't have any time to try any of this out right now.

Chris, are you feeling bored?

-tim


>I have a hard time imagining a 10x difference overall, 
>but I think Python does have to traverse the numarray class hierarchy
>rather than do a type pointer comparison so they are more expensive.
>  
>

[Numpy-discussion] bug ? in Records arrays in numarray

[Sebastian H. <ha...@ms...>]

Hi,
I have two record arrays. I was trying to assign one item from one recarray to 
the other:
>>> omx.zext[0] = main.ring4ext[0,0]
Traceback (most recent call last):
  File "<input>", line 1, in ?
  File "X:/PrWin\numarray\records.py", line 744, in _setitem
    self.field(self._names[i])[row] = value.field(self._names[i])
  File "C:\Python22\Lib\site-packages\numarray\numarraycore.py", line 619, in 
__tonumtype__
ValueError: Can't use non-rank-0 array as a scalar.
>>> `omx.zext[0]`
'<numarray.records.Record instance at 0x042AFC78>'
>>> `main.ring4ext[0,0]`
'<numarray.records.Record instance at 0x042AFC78>'
>>> q = omx.zext[0]
>>> w = main.ring4ext[0,0]
>>> q
(2097152107, -595656700, 91141, 1.0634608642000868e+037, -1.14841804241843e
+018, 1.2771574333702815e-040)
>>> w
(array([428]), array([75]), array([124]), array([  1.08846451e+09], 
type=Float32), array([ 99.25], type=Float32), array([ 1996.82995605], 
type=Float32))
>>> omx.zext._formats
['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']
>>> main.ring4ext._formats
['1Int32', '1Int32', '1Int32', '1Float32', '1Float32', '1Float32']


I can't track down why one (q) contains scalars and the other (w) constains 
arrays (is array([428]) a 'rank-0 array'? ) ! 
This is how I generate them:
main.ring4ext = rec.RecArray(main._extHdrRingBuffer, "i4,i4,i4,f4,f4,f4",
  shape=(ts,nc), names=("num","min","max","time","mean","darkVal"), aligned=1)
omx.zext      = rec.array(formats="i4,i4,i4,f4,f4,f4",
  names=("num","min","max","time","mean","darkVal"),shape=tuple(shapeZ),aligned=1 )

[[BTW: (shapeZ is a list, if I say: ...,shape=shapeZ    I get 
NameError, "Illegal shape %s" % `shape`   from "records.py" in line 462   -- 
usually type(shape) == types.ListType should be OK, right ?)
]]


Any ideas?

Thanks,
Sebastian Haase