numpy-discussion

[Numpy-discussion] Is numpy supposed to support the buffer protocol?

[Christopher B. <Chr...@no...>]

HI all,

Robin Dunn has been working on adding better support for dumping data 
directly to wxPython from the num* packages. I've been talking to him 
about the new array interface, and he might well support it 
(particularly if one of us contributes code), but in the meantime, he's 
got a number of things working with python buffers. For instance:

wx.Image.SetDataBuffer(dataBuffer)

That sets the data for a wxImage to the buffer handed in. This isn't as 
nice as the array protocol, as it has no way of checking anything other 
than if the length of the buffer is correct, but it is a good way to 
maximize performance for this sort of thing.

he's now working on adding methods for creating wx.Bitmaps directly from 
buffers. In the process if testing some of this, I discovered that 
numarray (which Robin is testing with) works fine, but numpy does not. I 
get:

File "/usr/lib/python2.4/site-packages/wx-2.6-gtk2-unicode/wx/_core.py", 
line 2814, in SetDataBuffer
     return _core_.Image_SetDataBuffer(*args, **kwargs)
TypeError: non-character array cannot be interpreted as character buffer

If I try to pass in a numpy array, while it works great with a numarray 
array.

While I'm a great advocate of the new array protocol, it seems 
supporting the buffer protocol also would be a good idea. I've enclosed 
some simple test code. It works with numarray, but not numpy 1.0b4

Tested with Python 2.4.3, wxPython 2.6.3.0, Linux fedora core4

-Chris





-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Is numpy supposed to support the buffer protocol?

[Travis O. <oli...@ie...>]

Christopher Barker wrote:
> HI all,
>
> File 
> "/usr/lib/python2.4/site-packages/wx-2.6-gtk2-unicode/wx/_core.py", 
> line 2814, in SetDataBuffer
>     return _core_.Image_SetDataBuffer(*args, **kwargs)
> TypeError: non-character array cannot be interpreted as character buffer
>
> If I try to pass in a numpy array, while it works great with a 
> numarray array.

This error sounds like wx is using the *wrong* buffer protocol.  Don't 
use bf_getcharbuffer as it is of uncertain utility.  It is slated for 
removal from Python 3000.  It was meant to be used as a way to determine 
buffers that were supposed to contain characters (not arbitrary data).

Just use bf_getreadbuffer and bf_getwritebuffer  from tp_as_buffer.

More support for the buffer protocol all the way around is a good idea. 
  NumPy has always supported it very well (just make sure to use it 
correctly).   FYI, I'm going to write a PEP to get the array protocol 
placed as an add-on to the buffer protocol for Python 2.6

-Travis

Re: [Numpy-discussion] Is numpy supposed to support the buffer protocol?

[Robin D. <ro...@al...>]

Travis Oliphant wrote:
> Christopher Barker wrote:
>> HI all,
>>
>> File 
>> "/usr/lib/python2.4/site-packages/wx-2.6-gtk2-unicode/wx/_core.py", 
>> line 2814, in SetDataBuffer
>>     return _core_.Image_SetDataBuffer(*args, **kwargs)
>> TypeError: non-character array cannot be interpreted as character buffer
>>
>> If I try to pass in a numpy array, while it works great with a 
>> numarray array.
> 
> This error sounds like wx is using the *wrong* buffer protocol.  Don't 
> use bf_getcharbuffer as it is of uncertain utility.  It is slated for 
> removal from Python 3000.  It was meant to be used as a way to determine 
> buffers that were supposed to contain characters (not arbitrary data).
> 
> Just use bf_getreadbuffer and bf_getwritebuffer  from tp_as_buffer.

I'm using PyArg_Parse($input, "t#", ...) to get the buffer pointer and 
size.  Is there another format specifier to use for the buffer pointer 
using the other slots or do I need to drop down to a lower level API to 
get it?  I didn't realize there was a distinction between buffer and 
character buffer.  Another read of the PyArg_Parse docs with that new 
fact makes things a little more clear.

Looking at the code I guess "s#" will do it, I guess I thought it would 
try to coerce the object to a PyString like some other APIs do, which I 
was trying to avoid, but it doesn't appear to do that, (only encoding a 
unicode object if that is passed.)    I think I'll take a shot at using 
tp_as_buffer directly to avoid any confusion in the future and avoid the 
arg parse overhead...  Any other suggestions?

BTW Chris, try using buffer(RGB) and buffer(Alpha) in your sample, I 
expect that will work with the current code.

-- 
Robin Dunn
Software Craftsman
http://wxPython.org  Java give you jitters?  Relax with wxPython!

Re: [Numpy-discussion] Is numpy supposed to support the buffer protocol?

[Christopher B. <Chr...@no...>]

Robin Dunn wrote:
> BTW Chris, try using buffer(RGB) and buffer(Alpha) in your sample, I 
> expect that will work with the current code.

yup. that does work. I was concerned that it would make a copy, but it 
looks like it makes a new buffer object, but using the same data buffer, 
so that should be fine.

Thanks for all this,

-Chris


-- 
Christopher Barker, Ph.D.
Oceanographer
                                     		
NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chr...@no...

Re: [Numpy-discussion] Is numpy supposed to support the buffer protocol?

[Robert K. <rob...@gm...>]

Christopher Barker wrote:

> While I'm a great advocate of the new array protocol, it seems 
> supporting the buffer protocol also would be a good idea. I've enclosed 
> some simple test code. It works with numarray, but not numpy 1.0b4

Instead of

   I.SetDataBuffer(some_array)

you can use

   I.SetDataBuffer(buffer(some_array))

and it seems to work on OS X with Python 2.4, numpy 1.0b2 and wxMac 2.6.3.3 .

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco