pyobjc-dev

[Pyobjc-dev] PyBuffer* vs. array.array()

[Bill B. <bb...@co...>]

I'm in the process of bridging various methods in PyObjC that take  
(void *) types as arguments or return (void *) buffer references.

On the C side of the fence, I'm using PyBuffer_FromReadWriteMemory() or  
PyBuffer_FromMemory() as appropriate to create the buffer.  On the  
Python side, I use array.array('B') to create the buffer that is passed  
across the bridge and mapped to the (void *) method arguments.

However, I noticed that the two types are not the same.  They both work  
as byte buffers just fine, but the result of PyBuffer*() acts as,  
basically, a big <str> on the Python side whereas array.array('B') is a  
buffer of specifically typed unsigned chars.

In writing the unit tests, I came across a problematic situation that  
could easily arise in code (feel free to comment on the silliness of  
this code, if any... and note that I'm using the comprehension style  
even after that long rant I posted earlier :-):

         singlePlane = array.array('B')
         singlePlane.fromlist([0 for x in  range(0, width*height*3)] )
         for i in range(0, 256*256):
             si = i * 3
             singlePlane[si] = rPlane[i]
             singlePlane[si+1] = gPlane[i]
             singlePlane[si+2] = bPlane[i]

	    i2 = ... create i2 using data in singlePlane ...
....  then ...

         bitmapData = i2.bitmapData()
         self.assertEquals(len(bitmapData), len(singlePlane))
         for i in range(0,100):
             self.assertEquals(bitmapData[i], singlePlane[i],  
"bitmapData and singlePlane differ at byte %d" % i)

The contents of singlePlane and bitmapData are identical, but the unit  
test fails:

<type 'str'> -- type of element 0 of bitmapData
<type 'int'> -- type of element 0 of singlePlane
F.
======================================================================
FAIL: testImageData (__main__.TestNSBitmapImageRep)
----------------------------------------------------------------------
Traceback (most recent call last):
   File "Lib/AppKit/test/test_nsbitmapimagerep.py", line 74, in  
testImageData
     self.assertEquals(bitmapData[i], singlePlane[i], "bitmapData and  
singlePlane differ at byte %d" % i)
   File  
"/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/ 
unittest.py", line 292, in failUnlessEqual
     raise self.failureException, \
AssertionError: bitmapData and singlePlane differ at byte 0

----------------------------------------------------------------------
Ran 2 tests in 1.564s

FAILED (failures=1)

--

The real problem is that something created via array.array() comes back  
as an object that behaves differently.   Clearly, a bug in my bridge  
code, yes, but raises a question:

What should I be using?

 From the python side, it appears to only be possible to create buffer  
style objects-- big byte bags-- via the array module [yes, a <str> will  
work, but a byte-array seems so much more correct].  On the C side, it  
seems that the only way to create a byte buffer like object is to use  
the PyBuffer* API.

Advice, please?

BTW:  All of this lays the foundation for creating a PILImage object in  
Cocoa that is a seamless part of the NSImage family of classes.  I.e.  
PIL would then be fully integrated into any Cocoa app that can  
dynamically load the python interpreter (another problem I'm working  
through).

Thanks.

b.bum


b.bum
We gladly feast on those who would subdue us.

Re: [Pyobjc-dev] PyBuffer* vs. array.array()

[Ronald O. <ous...@ci...>]

On Sunday, Jan 5, 2003, at 05:56 Europe/Amsterdam, Bill Bumgarner wrote:
>
> The real problem is that something created via array.array() comes 
> back as an object that behaves differently.   Clearly, a bug in my 
> bridge code, yes, but raises a question:
>
> What should I be using?
>
> From the python side, it appears to only be possible to create buffer 
> style objects-- big byte bags-- via the array module [yes a <str> will 
> work, but a byte-array seems so much more correct].  On the C side, it 
> seems that the only way to create a byte buffer like object is to use 
> the PyBuffer* API.
You could use the array module on the C side. That is more work than 
just using the PyBuffer API but would solve your problem.

<advocate style=devil>
However, what if I raw strings into the API. If I get array objects 
back from the API there still is an inconsistency.
</advocate>

I agree that array objects are more suitable for 'bags of bytes' than 
the (builtin) alternatives. Note that some people seem to use Numeric 
python for image processing (http://www.pfdubois.com/numpy/). Adding 
support for using numeric arrays instead of array.array might be a 
usefull extension, but probably more as an item on the TODO list than 
something to implement right now.

All things considered I'd go for using array.array from C, with a 
fallback to the PyBuffer API when
you cannot import the array module.

Ronald

Re: [Pyobjc-dev] PyBuffer* vs. array.array()

[Bill B. <bb...@co...>]

On Sunday, Jan 5, 2003, at 11:06 US/Eastern, Ronald Oussoren wrote:
> On Sunday, Jan 5, 2003, at 05:56 Europe/Amsterdam, Bill Bumgarner=20
> wrote:
>> The real problem is that something created via array.array() comes=20
>> back as an object that behaves differently.   Clearly, a bug in my=20
>> bridge code, yes, but raises a question:
>>
>> What should I be using?
>>
>> =46rom the python side, it appears to only be possible to create =
buffer=20
>> style objects-- big byte bags-- via the array module [yes a <str>=20
>> will work, but a byte-array seems so much more correct].  On the C=20
>> side, it seems that the only way to create a byte buffer like object=20=

>> is to use the PyBuffer* API.
> You could use the array module on the C side. That is more work than=20=

> just using the PyBuffer API but would solve your problem.

Looking at this more closely, it seems that the array module requires=20
that it 'owns' the pointer to the memory it contains.  That is, it=20
allocates/deallocate the memory and-- the killer-- resizes the hunk of=20=

memory at will.

So, it appears that the answer is that I need to figure out how to=20
create true buffer objects from the python side.

Which is trivial.  I hadn't realized that 'buffer' is a primitive type=20=

and, therefore, is effectively built in...

Not so trivial-- primitive in 2.3, built-in function in 2.2.   Need to=20=

make sure that whatever I implement works with bost.

Deal killer:  results of buffer() are read-only on 2.2 (didn't check=20
'<buffer>' type on 2.3).  I would have to expose=20
PyBuffer_FromReadWriteObject() to Python to be able to create a=20
readwrite buffer object from within Python itself.

End result:  Stick with the current dichotomy.  It is annoying, but not=20=

killer.=13

> <advocate style=3Ddevil>
> However, what if I raw strings into the API. If I get array objects=20
> back from the API there still is an inconsistency.
> </advocate>

Raw string?  As in '1230x045678'?

Since the underlying code specifically parses for objects that=20
implement the character buffer API, strings work fine-- as do basically=20=

anything else that encapsulates a single-segment buffer.

> I agree that array objects are more suitable for 'bags of bytes' than=20=

> the (builtin) alternatives. Note that some people seem to use Numeric=20=

> python for image processing (http://www.pfdubois.com/numpy/). Adding=20=

> support for using numeric arrays instead of array.array might be a=20
> usefull extension, but probably more as an item on the TODO list than=20=

> something to implement right now.

Right.   I also want the code to build/run against a stock installation=20=

of Python 2.2 or greater.  I believe that array/buffer is a part of the=20=

core?  I.e. will always be there?

In the future, the answer may be to have a 'buffer factory' type API=20
[on the C side more than the Python side] that, given a pointer and=20
length, produces an instance of the appropriate buffer class, as=20
configured by the developer.  If the developer wishes to use the=20
Numeric array classes (which I have not looked at yet), it would simply=20=

be a matter of implementing the appropriate 'delegate' method and=20
making the factory aware of it.

But, as you say, it is a TODO item as opposed to being on the immediate=20=

radar.

> All things considered I'd go for using array.array from C, with a=20
> fallback to the PyBuffer API when
> you cannot import the array module.

I was going to go down that path, but the deeper analysis of array=20
(discussed above) indicates that it isn't the right answer.   Instead,=20=

I'm going to see what it takes to create a true 'buffer' instance in=20
Python.  The answer may be that it can't be done and that the developer=20=

is simply going to have to live with the dichotomy between buffer and=20
array.

If that is the case, then this does sound like an issue truly pertinent=20=

to discussion on python-dev.  Notably, it would be useful to have a=20
buffer/array that behaves the same on both sides of the Python/C wall=20
and can encapsulate an arbitrary, fixed length, buffer of memory=20
regardless of which side created the memory.  It should likely also=20
have the notion of 'ownership' of that memory-- i.e. whether or not it=20=

should deallocate the memory when the last reference to the object is=20
destroyed.

b.bum

Re: [Python-Dev] Re: [Pyobjc-dev] PyBuffer* vs. array.array()

[Jack J. <Jac...@or...>]

Note that there's the buffer *interface* and the buffer *object*. You 
want to use the buffer interface here, then on the Python side there's 
lots of objects that will work (arrays, NumPy arrays, strings for 
readonly access, etc).
--
- Jack Jansen        <Jac...@or...>        
http://www.cwi.nl/~jack -
- If I can't dance I don't want to be part of your revolution -- Emma 
Goldman -