pyobjc-dev

[Pyobjc-dev] How to get (void*) data into a Python array

[Trevor B. <mr...@gm...>]

Hello,

I am working on a small Python script to grab a single frame from the
iSight camera using PyObjC.  I want to import the frame into a python
numpy array for manipulation, and I want to write it exclusively in
Python.  I have fairly limited experience in Python and Objective-C, and
no prior experience with PyObjC.

I have made some progress, and currently it is able to find the camera,
open it, and capture a frame.  The frame data is (theoretically)
available in my QTCaptureDecompressedVideoOutput delegate's callback
function:

- (void)captureOutput:(QTCaptureOutput *)captureOutput
didOutputVideoFrame:(CVImageBufferRef)videoFrame
withSampleBuffer:(QTSampleBuffer *)sampleBuffer
fromConnection:(QTCaptureConnection *)connection

So I added "pdb.set_trace()" at the beginning of the callback, and am
trying to figure out how to get the raw bytes available in Python.

The closest I have come is examining sampleBuffer:
--------------------------------------------------
(Pdb) sampleBuffer
<QTSampleBuffer: 0x1044b42d0>
(Pdb) sampleBuffer.numberOfSamples()
1
(Pdb) sampleBuffer.lengthForAllSamples()
2624000
(Pdb) sampleBuffer.bytesForAllSamples()
4674363392L
--------------------------------------------------

However, this is where I'm confused.  bytesForAllSamples() returns a
(void*), which is literally, of course, a long integer.  I have no idea
how to get at those bytes, since I'm assuming I can't just dereference
an ObjC pointer somehow in Python.  I was hoping an NSData object
initialized with the address would work, but it does not:

--------------------------------------------------
(Pdb)
NSData.dataWithBytes_length_(sampleBuffer.bytesForAllSamples(),2624000)
*** TypeError: converting to a C array
--------------------------------------------------

Any guidance in getting past this point would be much appreciated!  And
thanks to everyone responsible for PyObjC, it is an excellent resource.

-Trevor

Re: [Pyobjc-dev] How to get (void*) data into a Python array

[Trevor B. <mr...@gm...>]

> IMHO your problem is not PyObjc - you have everything you need there. It's getting things into NumPy.
> 
> One way would be to use ctypes. You can create a pointer to whatever data-type you want, and initialize that with the returned integer. I don't have example code right here (wrong machine), but it's not that hard.
> 
> then at least you can access the data properly, and stuff it into a NumPy array.
> 
> However, the best solution would of course be to be able to create a NumPy array based on the pointer. I know NumPy has a C-API, maybe that's exposed somehow?
> 
> Diez

That's where I took it immediately after sending the e-mail: ctypes
pointers.  I managed to get a pointer to something... maybe it's pixel
data.  NumPy doesn't want to import my pointer as an array, and it turns
out I don't know what format the returned data is in, anyway.

Since efficiency isn't even a slight concern, I actually found a
technique that works... but the final step is Cocoa writing the image to
a file.

This saves a bitmap using Cocoa:
--------------------------------------------------
ciimage = CIImage.imageWithCVImageBuffer_(videoFrame)
rep = NSCIImageRep.imageRepWithCIImage_(ciimage)
bitrep = NSBitmapImageRep.alloc().initWithCIImage_(ciimage)
bitdata = bitrep.representationUsingType_properties_(NSBMPFileType,
objc.NULL)
bitdata.writeToFile_atomically_("grab.bmp", False)
--------------------------------------------------

But the real answer just came by mistake while poking around in pdb.
'bitdata' in the above code is an NSData object, which PyObjC can
apparently magically copy over to Python-land as a read-only buffer:

--------------------------------------------------
bitbuf = bitdata.bytes()
f = open("python.bmp", "w")
f.write(bitbuf)
f.close()
--------------------------------------------------

Strange but true, native Python saves it faster on my machine:

Save bitdata with Cocoa's writeToFile:atomically: takes 0.077 s
Save bitdata with Python's write() takes 0.0288 s

Thanks for the help, I think I have everything I need now.

Trevor

Re: [Pyobjc-dev] How to get (void*) data into a Python array

[Ronald O. <ron...@ma...>]

On 4 Feb, 2011, at 18:16, Trevor Bentley wrote:

> Hello,
> 
> I am working on a small Python script to grab a single frame from the
> iSight camera using PyObjC.  I want to import the frame into a python
> numpy array for manipulation, and I want to write it exclusively in
> Python.  I have fairly limited experience in Python and Objective-C, and
> no prior experience with PyObjC.
> 
> I have made some progress, and currently it is able to find the camera,
> open it, and capture a frame.  The frame data is (theoretically)
> available in my QTCaptureDecompressedVideoOutput delegate's callback
> function:
> 
> - (void)captureOutput:(QTCaptureOutput *)captureOutput
> didOutputVideoFrame:(CVImageBufferRef)videoFrame
> withSampleBuffer:(QTSampleBuffer *)sampleBuffer
> fromConnection:(QTCaptureConnection *)connection
> 
> So I added "pdb.set_trace()" at the beginning of the callback, and am
> trying to figure out how to get the raw bytes available in Python.
> 
> The closest I have come is examining sampleBuffer:
> --------------------------------------------------
> (Pdb) sampleBuffer
> <QTSampleBuffer: 0x1044b42d0>
> (Pdb) sampleBuffer.numberOfSamples()
> 1
> (Pdb) sampleBuffer.lengthForAllSamples()
> 2624000
> (Pdb) sampleBuffer.bytesForAllSamples()
> 4674363392L

That's a bug, PyObjC should be smart enough to know that it supposed to return a buffer (string/bytes/...) in bytesForAllSamples. I will fix this in a future version, but don't know yet when that will be.  Sadly enough I'll have to write some C code for that.

I haven't used ctypes myself beyond simple hello-world level scripts, but it should be possible to tell ctypes that the integer you get back from bytesForSamples is actually a 'void*', and that should give you a way to copy the buffer into a python string.

Ronald

Re: [Pyobjc-dev] How to get (void*) data into a Python array

[Ronald O. <ron...@ma...>]

On 4 Feb, 2011, at 21:55, Trevor Bentley wrote:

>> IMHO your problem is not PyObjc - you have everything you need there. It's getting things into NumPy.
>> 
>> One way would be to use ctypes. You can create a pointer to whatever data-type you want, and initialize that with the returned integer. I don't have example code right here (wrong machine), but it's not that hard.
>> 
>> then at least you can access the data properly, and stuff it into a NumPy array.
>> 
>> However, the best solution would of course be to be able to create a NumPy array based on the pointer. I know NumPy has a C-API, maybe that's exposed somehow?
>> 
>> Diez
> 
> That's where I took it immediately after sending the e-mail: ctypes
> pointers.  I managed to get a pointer to something... maybe it's pixel
> data.  NumPy doesn't want to import my pointer as an array, and it turns
> out I don't know what format the returned data is in, anyway.
> 
> Since efficiency isn't even a slight concern, I actually found a
> technique that works...
Which techinque did you find?

Ronald