[Pyobjc-dev] Problem with signature for QTKit Delegate (CVImageBufferRef)

[Ulrich E. <li...@ne...>]

hi,

i spend a couple of days to find out how to do video capturing using  
the current pyobc QTKit framework wrappers and run into a problem...

i couldn't find out, how to write the signature for the selector of my  
delegate for QTCaptureDecompressedVideoOutput
with the following definition:

- (void)captureOutput:(QTCaptureOutput *)captureOutput  
didOutputVideoFrame:(CVImageBufferRef)videoFrame
        withSampleBuffer:(QTSampleBuffer *)sampleBuffer fromConnection: 
(QTCaptureConnection *)connection

i tried several signature definitions like:

@objc.signature('v@:@{__CVBuffer=*}@@')
@objc.signature('v@:@{__CVBuffer=@}@@')
@objc.signature('v@:@^{__CVBuffer=}@@')
@objc.signature('v@:@@@@')

but they all end up in either a PyObcPointer created warning or a  
NSCFType for videoStream, which seems not be usable
at all (can i cast to an CVImageBufferRef somehow ?).

output example without signature:
PyObjCPointer created: at 0x29f4590 of type {__CVBuffer=}@@
got frame: <QTCaptureDecompressedVideoOutput: 0x2981ac0>  
<PyObjCPointer object at 0x25c8b78> <QTSampleBuffer: 0x29f3600>  
<QTCaptureConnection: 0x2982510 vide 2vuy>

output example for the "@objc.signature('v@:@@@@')" signature:
got frame: <QTCaptureDecompressedVideoOutput: 0x2982010> <NSCFType:  
0x29f7310> <QTSampleBuffer: 0x29f5bf0> <QTCaptureConnection: 0x2982a60  
vide 2vuy>

using "@objc.signature('v@:@{__CVBuffer=}@@')" results in an error  
message:
<type 'exceptions.RuntimeError'>: Cannot create FFI CIF: 1


now i'm lost ... has anyone here on the list an idea how to convice  
the pyobjc wrapper to pass me a valid object/reference to the
CVImageBufferRef or what to do with the NSCFType to get to the buffer  
-- as shown in the example below .. i want to extract the image-data
from the captured video (possibly as string-buffer reference for  
further use with numpy+ctypes).

any help is greatly appreciated !!

thanks for your help

cheers Ulrich


here is a minimal example of my code (created a simple pyobjc app in  
xcode and this is the main app-delegate):
i can also provide the whole example for tests if needed.


############################
from Foundation import *
from AppKit import *
from QTKit import *

k24RGBPixelFormat             = 0x00000018    # 24 bit rgb
k32ARGBPixelFormat            = 0x00000020    # 32 bit argb    (Mac)

class qtkit_captureAppDelegate(NSObject):
     def applicationDidFinishLaunching_(self, sender):
         NSLog("Application did finish launching.")


         session = QTCaptureSession.alloc().init()

         device =  
QTCaptureDevice.defaultInputDeviceWithMediaType_(QTMediaTypeVideo)
         if device is None:
             print "failed to open device."
             return

         device.open_(None)

         input = QTCaptureDeviceInput.alloc().initWithDevice_(device)
         session.addInput_error_(input, None)

         output = QTCaptureDecompressedVideoOutput.alloc().init()

         settings = dict(kCVPixelBufferWidthKey=640.0,
                         kCVPixelBufferHeightKey=480.0,
                          
kCVPixelBufferPixelFormatTypeKey=k32ARGBPixelFormat,
                         )

         output.setPixelBufferAttributes_(settings)
         session.addOutput_error_(output, None)

         class logger(NSObject):

             current_buffer = None

             # Declaration:
             #- (void)captureOutput:(QTCaptureOutput *)captureOutput  
didOutputVideoFrame:(CVImageBufferRef)videoFrame
             #       withSampleBuffer:(QTSampleBuffer *)sampleBuffer  
fromConnection:(QTCaptureConnection *)connection

             #@objc.signature('v@:@{__CVBuffer=*}@@')
             #@objc.signature('v@:@{__CVBuffer=@}@@')
             #@objc.signature('v@:@^{__CVBuffer=}@@')
             @objc.signature('v@:@@@@')
             def  
captureOutput_didOutputVideoFrame_withSampleBuffer_fromConnection_ 
(self, captureOutput, videoFrame, sampleBuffer, connection):
                 NSLog("got frame: %s %s %s %s" % (captureOutput,  
videoFrame, sampleBuffer, connection))

                 # always have only one frame retained
                 #CVBufferRetain(videoFrame)
                 #buffer_to_release = self.current_buffer
                 #self.current_buffer = videoFrame
                 #if buffer_to_release is not None:
                 #    CVBufferRelease(buffer_to_release)

                 # convert the CVImageBuffer into an NSImage
                 #cimg = CIImage.initWithCVImageBuffer_(test)
                 #rep = NSCIImageRep.initWithCIImage_(cimg)
                 #nimg = NSImage.alloc().initWithSize_(rep.size())
                 #nimg.addRepresentation_(rep)


         cb = logger.alloc().init()
         output.setDelegate_(cb)
         session.startRunning()

         self._data = (session, device, input, output, cb)

############################