Thread: [Pyobjc-dev] Possible memory leak passing Cocoa NSData or NSDictionary instances from python plugin
Brought to you by:
ronaldoussoren
Menu
▾
▴
pyobjc-dev
|
From: Barry W. <bar...@gm...> - 2008-06-02 01:57:21
|
Hi all,
I have an Objective-C application that uses plugins to generate some
data. THe app is compiled against the 10.5 SDK and is running on
Leopard (with the system python and pyobjc). The data is returned as
NSData* instances and its parameters as an NSDictionary instance, as
defined by the app's plugin formal protocol. Some of our plugins are
written in Objective-C and some in Python ("compiled" with py2app).
The app is compiled against the 10.5 SDK and is running on Leopard
(with the system python and pyobjc). I've noticed that there's a
signficant apparent memory leak in the application when using
Python-based plugins, but not when using the Objective-C based
plugins. Therefore, I've concluded that the issue is with the plugins,
not the main application. I've written a small demo app that exhibits
the apparent leak behavior.
The relevant sections of the Objective-C side are:
- (IBAction)getDataFromPython:(id)sender {
id<TestProtocol> plugin = [[pluginClass alloc ] init];
NSAssert([plugin conformsToProtocol:@protocol(TestProtocol)],
@"Plugin not id<TestProtocol>.");
for(NSUInteger i=0; i<self.nReps; i++) {
NSData *d = [plugin getNumpyData];
}
[plugin release];
}
- (IBAction)getDictFromPython:(id)sender {
id<TestProtocol> plugin = [[pluginClass alloc ] init];
NSAssert([plugin conformsToProtocol:@protocol(TestProtocol)],
@"Plugin not id<TestProtocol>.");
for(NSUInteger i=0; i<self.nReps; i++) {
NSDictionary *d = [plugin getDictionary];
}
[plugin release];
}
and the corresponding "getNumpyData" and "getDictFromPython" methods
in the plugin are:
import numpy as np
def getNumpyData(self):
return np.zeros(10000)
def getDictionary(self):
return NSDictionary.dictionaryWithObjectsAndKeys_('abc',
'string',
pkl.dumps(np.zeros(1000)),
'pkl_string',
None)
Investigation with ObjectAlloc in Instruments suggests that the
instances of "GeneralBlock" in the first case (getNumpyData) and
"GeneralBlock" and CFDictionary in the second (getDictionary) case are
created but not released, even after the event loop (and associated
NSAutoreleasePool.drain) are completed.
Is it possible that this is a memory leak caused by improper reference
management across the Objective-C/python bridge?
I am happy to file a bug in Apple's radar or the pyobjc sourceforge
project, but I'm not sure 1) if this is a bug or my fault and 2) where
the bug is most useful...
Would anyone be able to help me?
I've made a zip file with the Xcode project (app and plugin). The
project folder also contains an Instruments file that will drive and
record the behavior I describe above. The zip file is available from
http://rieke-server.physiol.washington.edu/~barry/python/PyObjCTest.zip
Any help would be greatly appreciated!
Thanks!
Barry
|
|
From: s s <li...@in...> - 2008-06-02 02:17:35
|
On Jun 1, 2008, at 9:57 PM, Barry Wark wrote:
> and the corresponding "getNumpyData" and "getDictFromPython" methods
> in the plugin are:
>
> import numpy as np
>
> def getNumpyData(self):
> return np.zeros(10000)
>
> def getDictionary(self):
> return NSDictionary.dictionaryWithObjectsAndKeys_('abc',
> 'string',
> pkl.dumps(np.zeros(1000)),
> 'pkl_string',
> None)
What do the Obj-C test versions of these two same methods look like?
S
|
|
From: Barry W. <bar...@gm...> - 2008-06-02 02:43:15
|
The equivalent Objective-C methods look like:
- (NSData*)getNumpyData {
return [NSMutableData dataWithLength:10000*sizeof(float)];
}
- (NSDictionary*)getDictionary {
char str[10000];
str[10000-1]=0;
return [NSDictionary dictionaryWithObjectsAndKeys:@"abc",
@"string",
[NSMutableString stringWithCString:str
encoding:NSUnicodeStringEncoding],
@"pkl_string",
nil];
}
I didn't include the Objective-C version in the zip file/project since
calling these methods does NOT produce an increase in memory usage
following the end of the event loop (or any non-deallocated objects in
instruments). In the case of getDictionary, I've tried to replicate an
approximate equivalent string to the pickle.dumps call in the python
protocol.
As you can see, I'm relying on the bridging of buffer
protocol-implementing objects such as numpy's ndarray <-> NSData in
the pyobjc bridge.
Barry
On Sun, Jun 1, 2008 at 7:17 PM, s s <li...@in...> wrote:
>
> On Jun 1, 2008, at 9:57 PM, Barry Wark wrote:
>>
>> and the corresponding "getNumpyData" and "getDictFromPython" methods
>> in the plugin are:
>>
>> import numpy as np
>>
>> def getNumpyData(self):
>> return np.zeros(10000)
>>
>> def getDictionary(self):
>> return NSDictionary.dictionaryWithObjectsAndKeys_('abc',
>> 'string',
>> pkl.dumps(np.zeros(1000)),
>> 'pkl_string',
>> None)
>
> What do the Obj-C test versions of these two same methods look like?
>
> S
>
>
>
|
|
From: Ronald O. <ron...@ma...> - 2008-06-02 05:44:19
Attachments:
smime.p7s
|
Barry,
Could you please file a bug for this on SourceForge when you don't
find a solution? I don't think I'll be able to look into this this
week and will be traveling after that.
Ronald
On 2 Jun, 2008, at 3:57, Barry Wark wrote:
> Hi all,
>
> I have an Objective-C application that uses plugins to generate some
> data. THe app is compiled against the 10.5 SDK and is running on
> Leopard (with the system python and pyobjc). The data is returned as
> NSData* instances and its parameters as an NSDictionary instance, as
> defined by the app's plugin formal protocol. Some of our plugins are
> written in Objective-C and some in Python ("compiled" with py2app).
> The app is compiled against the 10.5 SDK and is running on Leopard
> (with the system python and pyobjc). I've noticed that there's a
> signficant apparent memory leak in the application when using
> Python-based plugins, but not when using the Objective-C based
> plugins. Therefore, I've concluded that the issue is with the plugins,
> not the main application. I've written a small demo app that exhibits
> the apparent leak behavior.
>
> The relevant sections of the Objective-C side are:
> - (IBAction)getDataFromPython:(id)sender {
> id<TestProtocol> plugin = [[pluginClass alloc ] init];
> NSAssert([plugin conformsToProtocol:@protocol(TestProtocol)],
> @"Plugin not id<TestProtocol>.");
>
> for(NSUInteger i=0; i<self.nReps; i++) {
> NSData *d = [plugin getNumpyData];
> }
>
> [plugin release];
> }
>
> - (IBAction)getDictFromPython:(id)sender {
> id<TestProtocol> plugin = [[pluginClass alloc ] init];
> NSAssert([plugin conformsToProtocol:@protocol(TestProtocol)],
> @"Plugin not id<TestProtocol>.");
>
>
> for(NSUInteger i=0; i<self.nReps; i++) {
> NSDictionary *d = [plugin getDictionary];
> }
>
> [plugin release];
> }
>
> and the corresponding "getNumpyData" and "getDictFromPython" methods
> in the plugin are:
>
> import numpy as np
>
> def getNumpyData(self):
> return np.zeros(10000)
>
> def getDictionary(self):
> return NSDictionary.dictionaryWithObjectsAndKeys_('abc',
> 'string',
> pkl.dumps(np.zeros(1000)),
> 'pkl_string',
> None)
>
>
> Investigation with ObjectAlloc in Instruments suggests that the
> instances of "GeneralBlock" in the first case (getNumpyData) and
> "GeneralBlock" and CFDictionary in the second (getDictionary) case are
> created but not released, even after the event loop (and associated
> NSAutoreleasePool.drain) are completed.
>
> Is it possible that this is a memory leak caused by improper reference
> management across the Objective-C/python bridge?
>
> I am happy to file a bug in Apple's radar or the pyobjc sourceforge
> project, but I'm not sure 1) if this is a bug or my fault and 2) where
> the bug is most useful...
>
> Would anyone be able to help me?
>
> I've made a zip file with the Xcode project (app and plugin). The
> project folder also contains an Instruments file that will drive and
> record the behavior I describe above. The zip file is available from
> http://rieke-server.physiol.washington.edu/~barry/python/
> PyObjCTest.zip
>
> Any help would be greatly appreciated!
>
> Thanks!
>
> Barry
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2008.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Pyobjc-dev mailing list
> Pyo...@li...
> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev
|
|
From: Barry W. <bar...@gm...> - 2008-06-02 13:27:42
|
Done. Thank you for taking a look at this. Although I can't confirm
this, I suspect the memory leak appeared when we moved from 10.4 to
10.5 and concurently from pyobjc 1.4 to 2.0 (I know that there was no
memory leak on 10.4 with pyobjc 1.4).
barry
On Sun, Jun 1, 2008 at 10:44 PM, Ronald Oussoren <ron...@ma...> wrote:
> Barry,
>
> Could you please file a bug for this on SourceForge when you don't find a
> solution? I don't think I'll be able to look into this this week and will be
> traveling after that.
>
> Ronald
>
> On 2 Jun, 2008, at 3:57, Barry Wark wrote:
>
>> Hi all,
>>
>> I have an Objective-C application that uses plugins to generate some
>> data. THe app is compiled against the 10.5 SDK and is running on
>> Leopard (with the system python and pyobjc). The data is returned as
>> NSData* instances and its parameters as an NSDictionary instance, as
>> defined by the app's plugin formal protocol. Some of our plugins are
>> written in Objective-C and some in Python ("compiled" with py2app).
>> The app is compiled against the 10.5 SDK and is running on Leopard
>> (with the system python and pyobjc). I've noticed that there's a
>> signficant apparent memory leak in the application when using
>> Python-based plugins, but not when using the Objective-C based
>> plugins. Therefore, I've concluded that the issue is with the plugins,
>> not the main application. I've written a small demo app that exhibits
>> the apparent leak behavior.
>>
>> The relevant sections of the Objective-C side are:
>> - (IBAction)getDataFromPython:(id)sender {
>> id<TestProtocol> plugin = [[pluginClass alloc ] init];
>> NSAssert([plugin conformsToProtocol:@protocol(TestProtocol)],
>> @"Plugin not id<TestProtocol>.");
>>
>> for(NSUInteger i=0; i<self.nReps; i++) {
>> NSData *d = [plugin getNumpyData];
>> }
>>
>> [plugin release];
>> }
>>
>> - (IBAction)getDictFromPython:(id)sender {
>> id<TestProtocol> plugin = [[pluginClass alloc ] init];
>> NSAssert([plugin conformsToProtocol:@protocol(TestProtocol)],
>> @"Plugin not id<TestProtocol>.");
>>
>>
>> for(NSUInteger i=0; i<self.nReps; i++) {
>> NSDictionary *d = [plugin getDictionary];
>> }
>>
>> [plugin release];
>> }
>>
>> and the corresponding "getNumpyData" and "getDictFromPython" methods
>> in the plugin are:
>>
>> import numpy as np
>>
>> def getNumpyData(self):
>> return np.zeros(10000)
>>
>> def getDictionary(self):
>> return NSDictionary.dictionaryWithObjectsAndKeys_('abc',
>> 'string',
>> pkl.dumps(np.zeros(1000)),
>> 'pkl_string',
>> None)
>>
>>
>> Investigation with ObjectAlloc in Instruments suggests that the
>> instances of "GeneralBlock" in the first case (getNumpyData) and
>> "GeneralBlock" and CFDictionary in the second (getDictionary) case are
>> created but not released, even after the event loop (and associated
>> NSAutoreleasePool.drain) are completed.
>>
>> Is it possible that this is a memory leak caused by improper reference
>> management across the Objective-C/python bridge?
>>
>> I am happy to file a bug in Apple's radar or the pyobjc sourceforge
>> project, but I'm not sure 1) if this is a bug or my fault and 2) where
>> the bug is most useful...
>>
>> Would anyone be able to help me?
>>
>> I've made a zip file with the Xcode project (app and plugin). The
>> project folder also contains an Instruments file that will drive and
>> record the behavior I describe above. The zip file is available from
>> http://rieke-server.physiol.washington.edu/~barry/python/PyObjCTest.zip
>>
>> Any help would be greatly appreciated!
>>
>> Thanks!
>>
>> Barry
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by: Microsoft
>> Defy all challenges. Microsoft(R) Visual Studio 2008.
>> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
>> _______________________________________________
>> Pyobjc-dev mailing list
>> Pyo...@li...
>> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev
>
>
|
Thanks for helping keep SourceForge clean.
X