Thread: [Pyobjc-dev] How do you write category code?

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Ok another newbie question: how do I write category code using PyObjC?
Searched documentation but didn't find anything...

Thank you!

On Jul 28, 2004, at 2:08 PM, Fortepianissimo wrote:

> Ok another newbie question: how do I write category code using PyObjC?
> Searched documentation but didn't find anything...

Painfully, with objc.classAddMethods(...).  I'd suggest not doing it at 
all unless you absolutely have to, until we have a better syntax for it 
(a metaclass mixin or something).

-bob

On Jul 28, 2004, at 11:16 AM, Bob Ippolito wrote:
> On Jul 28, 2004, at 2:08 PM, Fortepianissimo wrote:
>> Ok another newbie question: how do I write category code using PyObjC?
>> Searched documentation but didn't find anything...
>
> Painfully, with objc.classAddMethods(...).  I'd suggest not doing it 
> at all unless you absolutely have to, until we have a better syntax 
> for it (a metaclass mixin or something).

How about something like the following.   I like the second one better.

class NSObject(PyObjCAsCategory):
	pass

Or:

class MyCategory(ObjCCategory, NSObject):
	pass

---

objc.classAddMethods() serves a different purpose than categories.  It 
allows the developer to edit the live runtime as opposed to having to 
the formality of declaration required by traditional categories.   It 
can do everything that categories can do, as well.  There is also 
classAddMethod(cls, name, method) which can be used to add a single 
method to an ObjC class with any random selector as the name (as long 
as the selector's arg count matches the functions arg count).

Note that classAddMethods() and classAddMethod() are not limited to 
adding just Python based implementations to classes.  You can also 
implement a method in Objective-C and use classAddMethods() to add the 
method to some random class for which you do not have the header, 
something that cannot be done with Categories without writing some 
gnarly code (see the implementation of classAddMethod* for an example 
of the gnarly code).

An example:

bbum% python
 >>> from Foundation import *
 >>> from objc import classAddMethods
 >>> x = NSObject.new()
 >>> x.foo()
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
AttributeError: 'NSObject' object has no attribute 'foo'
 >>> x.description()
u'<NSObject: 0x35ce70>'
 >>> def foo(self):
...   print "bar"
...
 >>> def description(self):
...   print "Hello", id(self)
...
 >>> classAddMethods(NSObject, [foo, description])
 >>> x.description()
Hello 7988048
 >>> x.foo()
bar
 >>>

Thank you very much Bill for the cool example - I can see this is not
difficult at all! :-)

Ben

On Wed, 28 Jul 2004 11:26:12 -0700, b.bum <bb...@ma...> wrote:
> An example:
> 
> bbum% python
>  >>> from Foundation import *
>  >>> from objc import classAddMethods
>  >>> x = NSObject.new()
>  >>> x.foo()
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
> AttributeError: 'NSObject' object has no attribute 'foo'
>  >>> x.description()
> u'<NSObject: 0x35ce70>'
>  >>> def foo(self):
> ....   print "bar"
> ....
>  >>> def description(self):
> ....   print "Hello", id(self)
> ....
>  >>> classAddMethods(NSObject, [foo, description])
>  >>> x.description()
> Hello 7988048
>  >>> x.foo()
> bar
>  >>>
> 
>

On Jul 28, 2004, at 2:26 PM, b.bum wrote:

> On Jul 28, 2004, at 11:16 AM, Bob Ippolito wrote:
>> On Jul 28, 2004, at 2:08 PM, Fortepianissimo wrote:
>>> Ok another newbie question: how do I write category code using 
>>> PyObjC?
>>> Searched documentation but didn't find anything...
>>
>> Painfully, with objc.classAddMethods(...).  I'd suggest not doing it 
>> at all unless you absolutely have to, until we have a better syntax 
>> for it (a metaclass mixin or something).
>
> How about something like the following.   I like the second one better.
>
> class NSObject(PyObjCAsCategory):
> 	pass
>
> Or:
>
> class MyCategory(ObjCCategory, NSObject):
> 	pass

I would say (knowing that the implementation is more flexible and 
easier like this):

class NSObject(objc.Category(NSObject)):
	def methodWithArgument_(self, arg):
		pass

The "return value" of the class statement will be the same instance of 
NSObject as that was passed in.

> objc.classAddMethods() serves a different purpose than categories.  It 
> allows the developer to edit the live runtime as opposed to having to 
> the formality of declaration required by traditional categories.   It 
> can do everything that categories can do, as well.  There is also 
> classAddMethod(cls, name, method) which can be used to add a single 
> method to an ObjC class with any random selector as the name (as long 
> as the selector's arg count matches the functions arg count).

Yeah but the ObjC runtime treats them the same, doesn't it?  I don't 
recall seeing any category registry..

-bob

On Jul 28, 2004, at 11:47 AM, Bob Ippolito wrote:
> Yeah but the ObjC runtime treats them the same, doesn't it?  I don't 
> recall seeing any category registry..

You can't add a category to a class without have the declaration of the 
class at compile time, including a size accurate representation of the 
instance variables.

Cateogories aren't really registered, but they can be detected within 
the isa pointer's method table.

Also, addClassMethod*() allows you to add a single implementation to 
multiple classes without having to redeclare and/or recompile, 
something that is not possible with compiled categories.

b.bum

On 28-jul-04, at 20:47, Bob Ippolito wrote:
> I would say (knowing that the implementation is more flexible and 
> easier like this):
>
> class NSObject(objc.Category(NSObject)):
> 	def methodWithArgument_(self, arg):
> 		pass

I like that much more than the other proposals, it makes if very clear 
that something fishy is going on.

Ronald
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

On Jul 28, 2004, at 12:31 PM, Ronald Oussoren wrote:
>> class NSObject(objc.Category(NSObject)):
>> 	def methodWithArgument_(self, arg):
>> 		pass
>
> I like that much more than the other proposals, it makes if very clear 
> that something fishy is going on.

+1

On Jul 28, 2004, at 4:18 PM, b.bum wrote:

> On Jul 28, 2004, at 12:31 PM, Ronald Oussoren wrote:
>>> class NSObject(objc.Category(NSObject)):
>>> 	def methodWithArgument_(self, arg):
>>> 		pass
>>
>> I like that much more than the other proposals, it makes if very 
>> clear that something fishy is going on.
>
> +1

It might also be interesting to have a module-level statement like this:

__metaclass__ = objc.Development

Which could mean that classes can be multiply defined as long as they 
have the same ivars (which should be the case 99% of the time given 
that "pure" PyObjC classes only need an ivar for a PyObject*).  As far 
as the implementation goes, it would check to see if the class is 
already in the runtime, and if it is, it would simply be treated as a 
category instead of a new class.

This would mean that reload(PyObjCModule) would do the right thing.

-bob

On 28-jul-04, at 20:47, Bob Ippolito wrote:
>
> I would say (knowing that the implementation is more flexible and 
> easier like this):
>
> class NSObject(objc.Category(NSObject)):
> 	def methodWithArgument_(self, arg):
> 		pass

Closing the lists seems to have broken checkin messages. Hence a manual 
heads-up: this code now actually works.

It would also be possible to get this working:

	class NSObject (objc.Category):
		pass

But: I don't like that, looking up the classname in objc.runtime is a 
bit too magic for me. The current version is also clearer.

Ronald
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

On 28-jul-04, at 20:26, b.bum wrote:

> On Jul 28, 2004, at 11:16 AM, Bob Ippolito wrote:
>> On Jul 28, 2004, at 2:08 PM, Fortepianissimo wrote:
>>> Ok another newbie question: how do I write category code using 
>>> PyObjC?
>>> Searched documentation but didn't find anything...
>>
>> Painfully, with objc.classAddMethods(...).  I'd suggest not doing it 
>> at all unless you absolutely have to, until we have a better syntax 
>> for it (a metaclass mixin or something).
>
> How about something like the following.   I like the second one better.
>
> class NSObject(PyObjCAsCategory):
> 	pass
>
> Or:
>
> class MyCategory(ObjCCategory, NSObject):
> 	pass

class MyCategory (objc.category, NSObject):
	pass

I don't really like this syntax, the decorator syntax would be better 
(when they ever get around choosing and implementing one...) but that 
would have the wrong semantics:

[ objc.category ]
class MyCategory (NSObject):
	pass

or

@objc.category
class MyCategory (NSObject):
	pass

BTW. If anyone wants to have a decorator a lot: Phillip J. Eby  has 
written an implementation of the first syntax in pure python, it seems 
to be part of PEAK.
>
> ---
>
> objc.classAddMethods() serves a different purpose than categories.

classAddMethods is as close to categories as we get :-). It's more 
powerful than objective-C categories, but that can't be helped.

Ronald
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

On Jul 28, 2004, at 3:28 PM, Ronald Oussoren wrote:

>
> On 28-jul-04, at 20:26, b.bum wrote:
>
>> On Jul 28, 2004, at 11:16 AM, Bob Ippolito wrote:
>>> On Jul 28, 2004, at 2:08 PM, Fortepianissimo wrote:
>>>> Ok another newbie question: how do I write category code using 
>>>> PyObjC?
>>>> Searched documentation but didn't find anything...
>>>
>>> Painfully, with objc.classAddMethods(...).  I'd suggest not doing it 
>>> at all unless you absolutely have to, until we have a better syntax 
>>> for it (a metaclass mixin or something).
>>
>> How about something like the following.   I like the second one 
>> better.
>>
>> class NSObject(PyObjCAsCategory):
>> 	pass
>>
>> Or:
>>
>> class MyCategory(ObjCCategory, NSObject):
>> 	pass
>
> class MyCategory (objc.category, NSObject):
> 	pass
>
> I don't really like this syntax, the decorator syntax would be better 
> (when they ever get around choosing and implementing one...) but that 
> would have the wrong semantics:
>
> [ objc.category ]
> class MyCategory (NSObject):
> 	pass
>
> or
>
> @objc.category
> class MyCategory (NSObject):
> 	pass

Pretty sure that we won't get class decorators in 2.4, if we get 
decorators at all.

>
> BTW. If anyone wants to have a decorator a lot: Phillip J. Eby  has 
> written an implementation of the first syntax in pure python, it seems 
> to be part of PEAK.

I think it's in PyProtocols.  Either way, I want to put this in PyObjC 
sooner or later as an alternative to objc.selector.  I'm sort of 
waiting to see what happens with decorators in 2.4 before I commit any 
code or examples.

-bob

On Wed, 28 Jul 2004 14:16:24 -0400, Bob Ippolito <bo...@re...> wrote:
> On Jul 28, 2004, at 2:08 PM, Fortepianissimo wrote:
> 
> > Ok another newbie question: how do I write category code using PyObjC?
> > Searched documentation but didn't find anything...
> 
> Painfully, with objc.classAddMethods(...).  I'd suggest not doing it at
> all unless you absolutely have to, until we have a better syntax for it
> (a metaclass mixin or something).

I guess what I want is a bit more than this: I want to not only
replace a method of a class with my own, but also retain the old one
so that I can call it after I'm done with my version of the method. To
be more specific, for the mail plugin I wrote I have a MessageRule
category implemented as following:

--- CUT HERE ---
#import <objc/objc-class.h>
#import <Message.h>
#import <MessageHeaders.h>

#import "MessageRule-JM.h"

typedef void (*PerformActionsOnMessagesIMP)(id, SEL, id, id, id, id);

@implementation MessageRule (JM)

static PerformActionsOnMessagesIMP _oldIMP = NULL;

+ (void)load  {
	Method oldMethod, newMethod;

	oldMethod = class_getInstanceMethod(self,
@selector(performActionsOnMessages:destinationStores:rejectedMessages:messagesToBeDeleted:));
	newMethod = class_getInstanceMethod(self,
@selector(myPerformActionsOnMessages:destinationStores:rejectedMessages:messagesToBeDeleted:));

	// replace performActionsOnMessages:destinationStores:rejectedMessages:messagesToBeDeleted:
with my version
	if(NULL != oldMethod && NULL != newMethod) {		
		_oldIMP = (PerformActionsOnMessagesIMP) oldMethod->method_imp;
		oldMethod->method_imp = newMethod->method_imp;
	}
}

- (void)myPerformActionsOnMessages:(id)messages
destinationStores:(id)fp12 rejectedMessages:(id)fp16
messagesToBeDeleted:(id)fp20 {
	if ([[self ruleName] isEqualToString:@"TriggerRule"]) {	
		NSEnumerator *enumerator = [messages objectEnumerator];
		Message *aMessage;
		NSData *msgData;
		NSString *msgID;

		while (aMessage = [enumerator nextObject]) {			
			msgData = [aMessage messageDataIncludingFromSpace:NO];
			msgID = [aMessage messageID];

            // TO-DO: do something about the message data!
		}
	}

	(*_oldIMP)(self,_cmd,messages,fp12,fp16,fp20);
}

@end
--- CUT HERE ---

The basic idea is to intercept the email raw data when it hits a rule
with a certain name ("TriggerRule"). I did that in my version of
-PerformActionsOnMessages:... and at the end of it I make a call to
the original version.

Now my task is (deep breath): rewrite this in Python with PyObjC! The
parts I don't know how to write:

1. How can I make a call to this function: class_getInstanceMethod() ?
2. Not to mention now we have pointers?

So I guess it's not possible to do this unless I'm prepared to wade
into the business of wrapping some C stuff for Python?

Or, there is some other workaround with the same desired effect?
(in the doc of objc.classAddMethods() it says if the class already has
implementation of the added methods, they'll be *replaced* - the key
is to keep them around so I can call them after my stuff)

Thank you!

On Jul 28, 2004, at 9:56 PM, Fortepianissimo wrote:
> Or, there is some other workaround with the same desired effect?
> (in the doc of objc.classAddMethods() it says if the class already has
> implementation of the added methods, they'll be *replaced* - the key
> is to keep them around so I can call them after my stuff)

Ideally, you would want to be able to do this:

 >>> oldDescription = NSObject.description
 >>> def description(self):
...   print "Hello ", oldDescription(self)
...
 >>> x.description()
u'<NSObject: 0x374670>'
 >>> classAddMethods(NSObject, [description])
 >>> x.description()
<<<<infinite loop>>>>

Unfortunately, that results in an infinite loop because oldDescription 
appears to not be tightly bound to the IMP.   This really isn't a bug 
as much as a nice to have feature.

ObjC does have....

- (IMP)methodForSelector:(SEL)aSelector;
+ (IMP)instanceMethodForSelector:(SEL)aSelector;

.... but they aren't really bridged correctly, I don't believe.  It 
would be nice if they were.

Oh, wait, Guido's time machine strikes again.  How amazingly cool!

 >>> oldDescription = NSObject.instanceMethodForSelector_("description")
 >>> oldDescription
<IMP description at 0x92cc0>
 >>> def description(self):
...   print "Hello", oldDescription(self)
...
 >>> x = NSObject.new()
 >>> x.description()
u'<NSObject: 0x3750b0>'
 >>> classAddMethods(NSObject, [description])
 >>> x.description()
Hello <NSObject: 0x3750b0>

b.bum

On Wed, 28 Jul 2004 23:00:29 -0700, b.bum <bb...@ma...> wrote:
> On Jul 28, 2004, at 9:56 PM, Fortepianissimo wrote:
> > Or, there is some other workaround with the same desired effect?
> > (in the doc of objc.classAddMethods() it says if the class already has
> > implementation of the added methods, they'll be *replaced* - the key
> > is to keep them around so I can call them after my stuff)

>  >>> oldDescription = NSObject.instanceMethodForSelector_("description")
>  >>> oldDescription
> <IMP description at 0x92cc0>
>  >>> def description(self):
> ....   print "Hello", oldDescription(self)
> ....
>  >>> x = NSObject.new()
>  >>> x.description()
> u'<NSObject: 0x3750b0>'
>  >>> classAddMethods(NSObject, [description])
>  >>> x.description()
> Hello <NSObject: 0x3750b0>

I almost wanted to kiss you - good news is it almost worked, bad news
is it didn't. When clicking on "Apply Rules" in Mail on an email it
just kept showing "Performing rule actions..." with a constantly
running progress indicator.

My guess is there is some deadlock since rule actions are usually
engaged in a multi-threaded fashion. In other words, imagine your
description() being run in multiple threads. Could it be the dreaded
GIL? What is the usual tactic when dealing with multiple threads in
PyObjC?

On Thu, 29 Jul 2004 03:08:47 -0400, Fortepianissimo
<for...@gm...> wrote:
> 
> 
> On Wed, 28 Jul 2004 23:00:29 -0700, b.bum <bb...@ma...> wrote:
> > On Jul 28, 2004, at 9:56 PM, Fortepianissimo wrote:
> > > Or, there is some other workaround with the same desired effect?
> > > (in the doc of objc.classAddMethods() it says if the class already has
> > > implementation of the added methods, they'll be *replaced* - the key
> > > is to keep them around so I can call them after my stuff)
> 
> >  >>> oldDescription = NSObject.instanceMethodForSelector_("description")
> >  >>> oldDescription
> > <IMP description at 0x92cc0>
> >  >>> def description(self):
> > ....   print "Hello", oldDescription(self)
> > ....
> >  >>> x = NSObject.new()
> >  >>> x.description()
> > u'<NSObject: 0x3750b0>'
> >  >>> classAddMethods(NSObject, [description])
> >  >>> x.description()
> > Hello <NSObject: 0x3750b0>
> 
> I almost wanted to kiss you - good news is it almost worked, bad news
> is it didn't. When clicking on "Apply Rules" in Mail on an email it
> just kept showing "Performing rule actions..." with a constantly
> running progress indicator.
> 
> My guess is there is some deadlock since rule actions are usually
> engaged in a multi-threaded fashion. In other words, imagine your
> description() being run in multiple threads. Could it be the dreaded
> GIL? What is the usual tactic when dealing with multiple threads in
> PyObjC?

I found this tidbit in the documentation:

"You must also make sure that Objective-C only makes calls to Python
from a thread that owns the Python GIL (that's also the reason for not
being able to use NSThread to create new threads). This restriction
will be lifted in a future version of PyObjC (at least when using
Python 2.3)."

Is it still a problem? If so, is there any workaround? (transfer the
GIL to the correct thread?)

On 29-jul-04, at 15:14, Fortepianissimo wrote:

> On Thu, 29 Jul 2004 03:08:47 -0400, Fortepianissimo
> <for...@gm...> wrote:
>>
>>
>> On Wed, 28 Jul 2004 23:00:29 -0700, b.bum <bb...@ma...> wrote:
>>> On Jul 28, 2004, at 9:56 PM, Fortepianissimo wrote:
>>>> Or, there is some other workaround with the same desired effect?
>>>> (in the doc of objc.classAddMethods() it says if the class already 
>>>> has
>>>> implementation of the added methods, they'll be *replaced* - the key
>>>> is to keep them around so I can call them after my stuff)
>>
>>>>>> oldDescription = 
>>>>>> NSObject.instanceMethodForSelector_("description")
>>>>>> oldDescription
>>> <IMP description at 0x92cc0>
>>>>>> def description(self):
>>> ....   print "Hello", oldDescription(self)
>>> ....
>>>>>> x = NSObject.new()
>>>>>> x.description()
>>> u'<NSObject: 0x3750b0>'
>>>>>> classAddMethods(NSObject, [description])
>>>>>> x.description()
>>> Hello <NSObject: 0x3750b0>
>>
>> I almost wanted to kiss you - good news is it almost worked, bad news
>> is it didn't. When clicking on "Apply Rules" in Mail on an email it
>> just kept showing "Performing rule actions..." with a constantly
>> running progress indicator.
>>
>> My guess is there is some deadlock since rule actions are usually
>> engaged in a multi-threaded fashion. In other words, imagine your
>> description() being run in multiple threads. Could it be the dreaded
>> GIL? What is the usual tactic when dealing with multiple threads in
>> PyObjC?
>
> I found this tidbit in the documentation:
>
> "You must also make sure that Objective-C only makes calls to Python
> from a thread that owns the Python GIL (that's also the reason for not
> being able to use NSThread to create new threads). This restriction
> will be lifted in a future version of PyObjC (at least when using
> Python 2.3)."
>
> Is it still a problem? If so, is there any workaround? (transfer the
> GIL to the correct thread?)

The documentation is not up-to-date regarding this issue. PyObjC should 
be completely thread-safe.

Do you have a small example that demonstrates the problem?

Ronald
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

Hi folks,

On Thu, 29 Jul 2004 20:53:32 +0200, Ronald Oussoren
<ron...@ma...> wrote:
> On 29-jul-04, at 15:14, Fortepianissimo wrote:
> 
> > On Thu, 29 Jul 2004 03:08:47 -0400, Fortepianissimo
> > <for...@gm...> wrote:
> >>
> >>
> >> On Wed, 28 Jul 2004 23:00:29 -0700, b.bum <bb...@ma...> wrote:
> >>> On Jul 28, 2004, at 9:56 PM, Fortepianissimo wrote:
> >>>> Or, there is some other workaround with the same desired effect?
> >>>> (in the doc of objc.classAddMethods() it says if the class already
> >>>> has
> >>>> implementation of the added methods, they'll be *replaced* - the key
> >>>> is to keep them around so I can call them after my stuff)
> >>
> >>>>>> oldDescription =
> >>>>>> NSObject.instanceMethodForSelector_("description")
> >>>>>> oldDescription
> >>> <IMP description at 0x92cc0>
> >>>>>> def description(self):
> >>> ....   print "Hello", oldDescription(self)
> >>> ....
> >>>>>> x = NSObject.new()
> >>>>>> x.description()
> >>> u'<NSObject: 0x3750b0>'
> >>>>>> classAddMethods(NSObject, [description])
> >>>>>> x.description()
> >>> Hello <NSObject: 0x3750b0>
> >>
> >> I almost wanted to kiss you - good news is it almost worked, bad news
> >> is it didn't. When clicking on "Apply Rules" in Mail on an email it
> >> just kept showing "Performing rule actions..." with a constantly
> >> running progress indicator.
> >>
> >> My guess is there is some deadlock since rule actions are usually
> >> engaged in a multi-threaded fashion. In other words, imagine your
> >> description() being run in multiple threads. Could it be the dreaded
> >> GIL? What is the usual tactic when dealing with multiple threads in
> >> PyObjC?
> >
> > I found this tidbit in the documentation:
> >
> > "You must also make sure that Objective-C only makes calls to Python
> > from a thread that owns the Python GIL (that's also the reason for not
> > being able to use NSThread to create new threads). This restriction
> > will be lifted in a future version of PyObjC (at least when using
> > Python 2.3)."
> >
> > Is it still a problem? If so, is there any workaround? (transfer the
> > GIL to the correct thread?)
> 
> The documentation is not up-to-date regarding this issue. PyObjC should
> be completely thread-safe.
> 
> Do you have a small example that demonstrates the problem?

Here is the complete code of a very simple mail bundle - please drop
the built .mailbundle to your Library/Mail/Bundles folder (quit Mail
first of course):

--- main.py STARTS ---
#!/usr/bin/python

import sys

from AppKit import *
from Foundation import *
import objc

MessageRuleClass = objc.runtime.MessageRule
oldPerformActionsOnMessages_destinationStores_rejectedMessages_messagesToBeDeleted_
= MessageRuleClass.instanceMethodForSelector_("performActionsOnMessages:destinationStores:rejectedMessages:messagesToBeDeleted:")

def performActionsOnMessages_destinationStores_rejectedMessages_messagesToBeDeleted_
(self, messages, stores, rejectedMessages, messagesToBeDeleted):
    NSLog('yes')
    oldPerformActionsOnMessages_destinationStores_rejectedMessages_messagesToBeDeleted_(self,
messages, stores, rejectedMessages, messagesToBeDeleted)

objc.classAddMethods(MessageRuleClass,
[performActionsOnMessages_destinationStores_rejectedMessages_messagesToBeDeleted_])

class ToyMailBundle2 (objc.runtime.MVMailBundle):
    def applicationWillTerminate_ (self, notification):
        NSLog('ToyMailBundle2 is shutting down')
        NSNotificationCenter.defaultCenter().removeObserver_name_object_(self,
None, None)

    def init (self):
        self = super(ToyMailBundle2, self).init()
        if self:
            NSLog('ToyMailBundle2 -init called')
            NSNotificationCenter.defaultCenter().addObserver_selector_name_object_(self,

            'applicationWillTerminate:',

            NSApplicationWillTerminateNotification,

            None)
        return self

    def initialize (cls):
        cls.registerBundle()
        NSLog('ToyMailBundle2 registered with Mail')        

    initialize = classmethod(initialize)
--- main.py ENDS ---

--- buildplugin.py STARTS ---
from PyObjCTools.pluginbuilder import buildplugin

buildplugin(
    name = "ToyMailBundle2",
    mainmodule = "main.py",
    nibname = "ToyMailBundle2",
    principalClass = "ToyMailBundle2",
    resources = [],
    bundlesuffix = ".mailbundle",
)
--- buildplugin.py ENDS ---

WARNING: once you click "Apply Rules" on an email, Mail will be stuck
at "Performing rule actions...", but the UI is still responsive (so
the main thread is not jeopardized). You won't be able to quit Mail
(have to force quit it).

On 29-jul-04, at 21:17, Fortepianissimo wrote:

>>
>
> Here is the complete code of a very simple mail bundle - please drop
> the built .mailbundle to your Library/Mail/Bundles folder (quit Mail
> first of course):

I've added the plugin to my ~/Library/Mail/Bundles directory and yet 
nothing happens:

$ ls ~/Library/Mail/Bundles
ToyMailBundle2.mailbundle

When I load mail.app nothing gets logged, even when I add a print 
statement to main.py.

What I did:
- stop mail.app
$ mkdir ~/Library/Mail/Bundles
$ python buildplugin.py build
$ mv build/ToyMailBundle2.mailbundle ~/Library/Mail/Bundles
- start mail.app

... later...  the installation instructions for GPGMail mention: 
defaults write com.apple.mail EnableBundles YES

And sure enough something happens: Mail.app shows a pop-up that 
ToyMailBundle is not compatible with this version of mail.

That was easily solved: add an CFBundleGetInfoString that contains 
'(v37, 10.3)' as part of it's value, and then reenable plugins because 
mail.app reverted the EnableBundles default.

Some observations:
- The method your replacing has a simple signature (returns void and 
all arguments are objects), nothing special there
- The last message from the plugin is '-init called', the NSLog('yes') 
is never called.
- "performing rule actions" did go away after some time after I 
selected "Apply Rules" on a selection of 3 mails that should get them 
copied elsewhere (stupidly enough I forgot to check if they did get 
copied...). The second time I applied the rules "performing rule 
actions" did *not* go away and I had to force-quit mail.app
- The same thing happens when I add an explicit definition of the 
signature for the replacement method (again nothing is logged after the 
-init message)

I'm running 10.3.4 and a version of PyObjC from SVN.

Interestingly enough, when I manually load the bundle (b = 
NSBundle.bundleWithPath_('...'); b.load()) python crashes with a fatal 
error. It shouldn't do that :-)

... even later ...

I think I know what's happening: the ObjC code that is generated by 
pluginbuilder.py doesn't play nice with the multi-threading 
enhancements in PyObjC 1.1. I'll see if I can fix this tomorrow, it 
should be easy enough.

Ronald
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

On Thu, 29 Jul 2004 22:14:29 +0200, Ronald Oussoren
<ron...@ma...> wrote:
> 
> On 29-jul-04, at 21:17, Fortepianissimo wrote:
> 
> >>
> >
> > Here is the complete code of a very simple mail bundle - please drop
> > the built .mailbundle to your Library/Mail/Bundles folder (quit Mail
> > first of course):
> 

> 
> .... later...  the installation instructions for GPGMail mention:
> defaults write com.apple.mail EnableBundles YES
> 
> And sure enough something happens: Mail.app shows a pop-up that
> ToyMailBundle is not compatible with this version of mail.
> 
> That was easily solved: add an CFBundleGetInfoString that contains
> '(v37, 10.3)' as part of it's value, and then reenable plugins because
> mail.app reverted the EnableBundles default.

I see how I conveniently forgot what I did a while ago: add these two
puppies in com.apple.mail.plist in ~/Library/Preferences:

BundleCompatibilityVersion -> Number "1"
EnableBundles -> Boolean "Yes"

> 
> Some observations:
> - The method your replacing has a simple signature (returns void and
> all arguments are objects), nothing special there
> - The last message from the plugin is '-init called', the NSLog('yes')
> is never called.
> - "performing rule actions" did go away after some time after I
> selected "Apply Rules" on a selection of 3 mails that should get them
> copied elsewhere (stupidly enough I forgot to check if they did get
> copied...). The second time I applied the rules "performing rule
> actions" did *not* go away and I had to force-quit mail.app
> - The same thing happens when I add an explicit definition of the
> signature for the replacement method (again nothing is logged after the
> -init message)
> 
> I'm running 10.3.4 and a version of PyObjC from SVN.
> 
> Interestingly enough, when I manually load the bundle (b =
> NSBundle.bundleWithPath_('...'); b.load()) python crashes with a fatal
> error. It shouldn't do that :-)
> 
> .... even later ...
> 
> I think I know what's happening: the ObjC code that is generated by
> pluginbuilder.py doesn't play nice with the multi-threading
> enhancements in PyObjC 1.1. I'll see if I can fix this tomorrow, it
> should be easy enough.

Yess!!! If you could do that I will be *very* grateful.

One thing I'm still not clear: when the hijacked method got called,
who owns the GIL? Is that the thread that invokes the said method? If
not, how do you solve the GIL problem? (in high-level terms)

Thank you so much!

On 29-jul-04, at 22:40, Fortepianissimo wrote:
>>
>>
>> I think I know what's happening: the ObjC code that is generated by
>> pluginbuilder.py doesn't play nice with the multi-threading
>> enhancements in PyObjC 1.1. I'll see if I can fix this tomorrow, it
>> should be easy enough.
>
> Yess!!! If you could do that I will be *very* grateful.
>
> One thing I'm still not clear: when the hijacked method got called,
> who owns the GIL? Is that the thread that invokes the said method? If
> not, how do you solve the GIL problem? (in high-level terms)
>
> Thank you so much!
>

What happens is plain python PyObjC programs is that the bridge gives 
up the GIL when it calls into Objective-C. When we get called from 
Objective-C the first thing we do is acquire the GIL, the GIL gets 
released again when we return to ObjC. The C code in pluginbuilder.py 
should give up the GIL before it returns and should also acquire the 
GIL when it doesn't create a new Python interpreter. That way the GIL 
is unlocked while in ObjC code and locked while in Python code.

BTW. The same problem should be present in *any* program where the main 
event loop isn't started from Python.
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

On 30-jul-04, at 6:50, Ronald Oussoren wrote:

>
> On 29-jul-04, at 22:40, Fortepianissimo wrote:
>>>
>>>
>>> I think I know what's happening: the ObjC code that is generated by
>>> pluginbuilder.py doesn't play nice with the multi-threading
>>> enhancements in PyObjC 1.1. I'll see if I can fix this tomorrow, it
>>> should be easy enough.
>>
>> Yess!!! If you could do that I will be *very* grateful.
>>
>> One thing I'm still not clear: when the hijacked method got called,
>> who owns the GIL? Is that the thread that invokes the said method? If
>> not, how do you solve the GIL problem? (in high-level terms)
>>
>> Thank you so much!
>>
>
> What happens is plain python PyObjC programs is that the bridge gives 
> up the GIL when it calls into Objective-C. When we get called from 
> Objective-C the first thing we do is acquire the GIL, the GIL gets 
> released again when we return to ObjC. The C code in pluginbuilder.py 
> should give up the GIL before it returns and should also acquire the 
> GIL when it doesn't create a new Python interpreter. That way the GIL 
> is unlocked while in ObjC code and locked while in Python code.

I did find a way to release the GIL, but now Mail.app just crashes hard 
after the first call to the replaced method. This feels like refence 
count problem, yuck.

Ronald
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

On Fri, 30 Jul 2004 22:55:29 +0200, Ronald Oussoren
<ron...@ma...> wrote:
> 
> On 30-jul-04, at 6:50, Ronald Oussoren wrote:
> 
> >
> > What happens is plain python PyObjC programs is that the bridge gives
> > up the GIL when it calls into Objective-C. When we get called from
> > Objective-C the first thing we do is acquire the GIL, the GIL gets
> > released again when we return to ObjC. The C code in pluginbuilder.py
> > should give up the GIL before it returns and should also acquire the
> > GIL when it doesn't create a new Python interpreter. That way the GIL
> > is unlocked while in ObjC code and locked while in Python code.
> 
> I did find a way to release the GIL, but now Mail.app just crashes hard
> after the first call to the replaced method. This feels like refence
> count problem, yuck.
> 

Sorry to bug you about this Ron, but did you find any way to work
around that? Or should I assume it should be solved "Real Soon Now" ?

On 3-aug-04, at 5:41, Fortepianissimo wrote:

> On Fri, 30 Jul 2004 22:55:29 +0200, Ronald Oussoren
> <ron...@ma...> wrote:
>>
>> On 30-jul-04, at 6:50, Ronald Oussoren wrote:
>>
>>>
>>> What happens is plain python PyObjC programs is that the bridge gives
>>> up the GIL when it calls into Objective-C. When we get called from
>>> Objective-C the first thing we do is acquire the GIL, the GIL gets
>>> released again when we return to ObjC. The C code in pluginbuilder.py
>>> should give up the GIL before it returns and should also acquire the
>>> GIL when it doesn't create a new Python interpreter. That way the GIL
>>> is unlocked while in ObjC code and locked while in Python code.
>>
>> I did find a way to release the GIL, but now Mail.app just crashes 
>> hard
>> after the first call to the replaced method. This feels like refence
>> count problem, yuck.
>>
>
> Sorry to bug you about this Ron, but did you find any way to work
> around that? Or should I assume it should be solved "Real Soon Now" ?

Not yet. I don't like debugging undocumented API's.

I'm hope to have some time to work on this tonight. Until I find out 
what's wrong I've no idea how long it will take to fix this.

Ronald
>
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173

On 3-aug-04, at 5:41, Fortepianissimo wrote:

> On Fri, 30 Jul 2004 22:55:29 +0200, Ronald Oussoren
> <ron...@ma...> wrote:
>>
>> On 30-jul-04, at 6:50, Ronald Oussoren wrote:
>>
>>>
>>> What happens is plain python PyObjC programs is that the bridge gives
>>> up the GIL when it calls into Objective-C. When we get called from
>>> Objective-C the first thing we do is acquire the GIL, the GIL gets
>>> released again when we return to ObjC. The C code in pluginbuilder.py
>>> should give up the GIL before it returns and should also acquire the
>>> GIL when it doesn't create a new Python interpreter. That way the GIL
>>> is unlocked while in ObjC code and locked while in Python code.
>>
>> I did find a way to release the GIL, but now Mail.app just crashes  
>> hard
>> after the first call to the replaced method. This feels like refence
>> count problem, yuck.
>>
>
> Sorry to bug you about this Ron, but did you find any way to work
> around that? Or should I assume it should be solved "Real Soon Now" ?

I think I found where the code is crashing, but I'm not sure is why it  
happens. What I am sure about is that fixing this bug will introduce an  
infinite loop in the example you posted earlier :-(

At the time  
'oldPerformActionsOnMessages_destinationStores_rejectedMessages_messages 
ToBeDeleted_' is created  
'-performActionsOnMessages:destinationStores:rejectedMessages: 
messagesToBeDeleted:' seems to be _objc_msgForward, and not a real  
method. If I fix the crash this will end up calling your implemention  
of  
'-performActionsOnMessages:destinationStores:rejectedMessages: 
messagesToBeDeleted:', which will call _objc_msgForward again...

A suffciently patched version of your example and PyObjC prints:

2004-08-03 19:45:07.017 Mail[14182] ToyMailBundle2 -init called
2004-08-03 19:45:07.021 Mail[14182] ToyMailBundle2 registered with Mail
2004-08-03 19:45:26.744 Mail[14182] yes
<IMP  
performActionsOnMessages:destinationStores:rejectedMessages: 
messagesToBeDeleted: at 0x772c60 for 0x90836760>
2004-08-03 19:45:26.744 Mail[14182] SEL:  
performActionsOnMessages:destinationStores:rejectedMessages: 
messagesToBeDeleted: SELF: Rule <0x464cb0>: List: python-dev
         Criterion:List-ID
         Qualifier:(null)
         Expression:python-dev.python.org
2004-08-03 19:45:26.746 Mail[14182] IMP 0x90836760

The IMP the function that will be passed to libffi.

The crash dump contains (among lots of other stuff):

Thread 6 Crashed:
0   libobjc.A.dylib                     0x908311ec objc_msgSend + 0xc
1   libobjc.A.dylib                     0x90836810 _objc_msgForward +  
0xb0
2   _objc.so                            0x005e0bb8 ffi_call_DARWIN +  
0xd0
3   _objc.so                            0x005e05ac ffi_call + 0xc4  
(ffi_darwin.c:403)
4   _objc.so                            0x005da74c PyObjCFFI_Caller +  
0x1134 (libffi_support.m:1057)
5   _objc.so                            0x005de288 imp_call + 0x19c  
(method-imp.m:124)
6   org.python.Python.framework         0x95f4a8d0 PyObject_Call + 0x30
7   org.python.Python.framework         0x95fa9ba8 PyEval_GetFuncDesc +  
0x8dc
8   org.python.Python.framework         0x95fa9598 PyEval_GetFuncDesc +  
0x2cc
9   org.python.Python.framework         0x95fa6c64 PyEval_EvalCode +  
0x2560
10  org.python.Python.framework         0x95fa7e30 PyEval_EvalCodeEx +  
0x850
11  org.python.Python.framework         0x95f5f354  
PyFunction_SetClosure + 0xd6c
12  org.python.Python.framework         0x95f4a8d0 PyObject_Call + 0x30
13  _objc.so                            0x005d8b34 method_stub + 0x524  
(libffi_support.m:447)
14  _objc.so                            0x005e08f0  
ffi_closure_helper_DARWIN + 0x1e0 (ffi_darwin.c:701)
15  _objc.so                            0x005e0c84 ffi_closure_ASM +  
0x74
16  com.apple.MessageFramework          0x867dfc34  
_routeMessagesInDictionary + 0xa8
17  com.apple.MessageFramework          0x867df450 -[MessageRouter  
routeMessages:fromStores:] + 0x670

Note that IMP (0x90836760) is quite near the address on the line with   
_objc_msgForward (0x90836810), the difference is exacly 0xb0, which  
means IMP points to the start of _objc_msgForward.

Ronald
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by OSTG. Have you noticed the changes on
> Linux.com, ITManagersJournal and NewsForge in the past few weeks? Now,
> one more big change to announce. We are now OSTG- Open Source  
> Technology
> Group. Come see the changes on the new OSTG site. www.ostg.com
> _______________________________________________
> Pyobjc-dev mailing list
> Pyo...@li...
> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev
>
--
X|support bv            http://www.xsupport.nl/
T:  +31 610271479       F:  +31 204416173