pyobjc-dev

[Pyobjc-dev] TypeError: cannot change a method

[Daniel M. <mil...@gm...>]

My testing framework for the app I've been developing (previously with  
PyObjC 1.4, now attempting to migrate to PyObjC 2) is giving lots of  
errors due to a restriction in PyObjC 2 that methods cannot be  
overwritten [0]. My testing framework mocks Cocoa API calls to verify  
that the calls are being made as expected without actually hitting  
Cocoa (which would cause all sorts of unwanted side-affects). My first  
question is about circumventing this read-only method restriction for  
my tests. Is it possible to (temporarily?) make PyObjC methods  
writable? Maybe through some hack that would only be activated as a  
pre-test setup task, allowing me to overwrite methods just during the  
testing phase?

My second question is why such a restriction has been added? I'm not  
trying to start a flame war here, so please don't get that idea.  
Python is generally very liberal in what it allows one to do, even  
potentially damaging things such as shooting ones self in the foot:

"Python culture tends towards "we're all consenting adults here". If you
attempt to shoot yourself in the foot, you should get some kind of
warning that perhaps it is not what you really want to do, but if you
insist, hey, go ahead, it's your foot!" [1]

"...this is Python, where we vigorously defend the right to shoot  
ourselves in the foot." [2]

  Is there any particular technical reason why PyObjC tries harder- 
than-normal-for-Python to protect me from doing things that most  
people probably do not want to do? Thanks in advance for taking time  
to answer my questions.

~ Daniel

[0] http://www.mail-archive.com/mat...@li.../msg05035.html
[1] http://mail.python.org/pipermail/tutor/2006-December/051569.html
[2] http://www.gossamer-threads.com/lists/python/python/753888#753888

Re: [Pyobjc-dev] TypeError: cannot change a method

[Ronald O. <ron...@ma...>]

Could you post some code that shows what you're trying to do?

I haven't checked the source code yet, but one reason to be careful  
about overwriting methods in Cocoa classes is that will affect the  
actual Cocoa class as wel.

Footnote [0] refers to replacing a method in an instance by something  
else, e.g.:

      o = NSObject.alloc().init()
      o.description = 42

That's not allowed because this can lead to very hard to diagnose  
bugs, and the people that are most likely to run into those are people  
new to PyObjC that translate existing ObjC sample code into Python  
(the reason of that is that ObjC has different namespaces for methods  
and instance variables, while Python has only a single namespace for  
both of them).

BTW, PyObjC breaks another Python idiom as well: "Explicit is better  
than implicit.", and completely on purpuse ;-)

Ronald

On 16 Jul, 2009, at 4:30, Daniel Miller wrote:

> My testing framework for the app I've been developing (previously with
> PyObjC 1.4, now attempting to migrate to PyObjC 2) is giving lots of
> errors due to a restriction in PyObjC 2 that methods cannot be
> overwritten [0]. My testing framework mocks Cocoa API calls to verify
> that the calls are being made as expected without actually hitting
> Cocoa (which would cause all sorts of unwanted side-affects). My first
> question is about circumventing this read-only method restriction for
> my tests. Is it possible to (temporarily?) make PyObjC methods
> writable? Maybe through some hack that would only be activated as a
> pre-test setup task, allowing me to overwrite methods just during the
> testing phase?
>
> My second question is why such a restriction has been added? I'm not
> trying to start a flame war here, so please don't get that idea.
> Python is generally very liberal in what it allows one to do, even
> potentially damaging things such as shooting ones self in the foot:
>
> "Python culture tends towards "we're all consenting adults here". If  
> you
> attempt to shoot yourself in the foot, you should get some kind of
> warning that perhaps it is not what you really want to do, but if you
> insist, hey, go ahead, it's your foot!" [1]
>
> "...this is Python, where we vigorously defend the right to shoot
> ourselves in the foot." [2]
>
>  Is there any particular technical reason why PyObjC tries harder-
> than-normal-for-Python to protect me from doing things that most
> people probably do not want to do? Thanks in advance for taking time
> to answer my questions.
>
> ~ Daniel
>
> [0] http://www.mail-archive.com/mat...@li.../msg05035.html
> [1] http://mail.python.org/pipermail/tutor/2006-December/051569.html
> [2] http://www.gossamer-threads.com/lists/python/python/753888#753888
>
>
> ------------------------------------------------------------------------------
> Enter the BlackBerry Developer Challenge
> This is your chance to win up to $100,000 in prizes! For a limited  
> time,
> vendors submitting new applications to BlackBerry App World(TM) will  
> have
> the opportunity to enter the BlackBerry Developer Challenge. See  
> full prize
> details at: http://p.sf.net/sfu/Challenge
> _______________________________________________
> Pyobjc-dev mailing list
> Pyo...@li...
> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev

Re: [Pyobjc-dev] TypeError: cannot change a method

[Daniel M. <mil...@gm...>]

RESENDING -- inadvertently sent this directly to Ronald


> Could you post some code that shows what you're trying to do?

# test something involving NSDocument.removeWindowContorller_
doc = MyNSDocumentSubclass.alloc().init()
original = doc.removeWindowController_
fake = mocker.mock()
doc.removeWindowController_ = fake
try:
    fake(arg) # simulate call (record expectation)
    ...
    # replay
    ...
    # our fake method is called by the routine we are testing
    doc.removeWindowController_(arg)
    ...
finally:
    doc.removeWindowController_ = original
    ...
    # verify that simulated calls actually happened
    assert was_called(fake)


This is obviously only pseudo code. My framework makes it much more  
convenient to mock one or more methods and then replace them during  
the verify phase.

> I haven't checked the source code yet, but one reason to be careful  
> about overwriting methods in Cocoa classes is that will affect the  
> actual Cocoa class as wel.
>
> Footnote [0] refers to replacing a method in an instance by  
> something else, e.g.:
>
>    o = NSObject.alloc().init()
>    o.description = 42

Yeah, that's what I'm doing.

> That's not allowed because this can lead to very hard to diagnose  
> bugs, and the people that are most likely to run into those are  
> people new to PyObjC that translate existing ObjC sample code into  
> Python (the reason of that is that ObjC has different namespaces for  
> methods and instance variables, while Python has only a single  
> namespace for both of them).

I'm not quite sure what to say other than I'll need to look into  
rewriting my test framework. FWIW, it seems to be possible to replace  
a method directly on the PyObjC type. For example:

# this works
fake = descriptor_mock()
NSDocument.removeWindowController_ = fake

Assuming the 'fake' object implements the descriptor protocol, the  
fake/mocked method will get called as expected. But it then it fails  
when trying to restore the original method:

NSDocument.removeWindowController_ = original
# TypeError: Assigning native selectors is not supported

However, this seems to restore the original:

del NSDocument.removeWindowController_

Strange... while I may be able to use this approach, it has the  
disadvantage of replacing the method on ALL instances of the type  
rather than on just a the instance that I'm testing--I may need to  
rewrite some tests to work with that restriction, but it probably   
wouldn't be the end of the world.

~ Daniel

[Pyobjc-dev] Fwd: TypeError: cannot change a method

[Ronald O. <ron...@ma...>]

Begin forwarded message:

> From: Ronald Oussoren <ron...@ma...>
> Date: 17 July, 2009 6:53:01 GMT+02:00
> To: Daniel Miller <mil...@gm...>
> Subject: Re: [Pyobjc-dev] TypeError: cannot change a method
>
>
> On 17 Jul, 2009, at 3:09, Daniel Miller wrote:
>>>
>>
>> I'm not quite sure what to say other than I'll need to look into  
>> rewriting my test framework. FWIW, it seems to be possible to  
>> replace a method directly on the PyObjC type. For example:
>>
>> # this works
>> fake = descriptor_mock()
>> NSDocument.removeWindowController_ = fake
>>
>> Assuming the 'fake' object implements the descriptor protocol, the  
>> fake/mocked method will get called as expected.
>> But it then it fails when trying to restore the original method:
>>
>> NSDocument.removeWindowController_ = original
>> # TypeError: Assigning native selectors is not supported
>>
>> However, this seems to restore the original:
>>
>> del NSDocument.removeWindowController_
>>
>> Strange... while I may be able to use this approach, it has the  
>> disadvantage of replacing the method on ALL instances of the type  
>> rather than on just a the instance that I'm testing--I may need to  
>> rewrite some tests to work with that restriction, but it probably   
>> wouldn't be the end of the world.
>
> You're not going to like this, but the behaviour you describe here  
> sounds like a bug. In particular, it is possible to overwrite an  
> existing method using a new function object but it should not be  
> possible to overwrite it with something else. Removing the fake  
> descriptor should also not result in revival of the original method  
> because a replacement method should result in a changed method in  
> the ObjC runtime.
>
> I need to think about your testing needs, if it is possible to  
> support those and if so how.
>
> A number of limititations in PyObjC, such as this one, are caused by  
> the need to interact with the Objective-C runtime. In particular  
> "objc.selector" objects for native methods are basicly references to  
> a named selector on the ObjC class, they do not contain a reference  
> to the code that will be called when you call the method. That's why  
> it is not possible to assign objc.selector instances to a class  
> attribute.
>
> BTW. How does your mocking library deal with other native types:
>
> >>> l = list()
> >>> l.append = 42
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> AttributeError: 'list' object attribute 'append' is read-only
> >>>
>
> Or more interesting, how does it deal with special methods like  
> __getitem__, those always get resolved through the class:
>
> >>> class List (list): pass
> ...
> >>> l = List()
> >>> l.append(42)
> >>> l[0]
> 42
> >>> l.__getitem__ = 42
> >>> l[0]
> 42
> >>>
>
> As you can see setting __getitem__ on an instance has no effect on  
> which special method is actually used.
>
> Ronald
>>
>> ~ Daniel
>>
>

Re: [Pyobjc-dev] TypeError: cannot change a method

[Daniel M. <mil...@gm...>]

On Jul 17, 2009, at 12:53 AM, Ronald Oussoren wrote:
> On 17 Jul, 2009, at 3:09, Daniel Miller wrote:
>>
>>
>> FWIW, it seems to be possible to replace a method directly on the  
>> PyObjC type. For example:
>>
>> # this works
>> fake = descriptor_mock()
>> NSDocument.removeWindowController_ = fake
>>
>> Assuming the 'fake' object implements the descriptor protocol, the  
>> fake/mocked method will get called as expected.
>> But it then it fails when trying to restore the original method:
>>
>> NSDocument.removeWindowController_ = original
>> # TypeError: Assigning native selectors is not supported
>>
>> However, this seems to restore the original:
>>
>> del NSDocument.removeWindowController_
>>
>> Strange... while I may be able to use this approach, it has the  
>> disadvantage of replacing the method on ALL instances of the type  
>> rather than on just a the instance that I'm testing--I may need to  
>> rewrite some tests to work with that restriction, but it probably   
>> wouldn't be the end of the world.
>
> You're not going to like this, but the behaviour you describe here  
> sounds like a bug. In particular, it is possible to overwrite an  
> existing method using a new function object but it should not be  
> possible to overwrite it with something else. Removing the fake  
> descriptor should also not result in revival of the original method  
> because a replacement method should result in a changed method in  
> the ObjC runtime.

Does a descriptor (i.e. an object with a __get__() member) count as a  
function object?

Also, since it is possible to overwrite a method with a function,  
wouldn't it make sense to provide some way to restore it to its  
original state? That should be all I need.

I can understand the motivation to make it illegal to overwrite a  
method on an instance with a non-callable object. However, could the  
restriction be relaxed to allow a callable object (function, object  
with __call__, etc.) to overwrite a method on an instance? As far as I  
know any attempt to replace a method on an instance raises an error  
right now.


> I need to think about your testing needs, if it is possible to  
> support those and if so how.

If there is some way to replace a method with some other callable, and  
then restore to the original state, I think that's all I need. I can  
even deal if it can only be done on the type (not instance), but I  
think my above proposal to allow overwriting an instance method with  
some other callable would probably make sense, while still protecting  
the newbies from overwriting methods unintentionally.


> A number of limititations in PyObjC, such as this one, are caused by  
> the need to interact with the Objective-C runtime. In particular  
> "objc.selector" objects for native methods are basicly references to  
> a named selector on the ObjC class, they do not contain a reference  
> to the code that will be called when you call the method. That's why  
> it is not possible to assign objc.selector instances to a class  
> attribute.

So you're saying the name to which the selector is bound is the  
important part, not the contents of the selector itself? That's  
interesting... That should not cause a problem as long as there is  
some way to restore the selector to its original state after it has  
been overwritten with a mock method. Maybe PyObjCTools could gain a  
special setattr-like function that could be used to overwrite  
selectors. This function would carry a warning that it should only be  
used by people who know what they are doing. It's more kludgy than  
normal for python, but in this case PyObjC violates the normal python  
conventions, so a kludge may be justified.


> BTW. How does your mocking library deal with other native types:
>
> >>> l = list()
> >>> l.append = 42
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
> AttributeError: 'list' object attribute 'append' is read-only
> >>>

I have not needed to mock methods on built-in types, such as  
list.append(), because they do not generally have wide-reaching side  
affects (unlike NSWindowController.window() for example). In a case  
where I really did not want to mutate a list, I would mock/replace the  
entire list instance for the duration of the test (I do this quite  
often). In a framework such as PyObjC, with unconventional write  
restrictions and where many objects are created and/or owned privately  
by the framework, this type of replacement becomes difficult or  
impossible. Hence the need to be able to replace methods with far- 
reaching side affects. If I can't do that then it becomes nearly  
impossible to write tests for code that interacts with the GUI.


> Or more interesting, how does it deal with special methods like  
> __getitem__, those always get resolved through the class:
>
> >>> class List (list): pass
> ...
> >>> l = List()
> >>> l.append(42)
> >>> l[0]
> 42
> >>> l.__getitem__ = 42
> >>> l[0]
> 42
> >>>

Well, in this case I would replace List.__getitem__ (i.e. replace the  
method on the subclass) which is completely legal and possible. PyObjC  
types do not allow this. They are more restrictive than the python  
built-in types because they do not allow me to overwrite a method of  
my own subtype.

> As you can see setting __getitem__ on an instance has no effect on  
> which special method is actually used.

Yes, but if you replace __getitem__ on the subtype, then it works.  
Special methods need to be handled specially. For my test framework,  
PyObjC methods are one class of special method. PyObjC 2 has made them  
harder to deal with, but it's not impossible (yet). Although it sounds  
like you might be making it harder (if the scenario I pointed out  
above is actually a bug).

~ Daniel