[Pyobjc-dev] Reality check on memory management.

[<bb...@ma...>]

On Monday, October 28, 2002, at 03:51 AM, Ronald Oussoren wrote:
> Jonathan,
>
> I agree that we should try to abstract away manual management of 
> reference counts. Until a few days ago I believed we already reached 
> this point.

Even without the automatic compensation for the methods that transfer 
ownership, we have abstracted away the manual management of reference 
counts in almost all cases.

The remaining issues are not specific to Obj-C, they are specific to 
particular kits of objects implemented in using Objective-C.   Therein 
lies the problem-- every developer who builds a kit of objects can 
choose [consciously or not, doesn't matter] to implement a completely 
different retain/release/autorelease pattern across their API.

Example:  I often use the +new pattern to return an object with 
ownership transfer implied.   I.e. +newWithFile: (NSString *) aPath; 
returns a freshly instantiated instance of my Image class that has been 
retained.   However, Apple's DragNDropOutlineView example uses 
+newGroupFromLeaf: to return an autoreleased object.

We cannot possibly predict what developers are going to do with the 
bridge.   Nor can we ever completely hide the 
retain/release/autorelease "feature" of Objective-C.   It is very much 
a reality and will always be a reality in any bridged environment.

However, what we can do is make the bridge work as transparently as 
possible in light of this reality (which it pretty much already does)!  
  In particular, we can make sure that if a program fails because of a 
retain/release/autorelease problem, it does so in as graceful of a 
fashion as possible (i.e. it dies in ObjC, not python, as often as 
possible :-).

To these ends, let's do a reality check on the problem.

The problem is not that we need to deal with -retain/-release in the 
normal course of using the bridge -- that is already very cleanly taken 
care of.

The problem is that there are a handful of methods-- +alloc, +copy, 
+mutableCopy, +new, and any of a number of methods that the developer 
might create that we cannot know about in advance-- that "transfer 
ownership" to the caller.  That is, they return retained objects.   In 
the context of the bridge, this yields an extra reference count and 
Ronald's goal is to try and automatically compensate for that reference 
count.

A worthy goal, but not one that can be achieved with 100% coverage.

OK-- so why not just leave it out?  What are the implications then?

To the ObjC developer, the apps "just work" as expected.

To the Python developer, the apps "just work" as expected, but leak a 
little bit of memory.

Leaks are a hell of a lot easier to fix than retain/release/autorelease 
crashes.

Furthermore, it means that taking advantage of a new framework/bundle 
of objects is as easy simply dynamically loading the code -- no need to 
worry about analyzing the API and trying to figure out what methods 
should be "special cased" before we can even write line-one of python 
code against that API.

I have worked on a number of language bridges -- Java<->ObjC, 
Perl<->ObjC (long before CamlBones), Tcl<->ObjC, Tcl<->Python [don't 
ask], and a couple that involved bridging languages I created to Python 
or Obj-C -- and in *every single case*, a decision to special case as 
would be [is] required by this [relatively small] problem was a *very 
bad idea*.   It caused confusion in the developer community-- including 
confusion among the authors of the bridge-- and was a constant source 
of maintenance and support headaches.

> There are now two sets of ...

There will always be situations that expose retain/release in contexts 
that do not map directly to a pattern we have quantified.

An example: when implementing a window controller, it is not uncommon 
to do a [self retain] when the NIB is loaded (or the window is 
displayed) and a [self autorelease] in windowWillClose:.   That is, the 
window controller implements a "as long as my window is open-- 
miniaturized or not, doesn't matter-- I should remain in existence" 
notion.  This pattern makes it extremely easy to implement 
multi-document apps where the application doesn't manage the list of 
documents in some formal "ownership implied" fashion.

(This is the paradigm that the Web Services Tool follows, btw).

> 1. Bill mentioned that it is common in some Objective-C code to 
> transfer
>    ownership of returned objects to the caller (He's the resident 
> expert
>    on these issues, I'm learning Objective-C/Cocoa as a go along).
>
>    It will be hard to correctly identify all these cases. IMHO this 
> shouldn't
>    keep us from doing away with autorelease/release/retain in Python 
> code:
>    it will just be a little harder to correctly wrap an Objective-C 
> class
>    library. As wrapping an Objective-C class is an order of magnitude 
> simpler
>    than wrapper other 'native' code that should not be a problem.

I suppose that this basically introduces a wrapping step into the 
"let's use some random hunk of dynamically loadable API" process?  If 
that step is not taken, then the retains will not be compensated for in 
the bridge?  I.e. the step is optional.

As long as the wrapping step is optional, it isn't a big deal.   If it 
becomes non-optional, then it vastly increases the complexity of using 
external frameworks/bundles over what we have today (a one liner to 
load the bundle/framework w/some optional glue code that Ronald has 
nicely documented in the wrapping.txt document).

> 2. Bill also mentioned a coding style that uses 'autorelease' to make 
> sure
>    that an object is around to the end of the current loop through the
>    eventloop (which in case of non-GUI programs may be quite a long 
> time).
>
>    I don't really like this coding style, so IMHO it is not a problem 
> to use
>    'obj.retain().autorelease()' in these cases, instead of just
>    'obj.autorelease()'. I'm also pretty sure that doesn't really have 
> anything
>    to do with the automatic management of refcounts by the bridge.

The end result would be that the retain/autorelease is still exposed to 
the Python developer while the ObjC developer has to remember a special 
case.... I would rather the solution that exposes only the autorelease 
to the python developer and leaves things as is for the ObjC developer.

The reality in this situation is that to use this pattern requires 
knowledge of the ObjC patterns of object management and the various 
APIs (most likely NSTimer or NSNotification) therein.

Keep in mind that an autorelease pool's lifespan may not be tied to one 
pass through a run loop and, therefore, object's like NSTimer-- which 
are processed as a part of passing through the run loop-- are perfectly 
valid in this context.

We cannot possible predict what the developer is going to do...

> 3. The NSMutableArray class cluster (and probably all 
> NS<DataStructure> class
>    clusters) play games with reference counts during allocation. This 
> is not
>    a problem if you use the common Objective-C idiom, but it is when 
> you try
>    to automate management of reference counts.
>
>    Basicly the issue is that the Objective-C manual strongly suggest 
> that the
>    following two lines should be equivalent, but they are not for 
> these class
>    clusters:
>
>    NSArray.alloc().autorelease().init()	# Basicly what the bridge does
>    NSArray.alloc().init().autorelease()   # Common Objective-C idiom
>
>    I'm thinking on how to fix this. The one requireing the least 
> amount of
>    engeneering is saying "Don't use 'alloc().init()' for these class 
> clusters",
>    or even "Don't allocate Objective-C containers from Python". I 
> don't really
>    like this and am thinking on how to fix this in a correct manner.

If we create a special case to handle this situation, it will only 
cause maintenance hell down the road.  What if Apple changes the 
semantics?  What if new collection classes are introduced that suddenly 
start breaking in the way NSArray does now?   What if a third party 
creates collection classes with the same behavior as Apple's?

b.bum