pyobjc-dev — CVS commit notices go here

Re: [Pyobjc-dev] NSString & mutability

[Bob I. <bo...@re...>]

On Wednesday, Feb 5, 2003, at 10:23 America/New_York, bb...@ma... 
wrote:
>
> What we really need is a way to attach the NSString instance to the 
> PythonString instance such that any crossing of the bridge can 
> directly grab the NSString on entry into ObjC.
>
> I have realized that I'm wrong in regards to avoiding conversion -- 
> conversion isn't the problem.  The problem is the lack of preserving 
> the identity as a string crosses from Python to ObjC and back again.   
> That the developer will always have to access the NSString methods via 
> unbound methods is annoying, but not catastrophically so.
>

The method I proposed the other day, subclassing str (and converting on 
first entry from objc->python) and implementing a custom version of 
__new__ should solve (almost?) all of these problems... with the caveat 
that it takes more memory because it converts the string (perhaps only 
the first time across the bridge).  It would certainly fill in for a 
python str in 99% of the places that a python str is accepted (because 
it's a subclass of str itself).  You can also solve the NSString 
accessibility as well this way with the __getattr__ to the instance of 
the NSString it keeps around.

Of course, you'd still need the ObjC class on the other side to accept 
Python strings.. which I think already works just fine?

-bob

Re: [Pyobjc-dev] NSString & mutability

[David E. <epp...@ic...>]

On 2/5/03 9:39 AM +0100 Ronald Oussoren <ous...@ci...> wrote:
> I think you should read this as 'even though some methods are defined as
> returning an NSFoo they may return any subclass (like NSMutableFoo),
> depending on the actuall type is bad'. Read like that this is pretty
> relevant to our discussion: If we convert NSString and wrap
> NSMutableString (a subclass of NSString) the user may sometimes get a
> python string object and sometimes not from functions/methods that are
> documented as returning a NSString.

The bad part about this is that the mutable returns may be quite rare, so 
you won't notice for a long time that your code has a bug.  I'd rather find 
out quickly when a str() wrapper is necessary.
--
David Eppstein       UC Irvine Dept. of Information & Computer Science
epp...@ic... http://www.ics.uci.edu/~eppstein/

Re: [Pyobjc-dev] NSString & mutability

[<bb...@ma...>]

On Tuesday, Feb 4, 2003, at 18:28 US/Eastern, Just van Rossum wrote:
> Let's focus on making NSString instances work as much as Python strings
> as possible.
>
> For fun, I removed the conversion, and (quite obviously) as it stands
> stuff breaks horribly. The first error thrown by starting a random
> PythonCocoa app is in NibClassBuilder, where an NSString is fed to an
> os.path function. I _think_ for many cases it's enough if we implement
> all string methods. There aren't even that many:
>
>     'capitalize', 'center', 'count', 'encode', 'endswith', 
> 'expandtabs',
>     'find', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit',
>     'islower', 'isnumeric', 'isspace', 'istitle', 'isupper', 'join',
>     'ljust', 'lower', 'lstrip', 'replace', 'rfind', 'rindex', 'rjust',
>     'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase',
>     'title', 'translate', 'upper', 'zfill'

In objc_support.m, I changed the thing that converts from NSString to 
PythonStrings to this:

         else if ([obj isKindOfClass:[NSString class]])
         {
	    retobject = buffer_from_nsstring(obj);
         }

It is really just a convenience hook via which I can try different 
recipes for converting between.   I current implement 
buffer_from_nsstring() as:

static PyObject* buffer_from_nsstring(NSString *aString)
{
     PyObject* result;
     NSData* data = [aString dataUsingEncoding:NSASCIIStringEncoding];
     if (data == NULL) {
	const char* utf8 = [aString UTF8String];
	result = PyUnicode_DecodeUTF8(utf8, strlen(utf8), "strict");
     } else {
	result = PyString_FromStringAndSize([data bytes], [data length]);
     }
     return result;
}

This works-- it is basically *exactly* like the implementation in CVS.  
  I tried an implementation that returns a character buffer, but a unit 
test that does...

self.assertEquals("foo", NSString.stringWithString_("foo"))

... fails because "foo" cannot be compared to an instance of <character 
buffer>.

Nuts.

What we really need is a way to attach the NSString instance to the 
PythonString instance such that any crossing of the bridge can directly 
grab the NSString on entry into ObjC.

I have realized that I'm wrong in regards to avoiding conversion -- 
conversion isn't the problem.  The problem is the lack of preserving 
the identity as a string crosses from Python to ObjC and back again.   
That the developer will always have to access the NSString methods via 
unbound methods is annoying, but not catastrophically so.

Example:  By 'preserving the identity', I mean that the assertion error 
in the following should not happen [should behave like the Python 
equivalent]:

 >>> from Foundation import *
 >>> a = NSMutableArray.array()
 >>> s = "foo bar baz"
 >>> a.addObject_(s)
 >>> id(s)
1669344
 >>> id(a[0])
6531696
 >>> assert(id(s) == id(a[0]))
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
AssertionError
 >>> k = [s]
 >>> assert(id(s) == id(k[0]))
 >>>

Mutable strings raise other issues -- mainly that we have to deal with 
them at all. :-)

b.bum

[Pyobjc-dev] Re: [Python-Dev] Bridging strings from Python to other languages

[Guido v. R. <gu...@py...>]

Hi Bill,

I've read though the end of your post, and while I appreciate the
problem, I have little help to offer.  It's not even clear whether
you're asking for changes to Python or simply a hint as to how to
write your bridge.

To me, it seems that the crux is your requirement

>      (3) identity is maintained;   pass a string with id() 7570720 from 
> Python into the alien runtime and subsequently from the alien runtime 
> back into python and the same string instance with id() 7570720 really 
> should come back

In my experience almost no Python code depends on this property, and
it seems to be the most problematic one.  So why is this a
requirement?

If you can live with only using Unicode strings (even when all they
contain is ASCII or Latin-1 values), I think subclassing Unicode might
be the way to go.

I don't have time to dig deeper into this.  But if you think a small
change to Python can make life easier for you, I expect we'll be happy
to implement it, as long as it doesn't make life harder for Python
developers.

--Guido van Rossum (home page: http://www.python.org/~guido/)

[Pyobjc-dev] Documentation to aim for

[Dinu G. <gh...@da...>]

Hi folks,

I feel like we should aim for something like this
to happen by the end of this year!

Sorry, but I just couldn't resist...! ;-)

Dinu

--
Dinu C. Gherman
......................................................................
"Even if you do learn to speak correct English, whom are you going
to speak it to?" (Clarence Darrow)

Re: [Pyobjc-dev] NSSearchPathForDirectoriesInDomains doesn't recognize YES

[Ronald O. <ous...@ci...>]

On Wednesday, Feb 5, 2003, at 09:51 Europe/Amsterdam, Ronald Oussoren 
wrote:

>
> On Tuesday, Feb 4, 2003, at 22:21 Europe/Amsterdam, Martina Oefelein 
> wrote:
>>
>> should I file a bug?
> Please do.

Never mind, fixed in CVS.

Ronald

Re: [Pyobjc-dev] NSSearchPathForDirectoriesInDomains doesn't recognize YES

[Ronald O. <ous...@ci...>]

On Tuesday, Feb 4, 2003, at 22:21 Europe/Amsterdam, Martina Oefelein  
wrote:

> I don't know whether this is a PyObjC bug or a Cocoa bug, but  
> NSSearchPathForDirectoriesInDomains doesn't recognize PyObjC's YES.
>
> According to the docs, if the third parameter (expandTilde) is YES,  
> tildes are expanded as described in  stringByExpandingTildeInPath. But  
> this doesn't work. Tildes are expanded only if I pass -1:
>
> [iBook:~] martina% python
> Python 2.2 (#1, 07/14/02, 23:25:09)
> [GCC Apple cpp-precomp 6.14] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>> from objc import NO, YES
> >>> from Foundation import  
> NSSearchPathForDirectoriesInDomains,NSLibraryDirectory,NSUserDomainMask
> >>>  
> NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,NSUserDomainMask 
> ,NO)
> ("~/Library")
> >>>  
> NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,NSUserDomainMask 
> ,YES)
> ("~/Library")
> >>>  
> NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,NSUserDomainMask 
> ,1)
> ("~/Library")
> >>>  
> NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,NSUserDomainMask 
> ,-1)
> ("/Users/martina/Library")
>
> should I file a bug?
Please do.

Ronald

Re: [Pyobjc-dev] NSString & mutability

[Ronald O. <ous...@ci...>]

On Tuesday, Feb 4, 2003, at 21:56 Europe/Amsterdam, Just van Rossum 
wrote:

> bb...@ma... wrote:
>
>> Nice hack, but....
>>
>> What about all the code that might return a mutable string depending
>> on how the developer exercises the API?  Are you willing to assume
>> that every getter and setter that internally makes a immutable copy
>> of the set string value?  i.e. that...
>>
>> .... [fooObject setMyNameIs: someRandomMutableString]; return
>> [fooObject myNameIs] ....
>>
>> .... that -myNameIs will return an immutable string?
>
> I'm totally willing to accept a surprise like that. I don't even see 
> the
> problem when using such an API from Python: you call
> fooObject.setMyNameIs_() with a Python string, you call myNameIs() and
> get a Python string. Same string, different id. No problem in Python.
>
> Can you please give an example of an _actual_ problem? I'll have a go
> myself:
>
> Let's assume the API does the reverse: it takes an immutable string, 
> yet
> returns its internal mutable copy, although myNameIs is declared to
> return an NSString. Now there _is_ a surprise: instead of a 100%
> equivalent string we get an NSMutableString instance. Ok, a surprise,
> but is it so bad? If the object internally uses a mutable string, it's
> likely that this is clear from the nature of the object (eg. it's some
> sort of editor for the string). So I still think such a surprise will 
> be
> rare.

The 'scary' part is that the behavious could change without notice from 
one update of MacOS to the other, because NSMutableString is a subclass 
of NSString switching from a returnvalue of type NSString to one of 
NSMutableString is a backward compatible change.

>
>> See this message -- it is still quite relevant.  Mike Ferris is one
>> of the guys who helped write the AppKit and Project Builder.
>>
>> http://www.omnigroup.com/mailman/archive/macosx-dev/2000-May/001886.
>> html
>
> It's an example of NSArray, which we don't convert, so we live
> dangerously anyway in PyObjC as we don't even have the hint from the
> header that we _shouldn't_ mutate even if it is mutable.

I think you should read this as 'even though some methods are defined 
as returning an NSFoo they may return any subclass (like NSMutableFoo), 
depending on the actuall type is bad'. Read like that this is pretty 
relevant to our discussion: If we convert NSString and wrap 
NSMutableString (a subclass of NSString) the user may sometimes get a 
python string object and sometimes not from functions/methods that are 
documented as returning a NSString. Unless we can make sure that the 
NSMutableString wrapper is usable everywhere where a Python string is 
this will give us great headaches in the future.

Ronald

Re: [Pyobjc-dev] test for integer handling by undoManager

[<bb...@ma...>]

David's test case has been added to test_undomanager.py.

So -- either we fix the test case or we fix the underlying behavior. :-)

b.bum

[Pyobjc-dev] Bridging strings from Python to other languages

[Bill B. <bb...@co...>]

[This is a continuation of the thread that Just mentions below -- 
"NSString & mutability".  I finally had a chance to write enough code 
to figure out where the walls were that I kept bloodying my head 
against.  I believe I also came up with a more constructive way to 
think about this whole problem. More below -- the end of the 'bridging 
strings' section contains a series of what I believe are issues with 
the current Python implementation that should/may/could be addressed in 
a future version]

On Tuesday, Feb 4, 2003, at 22:13 US/Eastern, Just wrote on python-dev:
> (The use case is this. The PyObjC project marries Objective-C with
> Python. This is cool as it gives us direct access to almost all of
> Cocoa, the native OSX GUI interface. However, Cocoa defines its own
> string type and for reasons that are waaay beyond the scope of this 
> post
> (check the archives of the pyobjc-dev list if you're really really
> interested; see a recent thread called "NSString & mutability") it
> appears a bad idea to _convert_ these strings to Python unicode 
> strings.
> So we need to wrap them. Yet they should work as much like unicode
> strings as possible...)

Let me rephrase the problem in slightly different terms.   This will be 
long winded-- skip down to the 'bridging strings' section if you don't 
want to go through the initial discussion of the challenges of bridging 
two runtimes...

In creating a bridge between Python and other languages-- in this case, 
Objective-C-- the general goal is to provide seamless connectivity 
between the two runtime environments.   That is, you want to have a 
proxy to objects or structures found in the 'alien' runtime available 
in the 'native' runtime in a fashion that makes the proxy convenient to 
use.   Generally, this means that the proxy should act as much like the 
'native' runtime up to the point where it starts to obfuscate the 
behavior of the 'alien' runtime.   While a decent bridging and proxying 
mechanism can make "crossing the bridge" easy to do, one can never 
avoid the fact that there really is a bridge and on the other side 
there really is an 'alien' runtime.

Now-- there are a number of different ways to proxy objects/structures 
between the two runtimes:

- pure proxy:   the object/structure to be bridged is represented by a 
proxy that handles all requests for information or invocation of 
functions/methods by converting the request/invocation into a form that 
can be understood on the other side of the bridge.

Example -- the following creates a python native proxy to the alien 
Objective-C NSMutableArray instance (ignore that it really creates an 
NSCFArray-- that is an internal-to-Foundation implementation detail 
that is irrelevant).  The expression 'a.count()' actually causes the 
'count' Objective-C method to be invoked through the proxy a:

 >>> from Foundation import *
 >>> a = NSMutableArray.array()
 >>> type(a)
<objective-c class NSCFArray at 0x466d0>
 >>> a.count()
0
 >>> len(a)
0

The len(a) is just a demonstration of how far the proxying can go by 
defining the appropriate internal(?) attributes on the proxy.

- pure conversion:   the object/structure to be bridged is converted to 
the native type as it crosses the bridge.

Example -- the NSString is currently bridged to the Python String class 
such that string instances are converted to their native types as they 
cross the bridge [at least, this is the case in CVS -- I now have a 
proxy class that can wrap a Python PyString/PyUnicode instance and 
present it is a standard NSString instance on the ObjC side.  Avoids 
lots of unnecessary data copying when going from Python->Objective-C, 
but it needs a bunch of cleanup before I commit.]:

In the following, I create a new Objective-C NSString instance and 
assign it to 's'.  What results is a copy of the contents of the 
NSString instnace shoved into a normal Python string.

 >>> s = NSString.stringWithString_("Foobar")
 >>> type(s)
<type 'str'>

- mixed conversion/proxy:   this is a suboptimal case.   It generally 
converts to a native type in one direction, but potentially not in the 
other or not fully.

Example -- NSNumber is currently in this category.   It should change 
eventually, but there are issues to deal with:

 >>> a = NSArray.arrayWithObject_(1)
 >>> a[0]
<NSCFNumber objective-c instance 0x6704b0>
 >>> a[0] + 1
2

---

One of the key challenges is that proxying effectively causes two 
references to any given object to exist;   the native object reference 
and the 'alien' reference through the proxy.   Care must be taken to 
ensure that a single reference on either side of the bridge is enough 
to preserve both components of the hunk of data while also ensuring 
that the existence of a proxy without references does not prevent the 
item from being collected [may sound confusing:  consider the situation 
where a Python class is subclassed in the alien environment or 
vice-versa -- you effectively end up with instances that have part of 
their implementation in one runtime and the other part in the other 
runtime.  It can lead to issues.].

In general, these kinds of issues can be worked through by leveraging 
mechanisms such as weak references.  By providing a callback on the 
finalization of an object, it is possible to ensure that the 
alien-to-python component of the instance is destroyed, as well.

A final challenge is that sometimes an object's type or contents are 
completely irrelevant to a piece of code.   It is the object reference 
itself that is meaningful.  In these situations, if an object is passed 
across the bridge and back, what should come back really should be what 
went across in the first place-- if not, the contents may have been 
preserved, but the object's original identity has been lost.

Sometimes an object is just an object.

---

Strings provide a particular set of challenges in that no two runtime 
environments present exactly the same set of features in their string 
handling API, yet every runtime has some kind of a string API and, 
invariably, that API is very much at the core of the runtime.   The 
addition of Unicode to every string API over the last decade+ has not 
made things any simpler.

In python, strings are immutable and can encapsulate non-unicode data.  
  A separate unicode-- also immutable-- type is provided to encapsulate 
unicode data, but the standard string type can also encapsulate unicode 
data in certain circumstances [at least, it appears that PyString will 
happily consume and represent UTF8].

In Objective-C [and other languages], there is a single String class 
that can encapsulate both ASCII and unicode data in many different 
encodings.   Furthermore, there is a subclass of String that provides 
additional mutability API -- an instance of the mutable string class 
can have its contents changed by the developer while the identity of 
the object remains the same (unlike python where appending "b" to "a" 
results in a new string "ab").

To further complicate matters, most typed languages support the concept 
of 'upcasting' -- that is, of casting a particular instance to actually 
be an instance of a superclass.   For Objective-C, it can mean that a 
method that is declared as returning an immutable string or array 
actually returns a mutable string or array instance -- as long as the 
developer pays attention to the compiler warnings and doesn't do any 
stupid casting of their own, everything is fine.   Java offers similar 
casting "features".

- bridging strings -

So, how to bridge strings in such an environment?   In all cases, we 
can [fortunately] assume that strings pass across the bridge in one of 
a few choke points in the code -- that there is always a location to 
add a little bit of logic with which to help bridge the string [or any 
other random object].

The goal is to bridge strings in a fashion such that (not really in 
order of importance):

     (1) only one hunk of memory is used to contain the data within the 
string

     (2) conversion is kept to a minimum, if present at all, because 
strings will be passed back-and-forth across the bridge very frequently

     (3) identity is maintained;   pass a string with id() 7570720 from 
Python into the alien runtime and subsequently from the alien runtime 
back into python and the same string instance with id() 7570720 really 
should come back

     (4) 'alien' string specific API can still be used;   the 
Objective-C NSString provides a very rich API, including localization 
features that are not available in pure python.

For Python->Objective-C, bridging strings has proven to be fairly easy. 
  (1), (2), and (4) are quite straightforward.   (3) is not done yet.

For Objective-C->Python, bridging strings is not so easy.  The 
difficulty is compounded by certain features of the Python 
string/unicode APIs.

(1) is pretty easy -- the challenge is to figure out which API to call 
on the Python side such that the resulting Python object does not copy 
and re-encode the data.   If that is unavoidable, the cost of encoding 
or conversion (2) should be minimized [hopefully with a cache so that 
cost of conversion is paid once, then never again for immutable string 
instances].  There is also the ongoing challenge of determining when to 
use the PyString vs. PyUnicode APIs;  it seems that unicode objects are 
not welcome everywhere that string objects are?

(4) is actually quite easy and has been available for some time through 
the use of unbound methods.   However, the current implementation in 
CVS will always cause the python string to be converted to an NSString, 
the method invoked, and then the result-- if any and if a string-- is 
converted back to a python string.

(3) is not so easy-- at least, not from what I have determined so far.  
Most of the issues seem to be due to limitations in Python (which is 
really just another way of saying "I don't know enough to approach this 
problem from the right direction"):

     - can't use weakref because one can't have a weak reference to a 
string or unicode object.   This means that a callback when a string 
ref is finalized on the python side is not possible.   It also means 
that creating a hash between ObjC string instances and Python string 
instances can't be done without using strong references, thereby 
creating the potential for leaking memory.

     - can't subclass string (but can unicode) to provide a class that 
acts exactly like a regular string while containing a reference to the 
foreign string object.  There doesn't appear to be anywhere to hide a 
hunk of data in the string instance, either.

     - can't use the character buffer APIs because a character buffer 
cannot be used consistently throughout the python APIs in the same 
places as a string.  Using str() to turn a char buffer into a string 
violates (1) [and doesn't make much sense anyway].

End result -- it is very difficult to preserve the association between 
an alien string instance and a PyString instance consistently.  Even if 
PyString instances provide very thorough and consistent hashing 
behavior where two strings with the same contents always hash the same, 
the same cannot be said of all alien environments.   Even when it is 
true, there are cases where the developer may be relying on the 
identity of the object to not change outside of their control.

Mutable strings obviously present issues of their own, but they are not 
particularly relevant to discussion on python-dev outside of how future 
development might make the support of such common idiosyncrasies a bit 
more straightforward. Ideally, one could have an object on the python 
side that looks/feels/smells like a string instance, but whose contents 
may change.  This creates any number of exciting problems.  To further 
compound problems, anything that is declared as returning an NSString 
*may* return an NSMutableString at whim.  It doesn't happen often, but 
when it does, if the handling of mutable vs. immutable strings is too 
radically different, it'll cause code to blow up in highly unexpected 
and very difficult to debug ways.

Rambling on....
b.bum

Re: [Pyobjc-dev] NSString & mutability

[<bb...@ma...>]

On Tuesday, Feb 4, 2003, at 18:28 US/Eastern, Just van Rossum wrote:
>> It is unreasonable to expect that the internal implementation will
>> remain consistent over time, yet that is exactly what a partial
>> conversion solution expects.
>
> Ok, I'm about to give in.

Can we table this for a bit?

I'm in the middle of an implementation that *may* actually solve this 
problem in a somewhat graceful fashion that makes both of us happy.  
I'm not sure and it'll take more time for me to figure out how to 
explain it than to do some more research programming to determine if 
it'll work.

That, and I'm tired of tossing a bunch of words out there without 
backing it with useful examples/code.... and I'm tired of having time 
sucked from forward progress for the sake of a [very interesting] 
argument.   I'm sure that there are a lot of others on this list-- 
yourself at the head of the list-- in the same boat.

BTW:  I do have Python->ObjC working nicely -- Python strings pop into 
ObjC world without copying data.

b.bum

Re: [Pyobjc-dev] NSString & mutability

[Just v. R. <ju...@le...>]

bb...@ma... wrote:

> It is unreasonable to expect that the internal implementation will 
> remain consistent over time, yet that is exactly what a partial 
> conversion solution expects.

Ok, I'm about to give in.

Let's focus on making NSString instances work as much as Python strings
as possible.

For fun, I removed the conversion, and (quite obviously) as it stands
stuff breaks horribly. The first error thrown by starting a random
PythonCocoa app is in NibClassBuilder, where an NSString is fed to an
os.path function. I _think_ for many cases it's enough if we implement
all string methods. There aren't even that many:

    'capitalize', 'center', 'count', 'encode', 'endswith', 'expandtabs',
    'find', 'index', 'isalnum', 'isalpha', 'isdecimal', 'isdigit',
    'islower', 'isnumeric', 'isspace', 'istitle', 'isupper', 'join',
    'ljust', 'lower', 'lstrip', 'replace', 'rfind', 'rindex', 'rjust',
    'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase',
    'title', 'translate', 'upper', 'zfill'

It would perhaps be best if the ones that return a string would return
Python str or unicode instances and not a new NSString instance. In many
cases the implementation could consist of:

    def a_random_string_method(self):
        s = convert_to_python_string(self)
        return s.a_random_string_method()

If there is an NSString counterpart, it's perhaps better to do:

    def a_random_string_method(self):
        s = self.NSString_equivalent_of_a_random_string_method()
        return convert_to_python_string(s)

Hm, but then again, maybe not. Thoughts?

The buffer interface should make raw NSStrings work in many cases (eg.
system calls), yet I don't think it's possible to make it work for
unicode strings in all cases: we would need to provide a raw character
buffer that's compatible with the unicode internals. We need to figure
that out.

[I just looked at the implementation of the "et" format for
PyArg_ParseTuple() (gets a buffer with a specified encoding), and while
traversing down the call graph I came across PyUnicode_FromObject().
This call wants a buffer, which sucks for our purposes. However, a
comment in that function says
    /* XXX Perhaps we should make this API an alias of
           PyObject_Unicode() instead ?! */
which might be exactly what we need, as _that_ one actually calls
ob.__unicode__(). So by careful lobbying me might be able to fix this
for Python 2.3. Must hurry, though, as 2.3a2 is supposed to be out soon.
Code to be compatible with 2.2 would then still have to call
unicode(anNSString).]

Oh, it's that time again: Must Sleep...

Just

Re: [Pyobjc-dev] NSString & mutability

[Just v. R. <ju...@le...>]

Just van Rossum wrote:

> Ok, I understand why it happens: __str__ returns [obj description],
> and when we convert _all_ strings, this automatically works
> correctly, but if we don't it doesn't <wink>, since it seems any
> NSString just returns self from description. But now I don't know how
> to work around that... Any hints? (Btw. I've found
> Lib/objc/_convenience.py.)

I've managed to "fix" this. I've attached a patch that more or less
completely implements partial conversion. unicode(someMutableString)
works as expected (as well as str()).

And now I'm going to read Bill's epos a second time ;-)

Just

Re: [Pyobjc-dev] NSString & mutability

[<bb...@ma...>]

On Tuesday, Feb 4, 2003, at 15:56 US/Eastern, Just van Rossum wrote:
> I'm totally willing to accept a surprise like that. I don't even see 
> the
> problem when using such an API from Python: you call
> fooObject.setMyNameIs_() with a Python string, you call myNameIs() and
> get a Python string. Same string, different id. No problem in Python.

I'm not willing to accept a surprise like that because it will greatly 
reduce the value of PyObjC in quite a number of situations.   This is 
not idle speculation.   I have made the same mistake in the past 
[creating a bridge that was dependent on internal implementation of 
underlying frameworks] and it rendered the bridge [Tcl <-> ObjC, in 
this case] nearly useless until it was fixed -- it broke-- or sometimes 
didn't break-- depending on what order the user did things simply 
because some random bit of code somewhere chose to return *this* 
private subclass vs. *that* private subclass as some internal 
optimization hack that was *not* apparent, no relevant to, the 
advertised API.

The fundamental problem is that partial conversion relies on the 
internal implementation to stay the same over time.   The current 
behavior does not;  it remains consistent.  Partial conversion behavior 
that is dependent upon the internal implementation of 'third party' 
[Apple is a third party, in this case] frameworks will lead to behavior 
that changes over time and through factors outside of our control.

More below....

> Can you please give an example of an _actual_ problem? I'll have a go
> myself:

Off the top of my head, no-- I can't remember a specific example.  But 
that is mostly because I have been relying on the APIs as advertised by 
the classes and not their internal implementations for long enough that 
I haven't had this problem in a while.   It *does* come up on the 
mailing lists on occasion, but searching for "nsmutablestring nsstring 
return" yields an awful lot of noise hits.

> Let's assume the API does the reverse: it takes an immutable string, 
> yet
> returns its internal mutable copy, although myNameIs is declared to
> return an NSString. Now there _is_ a surprise: instead of a 100%
> equivalent string we get an NSMutableString instance. Ok, a surprise,
> but is it so bad? If the object internally uses a mutable string, it's
> likely that this is clear from the nature of the object (eg. it's some
> sort of editor for the string). So I still think such a surprise will 
> be
> rare.

I agree that the situations where this arise will be rare.   But, when 
they do arise, the results are the absolute worst kinds of bugs in the 
world to fix.  It breaks on customer A's computer, but not customer B's 
-- but they are [supposedly] identical.  Oh, wait a minute, Customer B 
upgraded to version 1.234b of the some random plugin while customer A 
is still using 1.233.

Another exmaple: What if Mike [who I included in this because he may 
have some useful input-- sorry, Mike :-] changes the implementation of 
TextExtras and throws an NSMutableString into some random object 
collection that ends up on the other side of the bridge at some point?  
  The Python developer is screwed if there code breaks.

What happens when Apple ships the 10.2.5 update and they change some 
random bit of internal implementation such that a method that used to 
return (NSMutableString*) now *always* returns (NSString*)?   Assuming 
the PyObjC developer can figure out what is going on, are they now 
supposed to write OS X version specific code at the x.x.1 level?

Now-- what about the developers that will be using PyObjC with multiple 
thousands of lines of code embedded in random frameworks across some 
large scale system?   For them, the choice of using PyObjC now comes 
with the cost of having to ensure that every method that *may* return 
an NSMutableString* when it is declared with an NSString* return value 
is identified and compensated for.   This is not a contrived example-- 
I and a number of people I have been in contact with since PyObjC 
became a visible project again-- have used PyObjC in this context in 
the past and are planning on doing so in the future [some already are]. 
   This would be a deal killer for them.

Bottom line:  PyObjC -- just like the AppleScript in an AS Studio 
project or a good chunk of the ObjC in a pure ObjC Cocoa app -- is the 
glue that holds together all of the random objects in a fashion that 
solves whatever problem the developer is trying to address.   The 
developer does not have full and complete control over those objects-- 
heck, there may be entire chunks of implementation present that their 
code has no awareness of whatsoever [TextExtras].    For such a system 
to work, the objects must be glued together as advertised by the APIs.  
  To do otherwise will render a system that is fragile.  Worse, it will 
create a system whose fragility is dependent upon unreasonable 
expectations.

It is reasonable to expect-- and history indicates it is true-- that 
the behavior of the NSCell API will remain consistent going forward.  
It is unreasonable to expect that the internal implementation will 
remain consistent over time, yet that is exactly what a partial 
conversion solution expects.

b.bum

[Pyobjc-dev] NSSearchPathForDirectoriesInDomains doesn't recognize YES

[Martina O. <Ma...@Oe...>]

I don't know whether this is a PyObjC bug or a Cocoa bug, but  
NSSearchPathForDirectoriesInDomains doesn't recognize PyObjC's YES.

According to the docs, if the third parameter (expandTilde) is YES,  
tildes are expanded as described in  stringByExpandingTildeInPath. But  
this doesn't work. Tildes are expanded only if I pass -1:

[iBook:~] martina% python
Python 2.2 (#1, 07/14/02, 23:25:09)
[GCC Apple cpp-precomp 6.14] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> from objc import NO, YES
 >>> from Foundation import  
NSSearchPathForDirectoriesInDomains,NSLibraryDirectory,NSUserDomainMask
 >>>  
NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,NSUserDomainMask, 
NO)
("~/Library")
 >>>  
NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,NSUserDomainMask, 
YES)
("~/Library")
 >>>  
NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,NSUserDomainMask, 
1)
("~/Library")
 >>>  
NSSearchPathForDirectoriesInDomains(NSLibraryDirectory,NSUserDomainMask, 
-1)
("/Users/martina/Library")

should I file a bug?

ciao
Martina

Re: [Pyobjc-dev] NSString & mutability

[Just v. R. <ju...@le...>]

bb...@ma... wrote:

> Nice hack, but....
> 
> What about all the code that might return a mutable string depending
> on how the developer exercises the API?  Are you willing to assume
> that every getter and setter that internally makes a immutable copy
> of the set string value?  i.e. that...
> 
> .... [fooObject setMyNameIs: someRandomMutableString]; return
> [fooObject myNameIs] ....
> 
> .... that -myNameIs will return an immutable string?

I'm totally willing to accept a surprise like that. I don't even see the
problem when using such an API from Python: you call
fooObject.setMyNameIs_() with a Python string, you call myNameIs() and
get a Python string. Same string, different id. No problem in Python.

Can you please give an example of an _actual_ problem? I'll have a go
myself:

Let's assume the API does the reverse: it takes an immutable string, yet
returns its internal mutable copy, although myNameIs is declared to
return an NSString. Now there _is_ a surprise: instead of a 100%
equivalent string we get an NSMutableString instance. Ok, a surprise,
but is it so bad? If the object internally uses a mutable string, it's
likely that this is clear from the nature of the object (eg. it's some
sort of editor for the string). So I still think such a surprise will be
rare.

> See this message -- it is still quite relevant.  Mike Ferris is one
> of the guys who helped write the AppKit and Project Builder.
> 
> http://www.omnigroup.com/mailman/archive/macosx-dev/2000-May/001886.
> html

It's an example of NSArray, which we don't convert, so we live
dangerously anyway in PyObjC as we don't even have the hint from the
header that we _shouldn't_ mutate even if it is mutable.

Just

Re: [Pyobjc-dev] NSString & mutability

[David E. <epp...@ic...>]

On 2/4/03 3:38 PM -0500 bb...@ma... wrote:
> On Tuesday, Feb 4, 2003, at 15:21 US/Eastern, Just van Rossum wrote:
>> Someone on comp.sys.mac.programmer.help suggested a trick to find out
>> whether an NS[CF]String is mutable or not: by comparing classForCoder
>> to
>> NSString. Nifty.
>>
>> This allowed me to patch objc_support.m so it only converts immutable
>> strings to Python strings:
>
> Nice hack, but....
>
> What about all the code that might return a mutable string depending on
> how the developer exercises the API?  Are you willing to assume that
> every getter and setter that internally makes a immutable copy of the set
> string value?  i.e. that...

If some code is going to do that, it is certainly worse to convert the 
mutable string to a python string, because then the converted string 
wouldn't see any mutations that might be made later on the objc side.

I could live with not converting any NSStrings, as long as the Python side 
views the unconverted NSString as something that's mostly interchangeable 
with real str's.  If there is some code that really needs its argument to 
be a str, one could always explicitly convert with str(), but I'd like such 
conversions to be rare.  Just like in pure Python, where most of the time 
anything that obeys the list or iterator protocol can be passed in place of 
a list, but sometimes you need an explicit call to list().

--
David Eppstein       UC Irvine Dept. of Information & Computer Science
epp...@ic... http://www.ics.uci.edu/~eppstein/

Re: [Pyobjc-dev] NSString & mutability

[Just v. R. <ju...@le...>]

Just van Rossum wrote:

> However, str() fails:
> 
>   >>> str(x)
>   Traceback (most recent call last):
>     File "<stdin>", line 1, in ?
>   TypeError: __str__ returned non-string (type NSCFString)
>   >>> 
> 
> I can't seem to find where I can repair this.

Ok, I understand why it happens: __str__ returns [obj description], and
when we convert _all_ strings, this automatically works correctly, but
if we don't it doesn't <wink>, since it seems any NSString just returns
self from description. But now I don't know how to work around that...
Any hints? (Btw. I've found Lib/objc/_convenience.py.)

Just

Re: [Pyobjc-dev] NSString & mutability

[<bb...@ma...>]

On Tuesday, Feb 4, 2003, at 15:21 US/Eastern, Just van Rossum wrote:
> Someone on comp.sys.mac.programmer.help suggested a trick to find out
> whether an NS[CF]String is mutable or not: by comparing classForCoder 
> to
> NSString. Nifty.
>
> This allowed me to patch objc_support.m so it only converts immutable
> strings to Python strings:

Nice hack, but....

What about all the code that might return a mutable string depending on 
how the developer exercises the API?  Are you willing to assume that 
every getter and setter that internally makes a immutable copy of the 
set string value?  i.e. that...

...
[fooObject setMyNameIs: someRandomMutableString];
return [fooObject myNameIs]
...

... that -myNameIs will return an immutable string?

See this message -- it is still quite relevant.  Mike Ferris is one of 
the guys who helped write the AppKit and Project Builder.

http://www.omnigroup.com/mailman/archive/macosx-dev/2000-May/001886.html

Re: [Pyobjc-dev] NSString & mutability

[David E. <epp...@ic...>]

On 2/4/03 9:21 PM +0100 Just van Rossum <ju...@le...> wrote:
> Bill, this is not to say I'm _completely_ opposed to wrapping instead of
> converting strings, but more that I'd like to see how "partial
> conversion" (ie. don't convert if the string is mutable) works in
> practice.

I also feel that partial conversion is likely to be the most intuitive 
option.
--
David Eppstein       UC Irvine Dept. of Information & Computer Science
epp...@ic... http://www.ics.uci.edu/~eppstein/

[Pyobjc-dev] NSString & mutability

[Just v. R. <ju...@le...>]

Someone on comp.sys.mac.programmer.help suggested a trick to find out
whether an NS[CF]String is mutable or not: by comparing classForCoder to
NSString. Nifty.

This allowed me to patch objc_support.m so it only converts immutable
strings to Python strings:

Index: objc_support.m
===================================================================
RCS file: /cvsroot/pyobjc/pyobjc/Modules/objc/objc_support.m,v
retrieving revision 1.20
diff -c -r1.20 objc_support.m
*** objc_support.m  31 Jan 2003 10:35:29 -0000  1.20
--- objc_support.m  4 Feb 2003 20:11:36 -0000
***************
*** 614,620 ****
            retobject = [(OC_PythonDictionary *)obj pyObject];
            Py_INCREF(retobject);
          }
!         else if ([obj isKindOfClass:[NSString class]])
          {
            /* All string classes seem to be subclasses of NSString.
            * We convert to a python string where possible, and a python
--- 614,621 ----
            retobject = [(OC_PythonDictionary *)obj pyObject];
            Py_INCREF(retobject);
          }
!         else if ([obj isKindOfClass:[NSString class]] &&
!                  [obj classForCoder] == [NSString class])
          {
            /* All string classes seem to be subclasses of NSString.
            * We convert to a python string where possible, and a python


This works nicely:

  >>> from Foundation import *
  >>> x = NSMutableString.stringWithString_("asdasd")
  >>> x
  <NSCFString objective-c instance 0x430840>
  >>> x.appendString_("aaa")
  >>> repr(x)
  '<NSCFString objective-c instance 0x430840>'

However, str() fails:

  >>> str(x)
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  TypeError: __str__ returned non-string (type NSCFString)
  >>> 

I can't seem to find where I can repair this.

Bill, this is not to say I'm _completely_ opposed to wrapping instead of
converting strings, but more that I'd like to see how "partial
conversion" (ie. don't convert if the string is mutable) works in
practice.

Just

Re: [Pyobjc-dev] testing NSCell

[Just v. R. <ju...@le...>]

bb...@ma... wrote:

> test because of representation issues.... anyone have an 
> 'assertApproximatelyEquals()' test?

assertAlmostEqual() was recently added I think. Since this isn't
available in 2.2 I just changed the value to one that _is_ representable
exactly in binary.

Just

Re: [Pyobjc-dev] testing NSCell

[David E. <epp...@ic...>]

On 2/4/03 2:44 PM -0500 bb...@ma... wrote:
> Fixed in CVS.   I forgot about that, as well.
>
> In any case, the comitted unit test still fails on the floatValue() test
> because of representation issues.... anyone have an
> 'assertApproximatelyEquals()' test?

Really?  It was working for me after I changed the test value from 3.14159 
to 3.0 (included in my previous correction).

> BTW:  self.assert_ or self.assertEquals is a better test in a TestCase
> than assert() in that assert() will go away with optimization.

Ok, thanks.  I guess assertEquals is the one to use here...
--
David Eppstein       UC Irvine Dept. of Information & Computer Science
epp...@ic... http://www.ics.uci.edu/~eppstein/

Re: [Pyobjc-dev] testing NSCell

[<bb...@ma...>]

Fixed in CVS.   I forgot about that, as well.

In any case, the comitted unit test still fails on the floatValue() 
test because of representation issues.... anyone have an 
'assertApproximatelyEquals()' test?

BTW:  self.assert_ or self.assertEquals is a better test in a TestCase 
than assert() in that assert() will go away with optimization.

b.bum

On Tuesday, Feb 4, 2003, at 14:25 US/Eastern, David Eppstein wrote:
> Sorry, a couple of the tests failed due to me not reading the NSCell 
> docs sufficiently carefully -- specifically the part about needing to 
> become a text-type cell before setIntValue_ and setFloatValue_ will 
> work.  Here's a corrected version.  Now all tests still fail, but in 
> the same place: the objc accessors work, but the pythonic coercions 
> don't.

Re: [Pyobjc-dev] testing NSCell

[<bb...@ma...>]

Tests committed as 'test_nscell.py' in Lib/AppKit/test/.

I refactored the tests slightly... have a look, they have been comitted.

There is definitely a bug -- but it may be fallout from using an NSCell 
outside of a true application [NSCell is one of the "weird ones", 
IIRC].   Specifically, -floatValue and -intValue are always returning 0.

 >>> from AppKit import *
 >>> c = NSCell.alloc().init()
 >>> c.setIntValue_(5)
 >>> c.intValue()
0
 >>> c.stringValue()
''
 >>> c.setStringValue_('5')
 >>> c.intValue()
5
 >>> c.stringValue()
'5'
 >>>

Huh? Specific problem appears to be with setIntValue_()... how odd.

str(), unicode(), int(), and float() should not work directly on a cell 
instance.  A Cell is not really a data container.   It is a thing that 
draws stuff [and may support editing/clicking].  That it can contain 
data is one of its features, but not a primary feature and not intended 
for persistence [example:  a single cell may be used to draw all of the 
geographic cells in a single column of a table view].

As such, you should generally use the *Value() query methods.

Of course, with that said, it would be trivial to add mappings between 
the various *Value() methods and the appropriate __python method__.  
Something like...

CONVENIENCE_METHODS['floatValue'] = (
     ('__float__', lambda self: self.floatValue()),
)

While trivial, I'm not really sure we *should* add such convenience 
hooks -- I'm exactly on the fence.  What is the Properly Pythonic 
answer?

b.bum

On Tuesday, Feb 4, 2003, at 14:16 US/Eastern, David Eppstein wrote:

> Which if any of the following tests should fail?  Currently, they all 
> do.
>
> import unittest
> import objc
>
> from AppKit import NSCell
>
> class TestNSCell(unittest.TestCase):
>    cell = NSCell.alloc().init()
>
>    def testString(self):
>        s = 'string'
>        self.cell.setStringValue_(s)
>        assert(self.cell.stringValue() == s)
>        assert(str(self.cell) == s)
>
>    def testUnicode(self):
>        u = u'\xc3\xbc\xc3\xb1\xc3\xae\xc3\xa7\xc3\xb8d\xc3\xa8'
>        self.cell.setStringValue_(u)
>        assert(self.cell.stringValue() == u)
>        assert(unicode(self.cell) == u)
>
>    def testInt(self):
>        i = 17
>        self.cell.setIntValue_(i)
>        assert(self.cell.intValue() == i)
>        assert(int(self.cell) == i)
>
>    def testFloat(self):
>        f = 3.14159
>        self.cell.setFloatValue_(f)
>        assert(self.cell.floatValue() == f)
>        assert(float(self.cell) == f)
>
> def suite():
>    suite = unittest.TestSuite()
>    suite.addTest(unittest.makeSuite(TestNSCell))
>    return suite
>
> if __name__ == '__main__':
>    unittest.main( )
>
> --
> David Eppstein       UC Irvine Dept. of Information & Computer Science
> epp...@ic... http://www.ics.uci.edu/~eppstein/
>
>
>
> -------------------------------------------------------
> This SF.NET email is sponsored by:
> SourceForge Enterprise Edition + IBM + LinuxWorld = Something 2 See!
> http://www.vasoftware.com
> _______________________________________________
> Pyobjc-dev mailing list
> Pyo...@li...
> https://lists.sourceforge.net/lists/listinfo/pyobjc-dev
>
b.bum
No Chunks...
	... No Foul!