Thread: [Pyobjc-dev] Achieving 1.0....
Brought to you by:
ronaldoussoren
Menu
▾
▴
pyobjc-dev
|
From: Bill B. <bb...@co...> - 2002-10-22 05:25:40
|
[This is not a request to do a 1.0 release without an official interim
release -- just some planning for the real live 1.0]
In the interests in achieving a 1.0 release, I thought I would take a
very brief moment to toss out some ideas of what the project needs to
get there.
---
First and foremost, we need a unit testing suite that provides complete
coverage for the modules features. As it stands, I keep stepping on
Ronald's toes and he keeps stepping on mine -- once we work through the
issues, the end result tends to work everywhere, but the interim is a
pain for both of us. This is largely because Ronald and I work against
different builds of python [he uses a framework build of 2.3 alpha and
I use the Apple supplied-- slightly incomplete-- build of 2.2] and work
with a different set of examples largely to provide testing.
I would prefer to see a set of unit tests for each of the major modules
that takes the form used by the rest of Python. If I remember
correctly, that had been changing over recent versions-- maybe so,
maybe not. I would also prefer to see the test suites ship with the
binary distribution of the module (like Python, though Apple's build
doesn't ship with the test suites).
---
Documentation:
- need API documentation of the major functionality that isn't a
part of the standard ObjC api. (example: the selector() function).
A lot of this material exists. Would be very good/cool if it were in
pydoc compatible form.
- need conceptual documentation. Ronald started a user's guide
that looks excellent -- need to flesh it out.
- need example documentation; fairly out of date -- mostly my
examples. :-)
---
Examples:
- need a general clean up and synchronization with underlying
features (example: web services tool currently does a bunch additional
selector() definitions that are likely completely unnecessary)
- remove examples that no longer work correctly or are rendered
moot by unit testing
---
Remove all references to True and False throughout codebase
[assuming we are to continue supporting 2.2 throughout] -- or declare
'em somewhere.
Make sure the 2.2 support is appropriately isolated and documented
[and doesn't affect 2.3+ support].
More as I think of it... I have to sleep now.
b.bum
|
|
From: Ronald O. <ous...@ci...> - 2002-10-22 20:40:08
|
On Tuesday, Oct 22, 2002, at 07:25 Europe/Amsterdam, Bill Bumgarner wrote: > [This is not a request to do a 1.0 release without an official interim > release -- just some planning for the real live 1.0] Speaking of interim releases... Now that people are getting interrested in PyObjC it would be better to get one out sometime soon. > > In the interests in achieving a 1.0 release, I thought I would take a > very brief moment to toss out some ideas of what the project needs to > get there. > > --- > > First and foremost, we need a unit testing suite that provides > complete coverage for the modules features. As it stands, I keep > stepping on Ronald's toes and he keeps stepping on mine -- once we > work through the issues, the end result tends to work everywhere, but > the interim is a pain for both of us. This is largely because Ronald > and I work against different builds of python [he uses a framework > build of 2.3 alpha and I use the Apple supplied-- slightly > incomplete-- build of 2.2] and work with a different set of examples > largely to provide testing. I try to test new features using 'Apple Python', but my primary Python environment is Python 2.3alpha. Having a unit testing suite would indeed be very usefull. I'd like to use PyUnit (which IIRC is what is used by the Python testsuite). > Documentation: Documentation is a very much needed. There is some documentation, but it needs to be fleshed out. Luckily we don't need a lot of it :-) > > --- > > Examples: > > - need a general clean up and synchronization with underlying > features (example: web services tool currently does a bunch > additional selector() definitions that are likely completely > unnecessary) > > - remove examples that no longer work correctly or are rendered > moot by unit testing <nod> Most of the current examples are actually scripts that I used to test the code. > --- > > Remove all references to True and False throughout codebase > [assuming we are to continue supporting 2.2 throughout] -- or declare > 'em somewhere. I'd prefer to declare True and False on Python 2.2.0. Although the values don't really add functionality they do help with readability. Ronald |
|
From: Bill B. <bb...@co...> - 2002-10-22 20:47:16
|
Agreed. The differences between 0.6.1 and 0.7.0 are huge and the visibility for the module within the dev community is going to grow quite a bit in the near future [I hope -- will say more when I know more]. How about a feature freeze -- fix the bugs we have right now that are relatively major [dictionary / array allocation, the instance identification] and add unit tests to ensure that whatever we ship as 0.7.0 (or whatever) is relatively solid? I'm pretty happy with the latest incarnation of bin-python-main.m -- need to commit a new version and update the python project template -- as it has proven to not be problematic in the production app I'm working on. It now supports standalone builds with frameworks (dynamically loading frameworks with @executable_path as their install_name AND dependencies on other frameworks w/@executable_path requires some fun to be had with the environment variables of the execve() process). I'll try to re-address the documentation on the examples that use Project Builder and put together a README for the Project Template stuff. b.bum On Tuesday, October 22, 2002, at 04:40 PM, Ronald Oussoren wrote: >> [This is not a request to do a 1.0 release without an official >> interim release -- just some planning for the real live 1.0] > Speaking of interim releases... Now that people are getting > interrested in PyObjC it would be better to get one out sometime soon. |
|
From: Ronald O. <ous...@ci...> - 2002-10-22 21:03:01
|
On Tuesday, Oct 22, 2002, at 22:47 Europe/Amsterdam, Bill Bumgarner wrote: > How about a feature freeze -- fix the bugs we have right now that are > relatively major [dictionary / array allocation, the instance > identification] and add unit tests to ensure that whatever we ship as > 0.7.0 (or whatever) is relatively solid? I feature freeze is fine with me, I don't have any new version planned for the forseeable future :-) I'll have a look at the remaining know bugs. BTW. I increased the version number in CVS to avoid confusion with the installer on your home-page. > > I'm pretty happy with the latest incarnation of bin-python-main.m -- > need to commit a new version and update the python project template -- > as it has proven to not be problematic in the production app I'm > working on. It now supports standalone builds with frameworks > (dynamically loading frameworks with @executable_path as their > install_name AND dependencies on other frameworks w/@executable_path > requires some fun to be had with the environment variables of the > execve() process). I'll also have a more serious look bin-python-main.m. I definitely like the idea of just exec-ing a normal python with the right argv. The only thing I'm not sure about w.r.t. binary distribution of applications is honouring a 'defaults' settings for the location of the python binary. But that is something we can fix in our in-house version of python (we ship extension modules, and those might not work with a different version of python). > > I'll try to re-address the documentation on the examples that use > Project Builder and put together a README for the Project Template > stuff. Documentation, the mortal enemy of coders :-) Ronald |
|
From: Bill B. <bb...@co...> - 2002-10-22 21:23:23
|
On Tuesday, October 22, 2002, at 05:03 PM, Ronald Oussoren wrote:
> On Tuesday, Oct 22, 2002, at 22:47 Europe/Amsterdam, Bill Bumgarner
> wrote:
> .....
>>
>> I'm pretty happy with the latest incarnation of bin-python-main.m --
>> need to commit a new version and update the python project template
>> -- as it has proven to not be problematic in the production app I'm
>> working on. It now supports standalone builds with frameworks
>> (dynamically loading frameworks with @executable_path as their
>> install_name AND dependencies on other frameworks w/@executable_path
>> requires some fun to be had with the environment variables of the
>> execve() process).
> I'll also have a more serious look bin-python-main.m. I definitely
> like the idea of just exec-ing a normal python with the right argv.
>
> The only thing I'm not sure about w.r.t. binary distribution of
> applications is honouring a 'defaults' settings for the location of
> the python binary. But that is something we can fix in our in-house
> version of python (we ship extension modules, and those might not work
> with a different version of python).
It already does that and much, much more! :-)
See both the main-bin-python.m and the Main.py in the Web Services Tool
example. It is the "most correct" version I have created so far and
will be copied to the other projects / locations soon. Some of the
nifty features:
- auto dumps the pid for attaching to the child python process via
gdb if SHOWPID is set in the environment
- correctly sets up environment variables to deal w/linking
frameworks in production vs. in project builder
- traverses list of frameworks linked into the main-bin-python.m
binary and passes 'em off to the Main.py. Main.py dynamically loads
each framework, then checks each framework to see if it has an Init.py
script in Resources/ and executes it, if it does. I.e. frameworks will
auto-bootstrap their python stuff as they are loaded into the "real"
environment.
- can customize which python binary is used via the PythonBinPath
user default
- can customize the Main script used via the main bundles
PrincipalPythonFile dictionary entry; defaults to Main.py (should add
this to the project template, now that I think about it).
- everything is done w/UTF-8 encoded strings and, as such, should
work on any OS X installation [as the BSD layer all deals w/UTF-8
encoded strings for paths and such].
- setting the DYLD_PRINT_LIBRARIES user default to 1 will cause the
child python process to dump all of the dylibs as they are loaded
>> I'll try to re-address the documentation on the examples that use
>> Project Builder and put together a README for the Project Template
>> stuff.
> Documentation, the mortal enemy of coders :-)
I like using unit tests as documentation, personally. We [CodeFab]
have been using XP in varying levels of intensity over the past few
years. When it works, it works really well.
b.bum
|
|
From: Ronald O. <ous...@ci...> - 2002-10-28 09:00:41
|
All,
I just remembered one implementation issue that should be worked on
before reaching a 1.0 release: Thread safety.
The current code is probably save for multi-threading, but I haven't
really checked this. Basicly I removed all static/global buffers while
working on the code.
There are some more issues:
1. Python has a global interpreter lock. When one thread owns this
lock, no
other thread can execute python code. The bridge currently never
gives up
the GIL when moving into native code and we probably should. I have
not yet
researched what we should do with callbacks from Objective-C to
Python.
E.g. the NSApplicationMain wrapper should release the GIL before
calling
the actuall function, but how do you regain the GIL during calls to
methods
implemented in Python?
This is necessary to avoid unecessary blockage of Python threads
during
calls to Objective-C code.
2. Threads started using the Cocoa thread class(es) won't have a python
interpreter state. Special wrapping code should be written to
arrange for the
creation of that state. This can wait until item 1 is finished.
Ronald
|
|
From: pluto <pl...@ob...> - 2002-10-28 09:42:11
|
hi folks, i'm new to this list, i'm also new to macs and obj-c. i've been programming python for a several years as a living... and thats enough of an intro ;-) On Monday, October 28, 2002, at 01:00 AM, Ronald Oussoren wrote: > All, > > I just remembered one implementation issue that should be worked on > before reaching a 1.0 release: Thread safety. > > The current code is probably save for multi-threading, but I haven't > really checked this. Basicly I removed all static/global buffers while > working on the code. > > There are some more issues: > 1. Python has a global interpreter lock. When one thread owns this > lock, no > other thread can execute python code. The bridge currently never > gives up > the GIL when moving into native code and we probably should. I have > not yet > researched what we should do with callbacks from Objective-C to > Python. > > E.g. the NSApplicationMain wrapper should release the GIL before > calling > the actuall function, but how do you regain the GIL during calls to > methods > implemented in Python? > write a (objective) c wrapper that gets registered for the callback and takes a pointer to the python object to call. on callback reacquire the gil and call the object. > This is necessary to avoid unecessary blockage of Python threads > during > calls to Objective-C code. > > 2. Threads started using the Cocoa thread class(es) won't have a python > interpreter state. Special wrapping code should be written to > arrange for the > creation of that state. This can wait until item 1 is finished. > maybe it would be helpful to look at what pyqt or wxpython do in such cases. cheers, -p |
|
From: Jack J. <Jac...@cw...> - 2002-10-28 09:56:14
|
On Monday, Oct 28, 2002, at 10:00 Europe/Amsterdam, Ronald Oussoren wrote: > All, > > I just remembered one implementation issue that should be worked on > before reaching a 1.0 release: Thread safety. > > The current code is probably save for multi-threading, but I haven't > really checked this. Basicly I removed all static/global buffers while > working on the code. > > There are some more issues: > 1. Python has a global interpreter lock. When one thread owns this > lock, no > other thread can execute python code. The bridge currently never > gives up > the GIL when moving into native code and we probably should. I have > not yet > researched what we should do with callbacks from Objective-C to > Python. > > E.g. the NSApplicationMain wrapper should release the GIL before > calling > the actuall function, but how do you regain the GIL during calls to > methods > implemented in Python? I have sample code to do this, based on pthreads. You can have it if you want it, but it's probably just as easy to recode it (as you'll have to replace the pthreads stuff anyway, I would guess). The trick I used is to put pthread's per-thread data to work (I assume ObjC threads have a similar feature, yes?). You allocate an item of per-thread global data, which is automatically initialized to zero in all threads. Then when you switch from Python->ObjC you store the current PyThreadState in there. When you switch from PObjC to Python you get the per-thread global data item. If it is non-null it is the PyThreadState and you simply do PyEval_AcquireThread. If it is null then this is a thread that has never yet made the Python->ObjC transition. As it is apparently executing ObjC code right now it must have been created outside of Python. So, you create a new PyThreadState for it (with PyThreadState_New), store that in the per-thread global data and call PyEval_AcquireThread. -- - Jack Jansen <Jac...@or...> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - |
|
From: Bill B. <bb...@co...> - 2002-10-28 16:45:50
|
On Monday, October 28, 2002, at 04:00 AM, Ronald Oussoren wrote: > All, > > I just remembered one implementation issue that should be worked on > before reaching a 1.0 release: Thread safety. Agreed. > The current code is probably save for multi-threading, but I haven't > really checked this. Basicly I removed all static/global buffers while > working on the code. > > There are some more issues: > 1. Python has a global interpreter lock. When one thread owns this > lock, no > other thread can execute python code. The bridge currently never > gives up > the GIL when moving into native code and we probably should. I have > not yet > researched what we should do with callbacks from Objective-C to > Python. > > E.g. the NSApplicationMain wrapper should release the GIL before > calling > the actuall function, but how do you regain the GIL during calls to > methods > implemented in Python? > > This is necessary to avoid unecessary blockage of Python threads > during > calls to Objective-C code. I believe Jack covered this, but wanted to add a couple of Obj-C and NS* specific bits of information: As of OS X, NSThreads are actually built on top of pthreads and it should be possible to use Jack's code directly. From the documentation: ... NSThreads are built upon POSIX threads (pthreads). You can create your own pthreads, but some Cocoa classes, such as NSAutoreleasePool, expect to be running within an NSThread and may not work properly in a pthread. Creating a pthread does not notify Cocoa classes that the application has become multithreaded, so the NSThread method isMultiThreaded may return NO. If you need access to the pthread within an NSThread, call the pthread_self function from within that thread. ... I would still recommend using the NSThread based API to create threads if the goal is to multi-thread an app that mixes Python and Obj-C in threads (for an app that uses pure-python in the threads, there is no need) because this is the only way that the ObjC runtime is going to become "thread aware". Specifically, see the documentation on these notifications: FOUNDATION_EXPORT NSString * const NSWillBecomeMultiThreadedNotification; FOUNDATION_EXPORT NSString * const NSDidBecomeSingleThreadedNotification; FOUNDATION_EXPORT NSString * const NSThreadWillExitNotification; However, once the NSThread is created, the pthreads API can be used to store state and otherwise manipulate the thread -- the NSThread API is just the glue to make sure the ObjC runtime has the opportunity to deal w/the appropriate notifications. If desired, NSThread provides for storage of per-thread state in a mutable dictionary. Simply invoke -threadDictionary while executing in the thread that wants to store/retrieve the state. When traversing from Python into Obj-C in a thread that was created in pure python (i.e. is just a pthread, does not have an NSThread instance), invoking NSThread's +currentThread method will return an NSThread that encapsulates the already created pthread. I would assume that threaded python already has the pthread initialized with the appropriate state and it would just be a matter of adding any bridge specific state at this point? It sounds like the harder problem is when the developer crosses from ObjC -> Python in a thread that was not created in Python and, therefore, the interpreter has not initialized the pthread state in the specific context of Python? Note that as of 10.2, NSThread offers these two *extremely* handy methods: - (void)performSelectorOnMainThread:(SEL)aSelector withObject:(id)arg waitUntilDone:(BOOL)wait modes:(NSArray *)array; - (void)performSelectorOnMainThread:(SEL)aSelector withObject:(id)arg waitUntilDone:(BOOL)wait; b.bum |
|
From: Jack J. <Jac...@cw...> - 2002-10-29 10:51:09
|
On Monday, Oct 28, 2002, at 17:45 Europe/Amsterdam, Bill Bumgarner wrote: > When traversing from Python into Obj-C in a thread that was created in > pure python (i.e. is just a pthread, does not have an NSThread > instance), invoking NSThread's +currentThread method will return an > NSThread that encapsulates the already created pthread. If you do this, does that make a pthread a fullblown NSThread? I.e. does it make the problems you sketched earlier (such as isMultiThreaded returning NO) go away? Hmm, this may well point to a solution to a completely different problem I have (my multithreaded OpenGL programs stopped working as of 10.2).... Is there an easy way to do this from C? > I would assume that threaded python already has the pthread > initialized with the appropriate state and it would just be a matter > of adding any bridge specific state at this point? Yes. -- - Jack Jansen <Jac...@or...> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - |
|
From: Jack J. <Jac...@cw...> - 2002-10-29 12:16:35
|
On Tuesday, Oct 29, 2002, at 11:31 Europe/Amsterdam, Jack Jansen wrote: > > On Monday, Oct 28, 2002, at 17:45 Europe/Amsterdam, Bill Bumgarner > wrote: >> When traversing from Python into Obj-C in a thread that was created >> in pure python (i.e. is just a pthread, does not have an NSThread >> instance), invoking NSThread's +currentThread method will return an >> NSThread that encapsulates the already created pthread. > > If you do this, does that make a pthread a fullblown NSThread? I.e. > does it make the problems you sketched earlier (such as > isMultiThreaded returning NO) go away? To answer my own question: no, it doesn't. isMultiThreaded still returns NO after this call. Is this a problem? If so, is there a way to tell NSThread that we *are* multithreaded? -- - Jack Jansen <Jac...@or...> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - |
|
From: Bill B. <bb...@co...> - 2002-10-29 13:07:13
|
On Tuesday, October 29, 2002, at 07:16 AM, Jack Jansen wrote: > On Tuesday, Oct 29, 2002, at 11:31 Europe/Amsterdam, Jack Jansen wrote: >> On Monday, Oct 28, 2002, at 17:45 Europe/Amsterdam, Bill Bumgarner >> wrote: >>> When traversing from Python into Obj-C in a thread that was created >>> in pure python (i.e. is just a pthread, does not have an NSThread >>> instance), invoking NSThread's +currentThread method will return an >>> NSThread that encapsulates the already created pthread. >> >> If you do this, does that make a pthread a fullblown NSThread? I.e. >> does it make the problems you sketched earlier (such as >> isMultiThreaded returning NO) go away? > > To answer my own question: no, it doesn't. isMultiThreaded still > returns NO after this call. > Is this a problem? If so, is there a way to tell NSThread that we > *are* multithreaded? That would seem to be a bug that should be reported to bugreport.apple.com. If you have a little hunk of test code, that typically accelerates Apple's bug fixing process considerably. It sounds like you'll need to create a zero resources thread via NSThread: [NSThread detachNewThreadSelector:@selector(intoTheVoid:) toTarget:[NSObject class] withObject: nil]; @implementation NSObject (TheVoid) + (void) intoTheVoid: aNothing { l = [[NSLock alloc] init]; [l lock]; [l lock]; } @end This will effectively dead lock the thread in a fashion that should use zero CPU cycles (assuming NSLock is correctly implemented). There is also no way to kill the thread -- export the NSLock and call [l unlock] once to do so. Bogus. Totally Bogus. But should do the trick. It is likely that we will need some similar bogosity in the PyObjC bridge. b.bum |
|
From: Jack J. <Jac...@or...> - 2002-10-29 21:17:55
|
On dinsdag, oktober 29, 2002, at 02:06 , Bill Bumgarner wrote: >> To answer my own question: no, it doesn't. isMultiThreaded >> still returns NO after this call. >> Is this a problem? If so, is there a way to tell NSThread that >> we *are* multithreaded? > > That would seem to be a bug that should be reported to > bugreport.apple.com. If you have a little hunk of test code, > that typically accelerates Apple's bug fixing process > considerably. I'll try and find the time, but feel free to beat me to it:-) > > It sounds like you'll need to create a zero resources thread > via NSThread: > > [NSThread detachNewThreadSelector:@selector(intoTheVoid:) > toTarget:[NSObject class] withObject: nil]; > > @implementation NSObject (TheVoid) > + (void) intoTheVoid: aNothing > { > l = [[NSLock alloc] init]; > [l lock]; > [l lock]; > } > @end I did this (but of course with a bit of code a lot less elegant than yours:-), and it solves the isMultiThreaded problem. Unfortunately it doesn't solve my PyOpenGL problem, but that's a different story... Something else you may want to try though, if this is needed for PyObjC, is to let the thread terminate. The comments in NSThread.h lead me to believe that once you've made the transition to a multithreaded program there's no going back. If that's indeed the case then letting the thread terminate isn't a problem. -- - Jack Jansen <Jac...@or...> http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - |
|
From: Bill B. <bb...@co...> - 2002-10-30 01:10:08
|
On Tuesday, October 29, 2002, at 04:17 PM, Jack Jansen wrote:
> Something else you may want to try though, if this is needed for
> PyObjC, is to let the thread terminate. The comments in NSThread.h
> lead me to believe that once you've made the transition to a
> multithreaded program there's no going back. If that's indeed the case
> then letting the thread terminate isn't a problem.
I wrote a little hunk of code to verify the behavior and you are
correct-- a program can only become multithreaded, but will never
revert to single threaded. However, given that there is both a
NSWillBecomeMultiThreadedNotification and a
NSDidBecomeSingleThreadedNotification notification, it would seem that
Apple intends to someday support the transition from multi->single
threaded.
As such -- I would recommend leaving in the deadlocked thread.
NSDidBecomeSingleThreadedNotification is currently documented as "Not
Implemented.".
(To test this case, create a new "foundation tool" project and replace
the contents of main.m with the code shown below.
// --- start of code
#import <Foundation/Foundation.h>
@interface NSObject (TheVoid)
+ (void) intoTheVoid: aNothing;
@end
@implementation NSObject (TheVoid)
+ (void) intoTheVoid: aNothing;
{
/* NSLock *l = [[NSLock alloc] init];
[l lock];
[l lock];*/
}
- (void) receiveNotification: aNotification
{
NSLog(@"Recieved %@", aNotification);
}
- (void) timerFired: anObject
{
NSLog(@"Timer fired.");
}
@end
int main (int argc, const char * argv[]) {
NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
[[NSNotificationCenter defaultCenter]
addObserver: [NSObject class]
selector: @selector(receiveNotification:)
name: nil
object: nil];
[NSThread detachNewThreadSelector:@selector(intoTheVoid:)
toTarget:[NSObject class] withObject: nil];
[NSTimer scheduledTimerWithTimeInterval: 5.0
target: [NSObject class]
selector: @selector(timerFired:)
userInfo: nil
repeats: NO];
[[NSRunLoop currentRunLoop] runUntilDate: [NSDate distantFuture]];
[pool release];
return 0;
}
// ------- end of code
2002-10-29 18:19:29.512 threadtest[14834] Recieved
NSConcreteNotification 5a740 {name =
NSWillBecomeMultiThreadedNotification}
2002-10-29 18:19:29.558 threadtest[14834] Recieved
NSConcreteNotification 5a740 {name = _NSThreadDidStartNotification}
2002-10-29 18:19:29.629 threadtest[14834] Recieved
NSConcreteNotification 5cec0 {name = NSThreadWillExitNotification;
object = <NSThread: 0x5ce80>{num = 2, threadDictionary =
(null)
}}
2002-10-29 18:19:34.559 threadtest[14834] Timer fired.
threadtest has exited with status 0.
b.bum
We gladly feast on those who would subdue us.
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X