From: Daniel A. S. <st...@ic...> - 2005-04-19 10:38:24
|
Darwin has two formats for dynamically loadable binaries: shared libraries aka .dylibs (MH_DYLIB) and .bundles (MH_BUNDLE). Currently tclLoadDyld.c only supports [load]ing .dylibs, the following patch adds [load]ing of .bundles: http://rutherglen.ics.mq.edu.au/~steffen/tcltk/patches/tcl- bundle_dyld.diff this modifies TclpDlopen() to attempt loading a file as a .bundle via NSCreateObjectFileImageFromFile() when NSAddImage() has failed to lad it as a .dylib. The main user-visible advantage of [load]ing .bundles is that such extensions can be [unload]ed, whereas dyld does not actually support unloading of .dylibs even if [unload] appears to succeed (the patch also removes attempting to unload .dylibs). MH_BUNDLE is also the binary format used by loadable extensions for other languages on the platform (perl, python, apache,...), which brings us more in line which their usage (the original tcl port to Darwin was using .bundles as well but we switched to .dylibs at some point to make porting of extension buildsystems easier). Indeed, the disadvantage of .bundles is that they need to be linked differently than shared libraries, which introduces a platform discrepancy in buildsystems (e.g. TEA3 does not support linking as .bundle at present), and that unlike shared libs, .bundles cannot be linked against from other binaries (which is a problem for Tk and for extensions like expect that have dependent binaries). Another advantage of .bundles is that OSX 10.3 allows MH_BUNDLE binaries to be loaded from memory; this allows direct [load]ing of .bundles from VFSs such as starkits, instead of having to first copy them out of VFS to a temporary location as is done in tcl currently. In addition to the changes above (which are independent of this), the patch adds that capability to TclLoadFile() in tclIOUtil.c (when TCL_LOAD_FROM_MEMORY is #defined), via new platform specific procs TclpLoadMemoryGetBuffer() and TclpLoadMemory(). These are only implemented in tclLoadDyld.c at present, but other platforms may have the capability to load binary code from memory as well? I would be grateful for review and comments of these changes, esp concerning availability of loading code from memory on other platforms. Given that this change only affects internal APIs I don't believe it needs to be TIPd but the requirements of other platforms should be considered in the design... Another question is if some or all of this should be backported to core-8-4-branch? As a separate issue, another small change in the patch to tcl.m4 shows how to use the new MODULE_SCOPE macro on Darwin to prevent tcl internal symbols to be exported from the tcl shared library. Unfortunately, as shown in tclLoadDyld.c, for this to work on Darwin, MODULE_SCOPE needs to be added to _both_ the declaration and the implementation of an internal function. Does anybody know if the presence of MODULE_SCOPE on function implementations will cause problems on other platforms? (if undefined, MODULE_SCOPE is #defined to be 'extern' in tclInt.h) If yes, we may need to introduce a separate macro for module scope implementations. Cheers, Daniel -- ** Daniel A. Steffen ** "And now for something completely ** Dept. of Mathematics ** different" Monty Python ** Macquarie University ** <mailto:st...@ma...> ** NSW 2109 Australia ** <http://www.maths.mq.edu.au/~steffen/> |
From: Daniel A. S. <st...@ic...> - 2005-05-15 05:45:30
|
On 19/04/2005, at 20:37, Daniel A. Steffen wrote: > OSX 10.3 allows MH_BUNDLE binaries to be loaded from memory; this > allows direct [load]ing of .bundles from VFSs such as starkits, > instead of having to first copy them out of VFS to a temporary > location as is done in tcl currently. > the patch adds that capability to TclLoadFile() in tclIOUtil.c (when > TCL_LOAD_FROM_MEMORY is #defined), via new platform specific procs > TclpLoadMemoryGetBuffer() and TclpLoadMemory(). These are only > implemented in tclLoadDyld.c at present, but other platforms may have > the capability to load binary code from memory as well? > > I would be grateful for review and comments of these changes, esp > concerning availability of loading code from memory on other > platforms. Given that this change only affects internal APIs I don't > believe it needs to be TIPd but the requirements of other platforms > should be considered in the design... I have backported these changes to core-8-4-branch and have updated the HEAD patch: http://sourceforge.net/tracker/index.php? func=detail&aid=1202209&group_id=10894&atid=110894 If I hear no objections, I'll commit this for 8.4.10/8.5a3 Cheers, Daniel -- ** Daniel A. Steffen ** "And now for something completely ** Dept. of Mathematics ** different" Monty Python ** Macquarie University ** <mailto:st...@ma...> ** NSW 2109 Australia ** <http://www.maths.mq.edu.au/~steffen/> |
From: Zoran V. <zv...@ar...> - 2005-05-15 06:46:37
|
Am 15.05.2005 um 07:44 schrieb Daniel A. Steffen: > > I have backported these changes to core-8-4-branch and have updated > the HEAD patch: > http://sourceforge.net/tracker/index.php? > func=detail&aid=1202209&group_id=10894&atid=110894 > > If I hear no objections, I'll commit this for 8.4.10/8.5a3 > +TclpLoadMemoryGetBuffer(interp, size) + Tcl_Interp *interp; /* Used for error reporting. */ + int size; /* Size of desired buffer */ +{ + static int haveLoadMemory = -1; + void * buffer = NULL; + + if (haveLoadMemory < 0) { Shouldn't this one be serialized because of the static storage used? Zoran |
From: Daniel A. S. <st...@ic...> - 2005-05-15 07:19:05
|
On 15/05/2005, at 16:46, Zoran Vasiljevic wrote: > +TclpLoadMemoryGetBuffer(interp, size) > + Tcl_Interp *interp; /* Used for error reporting. */ > + int size; /* Size of desired buffer */ > +{ > + static int haveLoadMemory = -1; > + void * buffer = NULL; > + > + if (haveLoadMemory < 0) { > > Shouldn't this one be serialized because of the static storage used? hmm, good point, the following should fix the potential race static int haveLoadMemory = -1; if (haveLoadMemory < 0) { /* NSCreateObjectFileImageFromMemory is available but always * fails prior to Darwin 7 */ struct utsname name; haveLoadMemory = !uname(&name) && strtol(name.release, NULL, 10) >= 7; } concurrent first runs of TclpLoadMemoryGetBuffer() would still cause the check to be run more than once, but given that it will yield the same result and that haveLoadMemory is now modified atomically, that will cause no problems. Cheers, Daniel -- ** Daniel A. Steffen ** "And now for something completely ** Dept. of Mathematics ** different" Monty Python ** Macquarie University ** <mailto:st...@ma...> ** NSW 2109 Australia ** <http://www.maths.mq.edu.au/~steffen/> |
From: Zoran V. <zv...@ar...> - 2005-05-15 07:25:26
|
Am 15.05.2005 um 09:18 schrieb Daniel A. Steffen: > concurrent first runs of TclpLoadMemoryGetBuffer() would still > cause the check to be run more than once, but given that it will > yield the same result and that haveLoadMemory is now modified > atomically, that will cause no problems. > Hm... you are relying on the setting the integer to be atomic operation. I can imagine this being so on many modern platforms, but I would not bet on that. I'd suggest to either move this into TSD or serialize it with the mutex. I know I'm sounding paranoid, but the experience has tought me to be extra careful. You never know when is this going to backfire at you. Zoran |
From: Daniel A. S. <st...@ic...> - 2005-05-19 12:23:27
|
On 15/05/2005, at 17:24, Zoran Vasiljevic wrote: > Hm... you are relying on the setting the integer to be atomic > operation. > I can imagine this being so on many modern platforms, but I would not > bet on that. I'd suggest to either move this into TSD or serialize it > with the mutex. given that this code is in tclLoadDyld.c it will only ever run on ppc or i386, where this is true I think. But you're right, it's better to avoid this type of fragile trickery, it doesn't cost anything much to do the uname check once in every thread instead of once per process, so I'll move the init check to TSD. Cheers, Daniel -- ** Daniel A. Steffen ** "And now for something completely ** Dept. of Mathematics ** different" Monty Python ** Macquarie University ** <mailto:st...@ma...> ** NSW 2109 Australia ** <http://www.maths.mq.edu.au/~steffen/> |