From: Daniel H. <da...@gm...> - 2008-07-21 12:05:41
|
I will explain some parts of the project again. The aim of this project is to allow shared libraries to be loaded from memory (instead from files on disk). The problem is that the first argument of function 'dlopen' is a path to a file. My first attempt was to make some modifications in the standard glibc library. Anyway, it was not acceptable because of licencions: glibc is on GPL whareas the Tcl is on FreeBSD licence. These two licences does not match with each other. Then I found your RTLD library (http://opengrok.creo.hu/dragonfly/xref/src/libexec/rtld-elf/.). It was a part of the FreeBSD kernel, so I could not compile it on Linux. In order to do it, I commented out some code and added a new file defs.h in which I included all #defines as well as some functions that are used in the RTLD but are defined in other files of the FreeBSD kernel. Anyway, this code does not work. At this point I have no idea why exactly. I mean it works for some simple libraries (like libcos, libstrcpy, libwrite_fun, libwrite_global which are defined in trunk/libs), but it does not work for example for libprintf which simply uses 'printf' function. 'printf' itself is a specific function because it may have various number of arguments, but if we use 'putc' instead it does not work as well. The sources of the project can be found in a public repository http://svn.assembla.com/svn/gsoc2008-tcl_dynamic_libraries. Again I would be grateful if you found some time to give us a little help. Best, Daniel 2008/7/21 Daniel Hans <da...@gm...>: > Hello John, > > I would not want you to think I am pushy, but have you already > returned? We tried to contact Alexander Kabaev but we have not receive > any response from him. We would be grateful if you could take a look > at our code and make some suggestions of what may be wrong there. > > Greetings, > Daniel > > 2008/7/12 Daniel Hans <da...@gm...>: >> Hello, >> >> I am working on a Google Summer of Code projects for the Tcl >> community. The main goal of this project is to allow loading dynamic >> libraries not only from files on disk but also from memory images. For >> this we want to use the RTLD library. Since the RTLD is like a part of >> the FreeBSD kernel, I made some changes there (for example I commented >> out some code) in order to compile it as an application on Linux. >> Anyway, it works for loading very simple libraries, but it does not >> work for other libraries (which are also simple). >> >> Since none of us is an expert in this area we asked for help John >> Polstra. However, he is going to be traveling for the next week and we >> are getting out of time (GSoC ends in a month), so he has given us >> your email as you have worked with this code as well. >> >> More details are described below. Could you be so kind to take a look >> at our code and make a guess what may be wrong? >> >> Here is a link to a project repository: >> http://svn.assembla.com/svn/gsoc2008-tcl_dynamic_libraries >> You can find rtld code in /trunk/rtld or in archive RTLD.zip in /trunk >> >> I will be grateful for any kind of help. >> >> Regards, >> Daniel >> >> 2008/7/12 John Polstra <jd...@po...>: >>> I'm glad you followed up, because I didn't receive Daniel's mail. More >>> about that later. >>> >>> I'd be happy to help with this, but I am going to be traveling for about a >>> week and can't look at it until I return. If you can wait that long, fine; >>> if not, please contact one of the more recent maintainers of the FreeBSD >>> dynamic linker. The person who has worked on it the most in the past 5 >>> years is Alexander Kabaev <ka...@fr...>. He no doubt knows the code >>> better than I do, because I haven't looked at it for many years. >>> >>> Daniel, it looks like my mail system discarded your mail because it had an >>> attachment containing a zip file. I accept .tar.gz files, but not zip >>> files. Sorry for the inconvenience. In my world, zip files are practically >>> always Windows viruses. >>> >>> John >>> >>> Andreas Kupries wrote: >>>> >>>> I hope nobody gets upset for me butting in here for a bit. >>>> >>>>> Hello, >>>>> >>>>> I was given your email address from Arnulf Wiedemann who is a member >>>>> of the Tcl community. >>>>> >>>>> I am currently working on an Google Summer of Code project and >>>>> writing to you >>>>> because you are an author of the RTLD for FreeBSD. >>>>> >>>>> The main object of my project is to allow loading libraries >>>>> dynamically from memory. Normally if we want to load a library from a >>>>> file we can just use dlopen(), but I have to load libraries which are >>>>> mapped in memory. >>>> >>>> For a bit more context, Tcl is (of course) just fine with loading shared >>>> libraries (containing binary Tcl packages) from disk, using dlopen(). >>>> >>>> Tcl however also has a feature called 'virtual filesystems' and uses this >>>> feature to enable the generation of single-file executables. >>>> >>>> Which is in essence a Tcl interpreter with a virtual filesystem image of >>>> some type attached to it. When such an executable is run the interpreter >>>> pulls the files it needs out of this embedded filesystem. At that point >>>> the >>>> files exist as blobs in memory. Tcl scripts can then be executed directly >>>> out of such blobs. Our GSoC project is about making this possible for >>>> shared >>>> libraries as well. The only platform there this is currently possible is >>>> OS >>>> X, which has direct OS support for this type of thing. On all other >>>> platforms the blob currently is (has to be) written to a temporary file on >>>> disk which is then handed to dlopen() to make it usable. >>>> >>>> Regarding all the people in the recipient list >>>> >>>> Daniel H. is the student tasked with implementing this feature. >>>> >>>> I (Andreas) am the primary mentor with advice on the Tcl side of >>>> things. >>>> >>>> Daniel S. is the person who implemented the above-mentioned >>>> load-from-memory feature for OS X and is my backup mentor. >>>> >>>> Tomasz is a mentor as well, his task is to watch for and help us in >>>> case of >>>> language/translation trouble cropping up. >>>> He is Polish as is Daniel H. >>>> >>>> Tcl-Core @ SF is the primary mailing list for development on the >>>> Tcl core. >>>> >>>> >>>>> My first thought was to modify some functions from glibc, but glibc is >>>>> GPL and this license is not appropriate for my project. Then I found >>>>> the RLTD library and thought of making some changes in its code. I am >>>>> not working with the latest version. The one I use can be found here >>>>> http://opengrok.creo.hu/dragonfly/xref/src/libexec/rtld-elf/. >>>> >>>> While Daniel attached the RTLD sources he used, the whole project can also >>>> be retrieved from a svn repository, its location is >>>> >>>> andreask@gila:~/workbench/shmemload/head> svn info >>>> Path: . >>>> URL: http://svn.assembla.com/svn/gsoc2008-tcl_dynamic_libraries >>>> Repository UUID: d7ba3245-3e7b-42d0-9cd4-356c8b94b330 >>>> [...] >>>> >>>> >>>>> I have modified some functions to make this RTLD compilable as an >>>>> application on Linux. Now I just want the RTLD to let me load >>>>> libraries from regular files. The problem is that it is not always >>>>> working. I mean it works for some simple libraries, but it does not >>>>> work for other which are also very simple. For example if my function >>>>> I call by 'your' dlsym calls printf or some other functions from >>>>> stdio.h its execution causes segmentation faults. >>>> >>>> >>>> Some relevant mails already exchanged regarding the problem are in the >>>> TclCore archives. >>>> >>>> It is known that the memory-RTLD (*) currently loads a second copy of >>>> libc.so. It is not yet understood where and how this leads to the crash. >>>> >>>> Now we are hoping that we can gain your assistance and pick your brain >>>> regarding RTLD's internals. >>>> >>>> >>>>> For the last few days I have tried to find what is wrong, but I have >>>>> not came up with any solution. I think there may be some problems with >>>>> initializations. In your code there is a function '_rtld' which I do >>>>> not call. The only initialization work I do is to initialize obj_main >>>>> with some random data. >>>> >>>>> I would be extremely grateful if you could just take a look at my code >>>>> and tell me what may be not ok. This is mainly your code. I just >>>>> commented out some functions and added some log messages. Maybe I >>>>> commented something which is crucial or I missed some >>>>> initializations... To make the RTLD work is very important, >>>>> because now I am >>>>> stacked and cannot go on with the project, but I would really want to >>>>> complete it. >>>>> I would appreciate any kind of advice from you. >>>>> >>>>> I am looking forward to hearing from you. >>>>> >>>>> Regards, >>>>> Daniel >>>>> >>>>> Note1. I also added defs.h file which contains defines which you use >>>>> in your code. They are architecture depended and work on 32-bit >>>>> Intels. >>>>> Note2. If managed to compile this code by yourself you could test it >>>>> ./rtld /full_path_to_library_you_want_to_load >>>> >>>> -- >>>> Andreas Kupries <andreask@ActiveState.com> >>>> Developer @ http://www.ActiveState.com >>>> Tel: +1 778-786-1122 >>>> >>> >> > |