From: daniel b. <da...@te...> - 2000-08-17 17:32:43
|
Humour a young man's curiosity. What are you using SBCL for? What features do you want to see? What features are you working on? What uses do you have in mind for it? To start the ball rolling, here's my list, in approximate order of likelihood that I actually do anything about it. 1) an alpha port (because I have one). It seems that it shouldn't be too hard to take the cmucl alpha backend and fit it back into sbcl. So far I have "successfully" built the from-host crosscompiler, but it won't compile anything. I've probably done something stupid. 0: (UNIX::SIGILL-HANDLER #<unavailable-arg> #<unavailable-arg> #.(SYSTEM:INT-SAP #x11FFFF8E0)) 1: ("Foreign function call land") 2: (SB!VM::IMPL-OF-VM-SUPPORT-ROUTINE-MAKE-RETURN-PC-PASSING-LOCATION T) If anyone is working on (or thinking of working on) porting other backends - and I think I've seen mail from someone hacking Sparc already - I'd be happy to compare notes. Utility of said notes is somewhat questionable, of course, as I haven't finished this yet. But, still. Actually, I'll put them on CLiki somewhere and let people hack them around as seems useful. For what it's worth, I'm cross-compiling from Julian Dolby's CMUCL binary of whatever-was-in-CVS in Feb 1999 and wishing it had the -safe core as in the Debian version. 2) unix convenience stuff; sockets, corba, xml, regexps etc etc. You can see the motive for CLiki here ;-) 3) port of MaiSQL or UncommonSQL, whichever looks more useful. 4) delivery issues, packaging stuff. SAVE-LISP{,-AND-DIE} is great, and all, but I get strange looks when I tell people the executables will each be 20Mb in size and won't share code pages between them. I'd like a single core file with the fundamental things in it, something akin to `shared libraries' for CLX, defsystem, CLIO, Garnet, etc, and an application that looks like a "normal" unix program that gets invoked from the shell. For a while I thought that x86f files would do nicely for the subsystems, but having just read the internals docs I find that there is approximately no opportunity for shared pages between multiple images that want to load the same fasl file. Oh well. 5) re: the build process, if we have an ELF system, why do we need lisp.nm? We can just link the 'sbcl' excecutable with -rdynamic, then dlsym(dlopen(NULL,0),"foo") should get us the address of "foo". In fact, the code for this already exists in alternate-get-global-address (code/foreign.lisp) so I wonder why we need both. 6) Berlin support (see http://www.berlinconsortium.org/) - not that I can think of an earthly use for it, but it looks like extreme fun. Are there any plans to check SBCL into the sourceforge CVS tree? (Or anyone else's CVS tree) "cvs diff" could potentially make it a lot easier to track new versions as they come out. -dan |
From: Raymond W. <Ray...@fa...> - 2000-08-21 09:33:43
|
daniel barlow writes: > > Humour a young man's curiosity. What are you using SBCL for? What > features do you want to see? What features are you working on? What > uses do you have in mind for it? I'm not actually *using* SBCL (yet). I'd very much like to use Common Lisp (and SBCL) for various things (both work-related and not). I have made a number of attempts at compiling CMUCL in the past, but I have never been able to get it to work. This is obviously much simpler with SBCL :-) Things I'd like to see in SBCL: 1) Reduce the interdependency between the runtime (sbc) and and the Lisp code. I would like to have a single function in the runtime, callable at load time, used for resolving all other foreign functions. This function would be called through a pointer at a fixed position (i.e, similarly to the way that NIL is given a fixed value.) With this setup, it would be possible to recompile the runtime with different compiler/linker options or preprocessor flags without having to rebuild the Lisp image. 2) Remove the Unix dependencies in the Lisp image, and replace them with a set of well-defined interfaces for doing various low-level things required by SBCL itself (i.e, before anny application code is loaded). This would facilitate porting to other platforms (MS-Win being one obvious candidate, but the Flux OS Toolkit is perhaps more interesting :-) 3) A mechanism for automatically generating foreign interfaces from C header files. I have been thinking about this for a while, and it should be doable. The main problem that I see is related to preprocessor functionality (which does not really fit into the normal lexer/parser pattern), but this can be overcome by simply using an external CPP. > 1) an alpha port (because I have one). It seems that it shouldn't be > too hard to take the cmucl alpha backend and fit it back into sbcl. > So far I have "successfully" built the from-host crosscompiler, but it > won't compile anything. I've probably done something stupid. A port to the Alpha would be way cool :-) > 2) unix convenience stuff; sockets, corba, xml, regexps etc etc. You > can see the motive for CLiki here ;-) > > 3) port of MaiSQL or UncommonSQL, whichever looks more useful. > > 4) delivery issues, packaging stuff. SAVE-LISP{,-AND-DIE} is great, > and all, but I get strange looks when I tell people the executables > will each be 20Mb in size and won't share code pages between them. > I'd like a single core file with the fundamental things in it, > something akin to `shared libraries' for CLX, defsystem, CLIO, Garnet, > etc, and an application that looks like a "normal" unix program that > gets invoked from the shell. For a while I thought that x86f files > would do nicely for the subsystems, but having just read the internals > docs I find that there is approximately no opportunity for shared > pages between multiple images that want to load the same fasl file. > Oh well. I think the 20MB core file is actually quite acceptable, as long as these 20MB can be reused between applications (which I think is the case). > 5) re: the build process, if we have an ELF system, why do we need > lisp.nm? We can just link the 'sbcl' excecutable with -rdynamic, then > dlsym(dlopen(NULL,0),"foo") should get us the address of "foo". > In fact, the code for this already exists in alternate-get-global-address > (code/foreign.lisp) so I wonder why we need both. I've been looking at this for the FreeBSD port. For FreeBSD, an additional complication is that you cannot use dlopen() & friends from a statically linked executable, and if it's dynamically linked, you (obviously) don't get the address of dlopen() in sbcl.nm. I did some experimenting with wrapping dlopen and dlsym, and moving the compile/load of foreign.lisp from src/cold/warm.lisp to stems-and-flags.lisp-expr, but this failed (I expect that I did something wrong in stems-and-flags.lisp-expr, like placing foreign.lisp before some of the code it uses, or using the wrong flags (I tried :not-host)). I've given up on this for the moment, and resorted to the ugly hack of wrapping each of the 50-odd foreign functions required from libc/libm in code looking like double _ldso_stub__cos(double x) { return cos(x); } > 6) Berlin support (see http://www.berlinconsortium.org/) - not that I > can think of an earthly use for it, but it looks like extreme fun. If it looks like fun, that should be reason enough :-) //Raymond. -- Raymond Wiker Ray...@fa... |
From: William H. N. <wil...@ai...> - 2000-08-19 03:07:05
|
On Thu, Aug 17, 2000 at 06:34:11PM +0100, daniel barlow wrote: > Humour a young man's curiosity. What are you using SBCL for? What > features do you want to see? What features are you working on? What > uses do you have in mind for it? I personally use SBCL to work on a computer program to play the game of Go. More generally, I see Common Lisp as particularly useful for prototyping hairy algorithms. I'd like to make SBCL better for implementing servers for the vaguely AI-flavored stuff that I see as Lisp's bread and butter problems. (E.g. pattern recognition, learning, advanced database stuff, and planning.) I wish that Common Lisp had been used as the basis for Perl (i.e. instead of Perl, start with CLISP and add UNIX primitives) so that UNIX glue applications could be added to the list of Lisp bread and butter problems, but I think it's too late to displace Perl. My main priorities right now for SBCL are basic standard conformance, fixing the abundance of stupid bugs, improving the programming environment, and fixing horrible performance problems (e.g. the recent improvements in the performance of MAP and non-SIMPLE vectors). If I had all the time in the world I'd be interested in multithreading and a clean extension to 64 bits, since they're both important, but they're both hard, and I've got much too much to do already, so I'm not likely to work on those in the foreseeable future. I'd also very much like to see better facilities for communicating between SBCL processes and the rest of the world, like the sockets and CORBA stuff you mention below, and that doesn't look too hard to do. However, right now I don't really need this for the application I'm most concerned with, so I'm not highly motivated to work on this. > To start the ball rolling, here's my list, in approximate order of > likelihood that I actually do anything about it. > > 1) an alpha port (because I have one). It seems that it shouldn't be > too hard to take the cmucl alpha backend and fit it back into sbcl. > So far I have "successfully" built the from-host crosscompiler, but it > won't compile anything. I've probably done something stupid. I can't help with this problem much -- I have no non-x86 machines and even if someone gave me one, I probably wouldn't pay much attention to this problem. At this point, my problem is making programs work, not making them run faster. (But I'll reevaluate my priorities when cheap 32+-processor machines become available.:-) > 2) unix convenience stuff; sockets, corba, xml, regexps etc etc. You > can see the motive for CLiki here ;-) I think the talking-to-the-rest-of-the-world stuff is a separate and particularly important category here. I'd love to see better CORBA communication for SBCL, and perhaps sockets too, but I think XML and regexps are better done as portable Common Lisp libraries. (On the other hand, I might use PCRE bindings as an example application for the FFI, just because it's so easy.) A good solution for GUI stuff would also be really really really nice. (Maybe that would come for free with CORBA?) > 3) port of MaiSQL or UncommonSQL, whichever looks more useful. I don't know enough about database stuff to have much of an opinion here. > 4) delivery issues, packaging stuff. SAVE-LISP{,-AND-DIE} is great, > and all, but I get strange looks when I tell people the executables > will each be 20Mb in size and won't share code pages between them. > I'd like a single core file with the fundamental things in it, > something akin to `shared libraries' for CLX, defsystem, CLIO, Garnet, > etc, and an application that looks like a "normal" unix program that > gets invoked from the shell. For a while I thought that x86f files > would do nicely for the subsystems, but having just read the internals > docs I find that there is approximately no opportunity for shared > pages between multiple images that want to load the same fasl file. > Oh well. At the current 20Mb level, the size of the system qualifies as a fairly horrible performance problem, and I'd like to fix it. If the byte compiler could be used with the cold boot system, which is one delivery improvement that I keep thinking about, it might be more like 10Mb in size, which is still pretty big, but at least better. It also seems to me that it should be possible to share memory pages for most of the system, since most of the system is code that's frozen into place by PURIFY before the core is dumped. If we could arrange for purified code to go into pages which were 100% read-only, those pages could be mmap()'ed read-only by all the processes which use SBCL, which might be enough to let the OS know it can share the memory. But I haven't looked at this at all, so I don't even know how much of this is happening now, much less how hard it would be make more of this happen. > 5) re: the build process, if we have an ELF system, why do we need > lisp.nm? We can just link the 'sbcl' excecutable with -rdynamic, then > dlsym(dlopen(NULL,0),"foo") should get us the address of "foo". > In fact, the code for this already exists in alternate-get-global-address > (code/foreign.lisp) so I wonder why we need both. Where is gcc -rdynamic documented? Both you and Peter Van Eynde evidently know about it (since he submitted a patch which uses it) but I don't know what it does. > Are there any plans to check SBCL into the sourceforge CVS tree? (Or > anyone else's CVS tree) "cvs diff" could potentially make it a lot > easier to track new versions as they come out. Peter Van Eynde has pretty much convinced me to make the CVS tree publicly accessible, but I don't know when I will get around to it. (Probably within a few months, maybe within a few weeks.) -- William Harold Newman <wil...@ai...> software consultant PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C |
From: Daniel B. <da...@te...> - 2000-08-19 14:19:58
|
William Harold Newman <wil...@ai...> writes: > Where is gcc -rdynamic documented? Both you and Peter Van Eynde > evidently know about it (since he submitted a patch which uses it) but > I don't know what it does. The dlsym(3) manual page mentions it in passing. Other than that, you'd have to look in the gcc 'specs' file (run gcc -v), which says among other things *link: -m elf_i386 %{shared:-shared} %{!shared: %{!ibcs: %{!static: %{rdynamic:-export-dynamic} %{!dynamic-linker:-dynamic-linker /lib/ld-linux.so.2}} %{static:-static}}} meaning "if -rdynamic was supplied and the -static flag was not, run the linker with the -export-dynamic flag". -export-dynamic is documented in the ld info page. When creating a dynamically linked executable, add all symbols to the dynamic symbol table. The dynamic symbol table is the set of symbols which are visible from dynamic objects at run time. > Peter Van Eynde has pretty much convinced me to make the CVS tree > publicly accessible, but I don't know when I will get around to it. > (Probably within a few months, maybe within a few weeks.) Cool! -dan -- http://ww.telent.net/cliki/ - CLiki: CL/Unix free software link farm |
From: Peter V. E. <pva...@de...> - 2000-08-23 18:30:41
|
Hi people, I've commented on a few things. As to my plans, I just want a lisp for students in the 21'th century :-). Stable and with a lot of libraries to easily integrate with the rest of the world. Simple, not? > 4) delivery issues, packaging stuff. SAVE-LISP{,-AND-DIE} is great, > and all, but I get strange looks when I tell people the executables > will each be 20Mb in size and won't share code pages between them. Sharing pages can only be done if they are in the oldest 'frozen' generation that is mmaped R/O. This is not possible with the current VM (AFAIK). Having then COW could only provide sharing until the first major GC. Having a 'lisp application server' that uses multithreading to execute users job's would be nicer. From a VM POV, not from a security POV of course... On Fri, Aug 18, 2000 at 10:03:11PM -0500, William Harold Newman wrote: > I'd also very much like to see better facilities for communicating > between SBCL processes and the rest of the world, like the sockets and > CORBA stuff you mention below, and that doesn't look too hard to do. > However, right now I don't really need this for the application I'm > most concerned with, so I'm not highly motivated to work on this. Having a simple-to-user webserver would be nice. Clim presentation based would be divine :-). > At the current 20Mb level, the size of the system qualifies as a > fairly horrible performance problem, and I'd like to fix it. If the > byte compiler could be used with the cold boot system, which is one > delivery improvement that I keep thinking about, it might be more like > 10Mb in size, which is still pretty big, but at least better. Note that the size of the files are almost irrelevant, the problem is the RSS... > 1) Reduce the interdependency between the runtime (sbc) and > and the Lisp code. I would like to have a single function in the > runtime, callable at load time, used for resolving all other foreign > functions. This function would be called through a pointer at a fixed > position (i.e, similarly to the way that NIL is given a fixed value.) IMHO it would be easier and simpler to (on Linux) call the kernel directly. The kernel interface is easier, simpler and above all more constant. Glibc has a tendency to break CMUCL now and again until I started using the kernel directly. On unixen with a less broken libc it could be used of course :-). Calling from lisp to the libc always has to happen via thunks (like the stubs). Having a fixed location for the address of the init function seems nice, but beware that the c core also does the signal and gc stuff. This is another connection to consider. > 2) Remove the Unix dependencies in the Lisp image, and replace > them with a set of well-defined interfaces for doing various low-level > things required by SBCL itself (i.e, before anny application code is Like a kernel interface :-) ? > loaded). This would facilitate porting to other platforms (MS-Win being Windows is very difficult, it doesn't seem to expose all the needed functionality (esp in signal handlers). > 3) A mechanism for automatically generating foreign interfaces > from C header files. I have been thinking about this for a while, and > it should be doable. The main problem that I see is related to > preprocessor functionality (which does not really fit into the normal > lexer/parser pattern), but this can be overcome by simply using an > external CPP. What you do about all the constants being #define'ed? > I've been looking at this for the FreeBSD port. For FreeBSD, > an additional complication is that you cannot use dlopen() & friends > from a statically linked executable, and if it's dynamically linked, Likewise on Linux. > you (obviously) don't get the address of dlopen() in sbcl.nm. I did Using stub functions _are_ the only way. You don't know the location of dlopen. It is filled in by ld.so when 'fixing-up' the executable. But it only does this in the c-core. It doesn't know about the lisp-core. So lisp doesn't know the address. The only other way is that the c-core tells the lisp core where dlopen is... This looks even dirtier and prone to errors. Groetjes, Peter -- It's logic Jim, but not as we know it. | pva...@de... for pleasure, "God, root, what is difference?" - Pitr| "God is more forgiving." - Dave Aronson| http://cvs2.cons.org/~pvaneynd/ |
From: Raymond W. <Ray...@fa...> - 2000-08-24 06:57:37
|
Peter Van Eynde writes: > Hi people, > > I've commented on a few things. > > As to my plans, I just want a lisp for students in the 21'th century > :-). Stable and with a lot of libraries to easily integrate with the > rest of the world. Simple, not? > > > 4) delivery issues, packaging stuff. SAVE-LISP{,-AND-DIE} is great, > > and all, but I get strange looks when I tell people the executables > > will each be 20Mb in size and won't share code pages between them. > > Sharing pages can only be done if they are in the oldest 'frozen' > generation that is mmaped R/O. This is not possible with the current VM > (AFAIK). Having then COW could only provide sharing until the first > major GC. I thought that the data in sbcl.core effectively became read-only. If I'm mistaken (as it seems I am), the 20 MB becomes more of an issue. > Having a simple-to-user webserver would be nice. Clim presentation based > would be divine :-). I suspect that Daniel Barlow's "Araneida" may be a candidate for the first. With respect to CLIM, there has been some activity on the free-clim mailing lists lately - some people have been working on a free CLIM over the summer. Most of this work has been done under CMUCL, so it should be possible to get this to work under SBCL, too. Somebody will have to get CLX working first, though (I think!) > > 1) Reduce the interdependency between the runtime (sbc) and > > and the Lisp code. I would like to have a single function in the > > runtime, callable at load time, used for resolving all other foreign > > functions. This function would be called through a pointer at a fixed > > position (i.e, similarly to the way that NIL is given a fixed value.) > > IMHO it would be easier and simpler to (on Linux) call the kernel > directly. The kernel interface is easier, simpler and above all more > constant. Glibc has a tendency to break CMUCL now and again until I > started using the kernel directly. > > On unixen with a less broken libc it could be used of course :-). I *think* it would (still) be better to have a well-defined set of services supplied by the runtime, and let the runtime sort out most of the platform dependencies. Among other things, this would eman no calls to ioctl() from (system) lisp code :-) > Calling from lisp to the libc always has to happen via thunks (like > the stubs). > > Having a fixed location for the address of the init function seems nice, > but beware that the c core also does the signal and gc stuff. This is > another connection to consider. Yup. > > 2) Remove the Unix dependencies in the Lisp image, and replace > > them with a set of well-defined interfaces for doing various low-level > > things required by SBCL itself (i.e, before anny application code is > > Like a kernel interface :-) ? Too system-dependent, IMHO... > > 3) A mechanism for automatically generating foreign interfaces > > from C header files. I have been thinking about this for a while, and > > it should be doable. The main problem that I see is related to > > preprocessor functionality (which does not really fit into the normal > > lexer/parser pattern), but this can be overcome by simply using an > > external CPP. > > What you do about all the constants being #define'ed? There's a way to make cpp emit all #defines at the end of the preprocessed output. Earlier, I think I said that this was done by specifying the "-M" switch, but that's wrong. It should be either "-dM" or "dD" - the former outputs just the #define directives, while the second emits #define directives along with preprocessed output. > > I've been looking at this for the FreeBSD port. For FreeBSD, > > an additional complication is that you cannot use dlopen() & friends > > from a statically linked executable, and if it's dynamically linked, > > Likewise on Linux. > > > you (obviously) don't get the address of dlopen() in sbcl.nm. I did > > Using stub functions _are_ the only way. But having to stub 45 functions (in addition to dlopen()/dlsym()) is clearly excessive. That's what I had to do for the FreeBSD port, but I *may* be able to avoid it if I can arrange so that alternate-get-global-address is called at load time for unknown external symbols. > You don't know the location of dlopen. It is filled in by ld.so when > 'fixing-up' the executable. But it only does this in the c-core. It > doesn't know about the lisp-core. So lisp doesn't know the address. The > only other way is that the c-core tells the lisp core where dlopen > is... NIL is already handled in a similar way - it is placed at a fixed place in the lisp core, known by both the runtime and the lisp core. A suitable wrapper for dlopen could be handled similarly, I think. > This looks even dirtier and prone to errors. Possibly :-) Anyway, it was just a suggestion. I expect that if there's any merit to the idea, it will be picked up. //Raymond. |
From: Peter V. E. <pva...@de...> - 2000-08-27 17:22:10
|
On Thu, Aug 24, 2000 at 08:59:00AM +0200, Raymond Wiker wrote: > > Sharing pages can only be done if they are in the oldest 'frozen' > > generation that is mmaped R/O. This is not possible with the current VM > > (AFAIK). Having then COW could only provide sharing until the first > > major GC. > > I thought that the data in sbcl.core effectively became > read-only. If I'm mistaken (as it seems I am), the 20 MB becomes more > of an issue. To demonstrate: these are the memory mappings of cmucl-small: 08048000-0805f000 r-xp 00000000 03:03 245159 /home/pvaneynd/fakeroot/work/cmucl-2.4.21/bin/lisp 0805f000-08061000 rw-p 00016000 03:03 245159 /home/pvaneynd/fakeroot/work/cmucl-2.4.21/bin/lisp 08061000-085ab000 rwxp 00000000 00:00 0 10000000-109f7000 rwxp 00001000 03:03 467574 /home/pvaneynd/fakeroot/work/cmucl-2.4.21/bin/lisp-small.core 109f7000-1ffff000 rwxp 009f7000 00:00 0 20000000-27fff000 rwxp 00000000 00:00 0 28000000-28239000 rwxp 009f8000 03:03 467574 /home/pvaneynd/fakeroot/work/cmucl-2.4.21/bin/lisp-small.core 28239000-37fff000 rwxp 00239000 00:00 0 38000000-3ffff000 rwxp 00000000 00:00 0 40000000-40012000 r-xp 00000000 03:03 160646 /lib/ld-2.1.3.so 40012000-40013000 rw-p 00011000 03:03 160646 /lib/ld-2.1.3.so 40013000-4001a000 rwxp 00000000 00:00 0 4001a000-4001c000 r-xp 00000000 03:03 160652 /lib/libdl-2.1.3.so 4001c000-4001e000 rw-p 00001000 03:03 160652 /lib/libdl-2.1.3.so 4001e000-4003a000 r-xp 00000000 03:03 160653 /lib/libm-2.1.3.so 4003a000-4003b000 rw-p 0001b000 03:03 160653 /lib/libm-2.1.3.so 4003b000-40110000 r-xp 00000000 03:03 160648 /lib/libc-2.1.3.so 40110000-40114000 rw-p 000d4000 03:03 160648 /lib/libc-2.1.3.so 40114000-40118000 rw-p 00000000 00:00 0 40118000-40119000 rwxp 00000000 00:00 0 48000000-48001000 rwxp 00c31000 03:03 467574 /home/pvaneynd/fakeroot/work/cmucl-2.4.21/bin/lisp-small.core 48001000-b8000000 rwxp 00001000 00:00 0 bfffd000-c0000000 rwxp ffffe000 00:00 0 Note that the .core file with rwxp is mapped. AFAIK this means COW... For sbcl it looks simmilar: 01000000-02278000 rwxp 00001000 03:03 259272 /usr/local/lib/sbcl.core 02278000-03800000 rwxp 01278000 00:00 0 05000000-05475000 rwxp 01279000 03:03 259272 /usr/local/lib/sbcl.core 05475000-07fff000 rwxp 00475000 00:00 0 08048000-0805e000 r-xp 00000000 03:03 306866 /usr/local/bin/sbcl 0805e000-0805f000 rw-p 00015000 03:03 306866 /usr/local/bin/sbcl 0805f000-08268000 rwxp 00000000 00:00 0 09000000-09001000 rwxp 016ee000 03:03 259272 /usr/local/lib/sbcl.core 09001000-29000000 rwxp 00001000 00:00 0 40000000-40012000 r-xp 00000000 03:03 160646 /lib/ld-2.1.3.so 40012000-40013000 rw-p 00011000 03:03 160646 /lib/ld-2.1.3.so 40013000-40014000 rwxp 00000000 00:00 0 40014000-40015000 rw-p 00000000 00:00 0 40015000-4001a000 rwxp 00000000 00:00 0 4001a000-4001c000 r-xp 00000000 03:03 160652 /lib/libdl-2.1.3.so 4001c000-4001e000 rw-p 00001000 03:03 160652 /lib/libdl-2.1.3.so 4001e000-4003a000 r-xp 00000000 03:03 160653 /lib/libm-2.1.3.so 4003a000-4003b000 rw-p 0001b000 03:03 160653 /lib/libm-2.1.3.so 4003b000-40110000 r-xp 00000000 03:03 160648 /lib/libc-2.1.3.so 40110000-40114000 rw-p 000d4000 03:03 160648 /lib/libc-2.1.3.so 40114000-40118000 rw-p 00000000 00:00 0 50000000-57fff000 rwxp 00000000 00:00 0 60000000-67fff000 rwxp 00000000 00:00 0 bfffd000-c0000000 rwxp ffffe000 00:00 0 Again rwxp... > > Having a simple-to-user webserver would be nice. Clim presentation based > > would be divine :-). > > I suspect that Daniel Barlow's "Araneida" may be a candidate > for the first. With respect to CLIM, there has been some activity I meant: use the presentation system to 'print' stuff to a 'stream' that might go to a web-page or a window. This should be possible, IIRC there is some support in cl-http for this. > > On unixen with a less broken libc it could be used of course :-). > > I *think* it would (still) be better to have a well-defined > set of services supplied by the runtime, and let the runtime sort out > most of the platform dependencies. Among other things, this would eman > no calls to ioctl() from (system) lisp code :-) <stunned> And what you do with constants? Translate them on-the-fly? > > > 2) Remove the Unix dependencies in the Lisp image, and replace > > > them with a set of well-defined interfaces for doing various low-level > > > things required by SBCL itself (i.e, before anny application code is > > > > Like a kernel interface :-) ? > > Too system-dependent, IMHO... You can still see the simple v7 interface in most unixen. We could use this. > > What you do about all the constants being #define'ed? > > There's a way to make cpp emit all #defines at the end of the > preprocessed output. Earlier, I think I said that this was done by > specifying the "-M" switch, but that's wrong. It should be either > "-dM" or "dD" - the former outputs just the #define directives, while > the second emits #define directives along with preprocessed output. Ok This looks usefull... Now, what constants are you going to define to get your data? All those ifdef's looked too complex for me. > > Using stub functions _are_ the only way. > > But having to stub 45 functions (in addition to > dlopen()/dlsym()) is clearly excessive. That's what I had to do for The unix interface uses too many functions. We should have to do with less. This is just what I mean: the kernel interfaces (a part of them, no need to support things like mount) are pretty minimal, actually quite portable (except calling interface), very stable and actually easy to program for (very easy to have thread-safety for instance). > NIL is already handled in a similar way - it is placed at a > fixed place in the lisp core, known by both the runtime and the lisp > core. A suitable wrapper for dlopen could be handled similarly, I > think. I agree. Note that the address can change from load to load... This stuff is pretty irritatingly complex. Groetjes, Peter -- It's logic Jim, but not as we know it. | pva...@de... for pleasure, "God, root, what is difference?" - Pitr| "God is more forgiving." - Dave Aronson| http://cvs2.cons.org/~pvaneynd/ |
From: Daniel B. <da...@te...> - 2000-08-28 04:39:27
|
Peter Van Eynde <pva...@de...> writes: > Note that the .core file with rwxp is mapped. AFAIK this means COW... Fair enough, but _is_ it written to? I believed that as the system was purified before it was dumped, basically everything should be in non-collected memory. On that basis, I can't see how it would be getting mutated. My issue with the size of the core file is that none of this sharing will happen _anyway_ if you deliver each application as its own core image. If they're all different files to start with, the OS is not going to be able to share pages between the apps. I think that the answer is not to deliver as a core file, but instead to ship the standard core file as a "shared library" and label some other file "the application". Ideas? - traditional way: a fasl file and a shell script to invoke it - a fasl file. Users type "sbcl -load myapp.x86f" - a runtime with attached blob of .fasl file that knows how to read fasl out of its own image. Users type "myapp" - a #!/usr/local/bin/sbcl script that can load fasl out of its own body (I've implemented this before, it should be in the archive for this list) (You can see my inclination is to put it all in a single file if possible. I don't really like shell wrappers, not least because nobody ever gets the quoting right) "Persistent lisp server" is another way to ensure code sharing between apps, but my suspicion is that it makes it far too easy to stomp on someone else's application for it to be a generally good idea -dan -- http://ww.telent.net/cliki/ - CLiki: CL/Unix free software link farm |
From: Peter V. E. <pva...@de...> - 2000-08-29 22:07:31
|
> Fair enough, but _is_ it written to? I believed that as the system Try it. Just change the lisp to map it read-only and see what breaks. That's how I did the lazy-allocation stuff ;-) > was purified before it was dumped, basically everything should be in > non-collected memory. On that basis, I can't see how it would be > getting mutated. When purify-ing again? > I think that the answer is not to deliver as a core file, but instead > to ship the standard core file as a "shared library" and label some > other file "the application". Ideas? cmucl debian package: /usr/doc/cmucl/examples/Demos/register-lisp-as-executables.sh /usr/doc/cmucl/examples/Demos/lisp-start Use the BINFMT_MISC stuff in the kernel... > "Persistent lisp server" is another way to ensure code sharing between > apps, but my suspicion is that it makes it far too easy to stomp on > someone else's application for it to be a generally good idea IMHO Lisp supports the "save because the compiler makes it safe" model of Bouroughs mainframes: trust the compiler to never emit 'bad' code... See comp.folklore.computers recently. If I understand the arguments correctly, they're doing something similar for Java... Groetjes, Peter -- It's logic Jim, but not as we know it. | pva...@de... for pleasure, "God, root, what is difference?" - Pitr| "God is more forgiving." - Dave Aronson| http://cvs2.cons.org/~pvaneynd/ |
From: Daniel B. <da...@te...> - 2000-08-29 23:07:01
|
Peter Van Eynde <pva...@de...> writes: > Try it. Just change the lisp to map it read-only and see what breaks. Heh. William Newan made the point that I would probably have made if I were less dopey. Of course there is writable stuff in there ... > cmucl debian package: > /usr/doc/cmucl/examples/Demos/register-lisp-as-executables.sh > /usr/doc/cmucl/examples/Demos/lisp-start > > Use the BINFMT_MISC stuff in the kernel... Good point. I'd forgotten about that approach Are there analogues for non-Linux platforms? > IMHO Lisp supports the "save because the compiler makes it safe" model > of Bouroughs mainframes: trust the compiler to never emit 'bad' code... > See comp.folklore.computers recently. It _should_, yes. But setting it up properly sounds like a major task, and given that security has never really been a goal for the cmucl project (see, for example, /tmp file race in run-program) I don't think I'd trust it against a determined attacker. Or even a novice hacker in its current state. The damage you can do just by TRACEing stuff is pretty extreme ... -dan -- http://ww.telent.net/cliki/ - CLiki: CL/Unix free software link farm |
From: Peter V. E. <pva...@de...> - 2000-08-31 19:13:36
|
On Wed, Aug 30, 2000 at 12:05:44AM +0100, Daniel Barlow wrote: > > Use the BINFMT_MISC stuff in the kernel... > > Good point. I'd forgotten about that approach > > Are there analogues for non-Linux platforms? No idea. Possibly Solaris has something like this... It was put in to support Java stuff better IIRC. Groetjes, Peter -- It's logic Jim, but not as we know it. | pva...@de... for pleasure, "God, root, what is difference?" - Pitr| "God is more forgiving." - Dave Aronson| http://cvs2.cons.org/~pvaneynd/ |
From: Raymond W. <Ray...@fa...> - 2000-08-23 06:59:42
|
[ Note: the message I'm replying to was sent directly to me; did you intend to send it to sbc...@li...? ] Daniel Barlow writes: > Raymond Wiker <Ray...@fa...> writes: > > 1) Reduce the interdependency between the runtime (sbc) and > > and the Lisp code. I would like to have a single function in the > > runtime, callable at load time, used for resolving all other foreign > > functions. This function would be called through a pointer at a fixed > > position (i.e, similarly to the way that NIL is given a fixed value.) > > That function to be called "dlsym"? :-) That's certainly an option, but I would prefer a function that took just a single parameter (the symbol to be resolved) and called dlopen()/dlsym() as required, preserving the dl "handle" between calls. > > 2) Remove the Unix dependencies in the Lisp image, and replace > > them with a set of well-defined interfaces for doing various low-level > > things required by SBCL itself (i.e, before anny application code is > > loaded). This would facilitate porting to other platforms (MS-Win being > > one obvious candidate, but the Flux OS Toolkit is perhaps more > > interesting :-) > > Oh look, another LispOS project. It must be the season for them ... > My road map is to start at the application end and work downwards: I > think there's a lot of mileage in this approach, not least because I > can develop and test on the same box :-) I agree with that approach. I'm not necessarily interested in a LispOS, but I think it would be neat to be able to run SBCL on top of Flux/OS. > Yes, it would be nice to clean up the OS interface. Then we can design > a sensible Unix interface without treading on the exsting one ... I think that SBCL would become more robust / easier to maintain if we formalised the interface between the C runtime and the Lisp code. Line 1260 of src/code/fd-stream.c is a good example of why it would be a good idea to change the runtime interface :-) > > 3) A mechanism for automatically generating foreign interfaces > > from C header files. I have been thinking about this for a while, and > > it should be doable. The main problem that I see is related to > > preprocessor functionality (which does not really fit into the normal > > lexer/parser pattern), but this can be overcome by simply using an > > external CPP. > > You also need to get the right preprocessor flags to pick up the > correct versions of everything. I don't have a FreeBSD box anywhere > so I don't know what it's like, but GNU-based systems are a rat's nest > of dependencies. "echo | gcc -v -E -" and you'll see what I mean. I think the main requirement is that you have something similar to the -M option to gcc (which prints out the #defines, but you know that already :-) > The way I did it for db-sockets was to have a "spec" file containing > the constants/struct definitions that I wanted, and turn it into a > C program which when compiled and run printed out the appropriate > vaues in lisp-readable form. That way I not only get cpp to expand my > preprocessor definitions, but I get the C compiler to work out the > offsets in my structure definitions, so I don't have to edit CL code > when the third-party library I'm using adds a field to a structure - I > just recompile. I was thinking of making calls to the foreign-interface functions, similar to the way the FFI is used from hand-written code. Something like: (generate-interface :files ("/usr/include/sys/types.h" "/usr/include/stat.h") :dest-package "MY-INTERFACE" :import ("stat" "lstat" "fstat" "struct stat")) => (def-alien-type nil (struct stat (st-dev dev-t) (st-ino ino-t) [ ... ] )) Note: unix.lisp implies that stat() & friends may need special handling. > http://ww.telent.net/lisp/sockets.html > http://loaclhost.telent.net/cgi-bin/cvsweb/telent/sockets/Makefile?rev=1.11 > (look at the rule for constants.lisp) I've looked at this already - in fact, I was playing with your "sockets" package a couple of days ago, using SBCL on FreeBSD. Araneida next, I think, unless I start playing with CLX and then free-clim. > If I were doing it again, I'd probably investigate either making it > part of SWIG, or at least making it read SWIG interface files > > http://www.swig.org/ > > > > So far I have "successfully" built the from-host crosscompiler, but it > > > won't compile anything. I've probably done something stupid. > > [ status update: I had indeed done something stupid. All manner of > stupid things, in fact. But work proceeds apace and it's now > generating a plausibly-sized cold-sbcl.core that blows up in > cold-init ] I did something remarkably similar on the FreeBSD "port" of SBCL (i.e, various stupid things, and getting a blowup in cold-init. :-) > > I think the 20MB core file is actually quite acceptable, as > > long as these 20MB can be reused between applications (which I think > > is the case). > > It is, and can be. But that means delivering applications as .x86f > files, rather than executables. This is more a packaging problem > than a technical one, but I do like nice packaging. > > > double _ldso_stub__cos(double x) > > { > > return cos(x); > > } > > That sounds like exactly the same thing as is done for the > Linux/x86 port (the Alpha one's still built static at the moment). > Yesterday I finally understood why it's currently necessary, but I > agree it's ugly. I'm going to make another attempt at getting around this for FreeBSD/i386, using alternate-get-global-address. This means (I think) that foreign.lisp has to be present in the system at load time, which means that it has to be moved from warm.lisp into stems-and-flags.lisp-expr. My previous attempt at getting this to work got me a crash somewhere in macro-expand land, but I think I know why (I have to replace "SB-" with "SB!" in foreign.lisp.) //Raymond. -- Raymond Wiker Ray...@fa... |
From: Peter V. E. <pva...@de...> - 2000-08-24 18:38:39
|
On Wed, Aug 23, 2000 at 09:01:05AM +0200, Raymond Wiker wrote: > > a sensible Unix interface without treading on the exsting one ... > > I think that SBCL would become more robust / easier to > maintain if we formalised the interface between the C runtime and the > Lisp code. Line 1260 of src/code/fd-stream.c is a good example of why > it would be a good idea to change the runtime interface :-) I think we should have Lisp <-> Basic OS interface (Lisp) <-> Os interface (OS specific, Lisp and/or C) <-> libc or kernel. The BOI should look a lot like the release 7 kernel interface I would imagine :-). > > You also need to get the right preprocessor flags to pick up the > > correct versions of everything. I don't have a FreeBSD box anywhere > > so I don't know what it's like, but GNU-based systems are a rat's nest > > of dependencies. "echo | gcc -v -E -" and you'll see what I mean. > > I think the main requirement is that you have something > similar to the -M option to gcc (which prints out the #defines, but > you know that already :-) Huh? What gcc is that? I get: pvaneynd@marvin:~/$ echo '#include <stdio.h>' | gcc -c -E - 2>&1 | grep 'SEEK_SET' pvaneynd@marvin:~/$ echo '#include <stdio.h>' | gcc -c -E -M - 2>&1 | grep 'SEEK_SET' pvaneynd@marvin:~/$ echo '#include <stdio.h>' | gcc -c -E -MM - 2>&1 | grep 'SEEK_SET' pvaneynd@marvin:~/$ echo '#include <stdio.h>' | gcc -c -E -MD - 2>&1 | grep 'SEEK_SET' pvaneynd@marvin:~/$ echo '#include <stdio.h>' | gcc -c -E -MMD - 2>&1 | grep 'SEEK_SET' pvaneynd@marvin:~/$ gcc --version 2.95.2 No SEEK_SET=1 here... :-( > > The way I did it for db-sockets was to have a "spec" file containing > > the constants/struct definitions that I wanted, and turn it into a > > C program which when compiled and run printed out the appropriate > > vaues in lisp-readable form. That way I not only get cpp to expand my If we go that route there is SWIG stuff we could use, or alternatively there is ./configure, not? > > preprocessor definitions, but I get the C compiler to work out the > > offsets in my structure definitions, so I don't have to edit CL code > > when the third-party library I'm using adds a field to a structure - I > > just recompile. Yes. Of course then the nice libc people decide that errno should look like: # define errno (*__errno_location ()) The joys this brought me... > I was thinking of making calls to the foreign-interface ... > (def-alien-type nil > (struct stat > (st-dev dev-t) > (st-ino ino-t) Including non-trivial notational changes dev_t -> dev-t and stuff? I don't want to sound agressive, but I've gone over the possibilities a million times already :-(. I think a small handcoded OS interface with a Lisp support library is the only way. _please prove me wrong!_ Groetjes, Peter -- It's logic Jim, but not as we know it. | pva...@de... for pleasure, "God, root, what is difference?" - Pitr| "God is more forgiving." - Dave Aronson| http://cvs2.cons.org/~pvaneynd/ |