From: David R. <da...@ry...> - 2013-06-28 02:36:20
|
This is rube goldbergian. I know there are a 100 better ways to solve this "problem" but my son wants to do it a particular way and I think we are pretty close. We've got a Raspberry Pi. He wants to write a program to run on it in lisp. But he doesn't want to have to install Linux and run in userspace. He wants his program to be the operating system, or at least the only thing running (as another project he wants to boot right into a game written in lisp). We are already able to cross-compile C/ASM from i386 to ARM. Today I figured out how to use clisp to compile lisp to our intel/amd machines (we are both running Debian, btw). I know clisp runs on the arm chip(s). What I want to be able to do is cross-compile to arm with clisp. Is this possible? Do I have to tell clisp where my cross-compilation toolchain is? Or will it only build binaries for the host architecture? |
From: Pascal J. B. <pj...@in...> - 2013-06-28 18:22:59
|
David Rysdam <da...@ry...> writes: > This is rube goldbergian. I know there are a 100 better ways to solve > this "problem" but my son wants to do it a particular way and I think we > are pretty close. > > We've got a Raspberry Pi. He wants to write a program to run on it in > lisp. But he doesn't want to have to install Linux and run in > userspace. He wants his program to be the operating system, or at least > the only thing running (as another project he wants to boot right into a > game written in lisp). AFAIK, it won't be simple to make clisp run on bare metal. Also, I don't remember anybody working on this problem. Somebody has worked on making sbcl run on bare metal. But this probably involved ix86, I'm not sure how this work would be useful for ARM. The CL implementation that runs on bare metal and which is designed to write OS in, is Movitz. But again, it's ix86 specific. You'd have to add an ARM backend to the Movitz compiler. Finally, there are also ccl and ecl that are able to run on ARM. CCL runs quite well there (as long as it's the right version of ARM). But like clisp, they both require a linux kernel, making them run on the bare metal would involve the same work as with clisp. This work consists in implementing the POSIX features that are used and necessary to the implementation. The easiest path to boot a CL running on the bare hardware on ARM today, would be, IMO, to use an ix86/arm emulator, and to run Movitz on it. Yes, it would not run at top-speed, but it would be usable and doable quickly enough. Have a look at this project: http://dmitry.gr/index.php?r=05.Projects&proj=07.%20Linux%20on%208bit Using a ix86/arm emulator to boot Movitz would be much easier than that. Otherwise, making clisp bootable on the bare hardware would have the advantage of being able to target any processor for which we have a C compiler. On the other hand, making ccl bootable on the bare hardware would give you a better ARM compiler. I don't know your objective, but it may still be a good enough plan to run the lisp implementation on a bare Linux kernel. Just something like: https://www.informatimago.com/linux/emacs-on-user-mode-linux.html With a CL implementation you wouldn't need the external mount(1) command, since we can do that with a syscall thru CFFI. Running clisp or ccl on a "bare" linux this way can be archived in an afternoon, and then you can have fun developping the rest of the OS (not the kernel, but the more fun parts, including AI user interface). > We are already able to cross-compile C/ASM from i386 to ARM. Today I > figured out how to use clisp to compile lisp to our intel/amd machines > (we are both running Debian, btw). I know clisp runs on the arm > chip(s). What I want to be able to do is cross-compile to arm with > clisp. Is this possible? More than just cross-compile, you want also to target the bare hardware. This means that you want to use the tool chain used to compile kernels (like linux), not the tool chain used to compile user space programs. Here is a good resource for kernel writers: http://wiki.osdev.org/Main_Page Of course, if you don't want to use linux, you may also want to avoid grub, and rather write your own boot loader in lisp too ;-) > Do I have to tell clisp where my cross-compilation toolchain is? Or will > it only build binaries for the host architecture? ISTR that it's possible. In any case, it shouldn't be too hard to do, just specifying CC as an environment variable to configure. See the tail of: ./configure --help -- __Pascal Bourguignon__ http://www.informatimago.com/ A bad day in () is better than a good day in {}. You know you've been lisping too long when you see a recent picture of George Lucas and think "Wait, I thought John McCarthy was dead!" -- Dalek_Baldwin |
From: David R. <da...@ry...> - 2013-06-28 20:02:58
|
"Pascal J. Bourguignon" <pj...@in...> writes: > David Rysdam <da...@ry...> writes: > >> This is rube goldbergian. I know there are a 100 better ways to solve >> this "problem" but my son wants to do it a particular way and I think we >> are pretty close. >> >> We've got a Raspberry Pi. He wants to write a program to run on it in >> lisp. But he doesn't want to have to install Linux and run in >> userspace. He wants his program to be the operating system, or at least >> the only thing running (as another project he wants to boot right into a >> game written in lisp). > > AFAIK, it won't be simple to make clisp run on bare metal. > Also, I don't remember anybody working on this problem. I hadn't even considered booting to CL. That could also be cool. Thanks for the ton of info on that idea. >> We are already able to cross-compile C/ASM from i386 to ARM. Today I >> figured out how to use clisp to compile lisp to our intel/amd machines >> (we are both running Debian, btw). I know clisp runs on the arm >> chip(s). What I want to be able to do is cross-compile to arm with >> clisp. Is this possible? > > More than just cross-compile, you want also to target the bare > hardware. This means that you want to use the tool chain used to > compile kernels (like linux), not the tool chain used to compile user > space programs. > > Here is a good resource for kernel writers: > http://wiki.osdev.org/Main_Page > > Of course, if you don't want to use linux, you may also want to avoid > grub, and rather write your own boot loader in lisp too ;-) Yes. Or maybe. I was kind of imagining a simple bootloader in asm, based on some info here: http://www.cl.cam.ac.uk/projects/raspberrypi/tutorials/os/#lessons But doing everything in relatively pure Lisp would be interesting. >> Do I have to tell clisp where my cross-compilation toolchain is? Or will >> it only build binaries for the host architecture? > > ISTR that it's possible. In any case, it shouldn't be too hard to do, > just specifying CC as an environment variable to configure. See the > tail of: > > ./configure --help I'm not convinced we are talking about the same thing, so let me be clear: clisp running natively on x86, producing a binary for arm. I can do this with a native x86 gcc producing arm binaries. But I don't know how clisp produces it's binaries. By calling CC? Or does it do it itself? |
From: Pascal J. B. <pj...@in...> - 2013-06-28 22:05:34
|
David Rysdam <da...@ry...> writes: > "Pascal J. Bourguignon" <pj...@in...> writes: >> David Rysdam <da...@ry...> writes: >> >>> This is rube goldbergian. I know there are a 100 better ways to solve >>> this "problem" but my son wants to do it a particular way and I think we >>> are pretty close. >>> >>> We've got a Raspberry Pi. He wants to write a program to run on it in >>> lisp. But he doesn't want to have to install Linux and run in >>> userspace. He wants his program to be the operating system, or at least >>> the only thing running (as another project he wants to boot right into a >>> game written in lisp). >> >> AFAIK, it won't be simple to make clisp run on bare metal. >> Also, I don't remember anybody working on this problem. > > I hadn't even considered booting to CL. That could also be cool. Thanks > for the ton of info on that idea. Oh, so you meant to write a kernel, but not necessarily to include CL in that kernel? Then you don't need to cross compile the CL implementation to ARM. You only need to have an ARM code generator written in lisp. You can easily write one (since ARM has quite simple and orthogonal instruction sets), or you can find one in free software. http://www.cliki.net/machine%20code You could also perhaps use an ARM code generator found in a CL compiler like in ccl. Then you design a language (lisp-like) that use this code generator to generate the ARM binaries you need to boot your computer. While your lisp-like language can be compiled to ARM, you can also very easily write functions and macros to have it run directly in CL, so you can also develop and debug the program in the lisp environment. Now, I say lisp-like, it could be a superset of some subset of Common Lisp itself. A subset because you may not want, and don't need to implement a whole CL implementation, to write an OS. A superset, because you may need some primitives not provided by CL, like register memory, or device accesses. Actually, you could just use ccl, by modifying its code generator, so that you can generate ARM binary files in the format you need to boot your kernel, instead of saving the usual .fasl files ccl saves when it works on normal systems. But you'd still have to do the work of implementing the dependencies that you'll lack on the bare metal, or take extreme care not to use any of CL in your system. See what I mean. If you write a little function to keep track of processes: ------------------------------------------------------------------------ (defpackage "MYOS-PROCESSES" (:use "CL") ; dangerous. it would be better to (:use "MYOS-LISP") (:export "ADD-PROCESS")) (in-package "MYOS-PROCESSES") (defvar *processes* '()) (defun add-process (new-process) (push new-process *processes*)) ------------------------------------------------------------------------ Now, when we compile this, we get a .fasl file that contains instructions to create the *PROCESSES* variable and initialize it to NIL, and to create a function named ADD-PROCESS, that will call the function CL:CONS. You can generate it in your specific binary file format to load it as a kernel. But if you want to run this you will need to include a function named CONS (as well, as all the internal functions used to create global special variables, functions, to manage symbol bindings, etc). The function CONS is rather simple, but it calls out to the garbage collector, so you will have to include also the garbage collector. The CL garbage collector may report memory collected, or may signal an STORAGE-CONDITION, so you need to include PRINC, TERPRI, and the condition system. When a STORAGE-CONDITION is signaled, if it is not handled, then the debugger is called, so you need to include the debugger along. The default debugger uses the I/O system (print, read, etc), so you need to include the lisp printer, the lisp reader. It then evaluates the expressions it read, so you need to include EVAL. That is, you need to include an interpreter or a compiler. Since the user may want to use any CL operator, and you may want to provide them this ability to use any CL operator to debug their kernel code, then you will want to include all the CL system. Therefore you just want to modify an implementation that has an ARM compiler to be able to generate a kernel image instead of generating a lisp image or an ELF executable image. A kernel image, is just a binary blob that can be loaded by a boot loader, it can be in any format. (Including ELF, so the modifications are not necessarily big on this part). But since you get the whole CL implementation, you must re-implement the parts that depend on linux, since you won't have linux. Instead of using write(2) to print bytes to the terminal, you must call the function that will output the characters to the device you use to show text (it can be a serial chip, a USB device, a frame buffer bit map, whatever). Since you don't have open(2) to open files, you need to write the function and the device driver that will read data from the sdcard, or a UBS disk, etc. I would advise you to try out Movitz, and see how it's organized. >>> We are already able to cross-compile C/ASM from i386 to ARM. Today I >>> figured out how to use clisp to compile lisp to our intel/amd machines >>> (we are both running Debian, btw). I know clisp runs on the arm >>> chip(s). What I want to be able to do is cross-compile to arm with >>> clisp. Is this possible? >> >> More than just cross-compile, you want also to target the bare >> hardware. This means that you want to use the tool chain used to >> compile kernels (like linux), not the tool chain used to compile user >> space programs. >> >> Here is a good resource for kernel writers: >> http://wiki.osdev.org/Main_Page >> >> Of course, if you don't want to use linux, you may also want to avoid >> grub, and rather write your own boot loader in lisp too ;-) > > Yes. Or maybe. I was kind of imagining a simple bootloader in asm, based > on some info here: > > http://www.cl.cam.ac.uk/projects/raspberrypi/tutorials/os/#lessons > > But doing everything in relatively pure Lisp would be interesting. > >>> Do I have to tell clisp where my cross-compilation toolchain is? Or will >>> it only build binaries for the host architecture? >> >> ISTR that it's possible. In any case, it shouldn't be too hard to do, >> just specifying CC as an environment variable to configure. See the >> tail of: >> >> ./configure --help > > I'm not convinced we are talking about the same thing, so let me be > clear: clisp running natively on x86, producing a binary for arm. > > I can do this with a native x86 gcc producing arm binaries. But I don't > know how clisp produces it's binaries. By calling CC? Or does it do it > itself? clisp is written in C, hence the "c" in "clisp". So indeed, it runs the compiler referenced in the environment variable CC, when there's one, or some other default (cc or gcc) otherwise. clisp has an interpreter (written in C), and it has a virtual machine (written in C) that interprets a lisp-friendly byte-code, and it has a lisp compiler (written in lisp) that generates this lisp-friendly byte-code (you can see that with CL:DISASSEMBLE or in the .fas files). ccl has native compilers for various processors (x86, ppc, arm). It is written almost entirely in lisp, plus some assembler and some C. ecl has an interpreter, a byte code virtual machine and a byte-code compiler, and a native compiler that works by generating C code, fed to gcc to produce elf .o files that are dynamically loaded into the ecl process. ecl is almost entirely written in lisp (plus some C). You can run ecl without gcc, but then you can't compile to native code. sbcl is like ccl, but it also has an interpreter, which is not used by default but that applications may activate in the case it would gives them a speed boost. Yes, an interpreter let you run code faster than compilers! (Notably when you have a very careful and slow compiler like sbcl). -- __Pascal Bourguignon__ http://www.informatimago.com/ A bad day in () is better than a good day in {}. You know you've been lisping too long when you see a recent picture of George Lucas and think "Wait, I thought John McCarthy was dead!" -- Dalek_Baldwin |
From: David R. <da...@ry...> - 2013-07-02 00:33:17
|
"Pascal J. Bourguignon" <pj...@in...> writes: > David Rysdam <da...@ry...> writes: > >> "Pascal J. Bourguignon" <pj...@in...> writes: >>> David Rysdam <da...@ry...> writes: >>> >>>> This is rube goldbergian. I know there are a 100 better ways to solve >>>> this "problem" but my son wants to do it a particular way and I think we >>>> are pretty close. >>>> >>>> We've got a Raspberry Pi. He wants to write a program to run on it in >>>> lisp. But he doesn't want to have to install Linux and run in >>>> userspace. He wants his program to be the operating system, or at least >>>> the only thing running (as another project he wants to boot right into a >>>> game written in lisp). >>> >>> AFAIK, it won't be simple to make clisp run on bare metal. >>> Also, I don't remember anybody working on this problem. >> >> I hadn't even considered booting to CL. That could also be cool. Thanks >> for the ton of info on that idea. > > Oh, so you meant to write a kernel, but not necessarily to include CL in > that kernel? Right. > But since you get the whole CL implementation, you must re-implement the > parts that depend on linux, since you won't have linux. Instead of > using write(2) to print bytes to the terminal, you must call the > function that will output the characters to the device you use to show > text (it can be a serial chip, a USB device, a frame buffer bit map, > whatever). Since you don't have open(2) to open files, you need to > write the function and the device driver that will read data from the > sdcard, or a UBS disk, etc. Yes, this is what I was envisioning. I had gotten as far as realizing I'd need to implement (a small subset of) glibc but hadn't really realized that I'd have to either do this outside of lisp OR modify my lisp. Actually, wait. If there was the equivalent of inline ASM, that would work. Make a (write)/(open)/etc that are just thin lisp wrappers around some inline ARM and call those. > I would advise you to try out Movitz, and see how it's organized. OK. >> I can do this with a native x86 gcc producing arm binaries. But I don't >> know how clisp produces it's binaries. By calling CC? Or does it do it >> itself? > > clisp is written in C, hence the "c" in "clisp". > > So indeed, it runs the compiler referenced in the environment variable > CC, when there's one, or some other default (cc or gcc) otherwise. > > > clisp has an interpreter (written in C), and it has a virtual machine > (written in C) that interprets a lisp-friendly byte-code, and it has a > lisp compiler (written in lisp) that generates this lisp-friendly > byte-code (you can see that with CL:DISASSEMBLE or in the .fas files). > > ccl has native compilers for various processors (x86, ppc, arm). It is > written almost entirely in lisp, plus some assembler and some C. In that case, I think this is what I want. I think I have enough information to take a few stabs at it, fail miserably and then come crawling back to ask more intelligent questions. :) Thanks! |