Thread: [Etherboot-developers] PXELINUX on top of Etherboot?
Brought to you by:
marty_connor,
stefanhajnoczi
Menu
▾
▴
etherboot-developers
|
From: H. P. A. <hp...@zy...> - 2003-04-11 01:18:31
|
[Hi all... this is a copy of a message I just sent to the SYSLINUX mailing list. I apologize if this is a bit clueless for this list... I'm not really that well-informed about Etherboot.] Question for people... Has anyone actually tried running PXELINUX on top of Etherboot? There is something in Etherboot 5.1.7 called FREEBSD_PXEEMU, which seems to be a PXE stack subset, in particular it seems to support the following calls: PXENV_GET_CACHED_INFO PXENV_UDP_OPEN PXENV_UDP_WRITE PXENV_UDP_READ PXENV_UDP_CLOSE PXENV_UNLOAD_STACK PXENV_UNDI_SHUTDOWN This happens to be pretty close to the subset used by PXELINUX, with two exceptions: PXELINUX will also invoke PXENV_STOP_UNDI (for version >= 2.00) or PXENV_UNDI_CLEANUP (for version < 2.00). I'd be really curious to see how this would play out. There are of course also other kinds of concerns, e.g. if the detection mechanism works properly, and if it stomps on memory it shouldn't be. I'm sorry if this has happened to be public knowledge to everyone but myself for a long time. If this happens to work, though, I'd like to post it on the PXELINUX web pages, and if not, I'd like to see what I can do to help debug it. -hpa |
|
From: <ebi...@ln...> - 2003-04-11 05:17:00
|
"H. Peter Anvin" <hp...@zy...> writes: > [Hi all... this is a copy of a message I just sent to the SYSLINUX mailing list. > > I apologize if this is a bit clueless for this list... I'm not really that > well-informed about Etherboot.] > > Question for people... > > Has anyone actually tried running PXELINUX on top of Etherboot? There > is something in Etherboot 5.1.7 called FREEBSD_PXEEMU, which seems to be > a PXE stack subset, in particular it seems to support the following calls: > > PXENV_GET_CACHED_INFO > PXENV_UDP_OPEN > PXENV_UDP_WRITE > PXENV_UDP_READ > PXENV_UDP_CLOSE > PXENV_UNLOAD_STACK > PXENV_UNDI_SHUTDOWN > > This happens to be pretty close to the subset used by PXELINUX, with two > exceptions: PXELINUX will also invoke PXENV_STOP_UNDI (for version >= > 2.00) or PXENV_UNDI_CLEANUP (for version < 2.00). I'd be really curious > to see how this would play out. > > There are of course also other kinds of concerns, e.g. if the detection > mechanism works properly, and if it stomps on memory it shouldn't be. > > I'm sorry if this has happened to be public knowledge to everyone but > myself for a long time. If this happens to work, though, I'd like to > post it on the PXELINUX web pages, and if not, I'd like to see what I > can do to help debug it. The short answer is that the code in at least one place was not integrated cleanly and a working version has never made it into an etherboot release. That said I surveyed the code a while ago and fixing it up does not look too difficult. The concept are at least clean. With respect to stomping memory except for a small real mode trampoline the development version of etherboot 5.1.x lives at the high end of memory below 4GB, so the code should be safe. It has been observed before that getting pxelinux running should not be too difficult, and no one was opposed. But so far no one has volunteered to debug it into operation. Besides the missing functions there are two significant stumbling blocks. 1) The callbacks are currently 32bit only. 2) The pxe entry points depend on freebsd headers so the does not currently compile on linux. Looking through the actual pxe wrappers the code should continue to work. But there was at least one location where the fact that etherboot can now live above 1MB breaks the code. But for the short term that can be worked around by just disabling -DRELOCATION though etherboot stomps on to much memory in that case. Eric |
|
From: H. P. A. <hp...@zy...> - 2003-04-11 05:36:11
|
Eric W. Biederman wrote: > > The short answer is that the code in at least one place was not > integrated cleanly and a working version has never made it into an > etherboot release. > > That said I surveyed the code a while ago and fixing it up does not > look too difficult. The concept are at least clean. > > With respect to stomping memory except for a small real mode > trampoline the development version of etherboot 5.1.x lives at the > high end of memory below 4GB, so the code should be safe. > OK; it really should hook INT 15h (and move the low DOS memory ceiling BIOS variable at physical address 0x413) to make itself invisible -- PXELINUX will actually avoid touching the top 128K of memory just in case a PXE stack doesn't do that, but that's the "right thing" to do. When you get the cleanup command (unload stack) then you remove the hooks, of course. > It has been observed before that getting pxelinux running should not > be too difficult, and no one was opposed. But so far no one has > volunteered to debug it into operation. > > Besides the missing functions there are two significant stumbling blocks. > 1) The callbacks are currently 32bit only. OK... not sure what you mean with callbacks here. Are you referring to the 32-bit PXE entrypoint, or the 32-bit PXE status callback? PXELINUX doesn't use the status callbacks, but it does use the 16-bit PXENV+ or !PXE entrypoints. > 2) The pxe entry points depend on freebsd headers so the does not > currently compile on linux. <nod> that doesn't sound too hard to fix. > Looking through the actual pxe wrappers the code should continue to > work. But there was at least one location where the fact that > etherboot can now live above 1MB breaks the code. But for > the short term that can be worked around by just disabling -DRELOCATION > though etherboot stomps on to much memory in that case. How much memory is "too much"? As far as I know, some actual PXE stacks will hog memory down to near the 448K mark. PXELINUX only requires 384K of conventional (low 1M) memory, although I might move that mark up to 448K in a future version in order to allow the use of large blocks through the TFTP blksize option. I'll see if I have a chance to look at this. If the learning curve isn't too steep I might be able to make it work. -hpa |
|
From: <ebi...@ln...> - 2003-04-11 05:59:21
|
"H. Peter Anvin" <hp...@zy...> writes: > Eric W. Biederman wrote: > > The short answer is that the code in at least one place was not > > integrated cleanly and a working version has never made it into an > > etherboot release. That said I surveyed the code a while ago and > > fixing it up does not > > look too difficult. The concept are at least clean. > > With respect to stomping memory except for a small real mode > > trampoline the development version of etherboot 5.1.x lives at the > > high end of memory below 4GB, so the code should be safe. > > > > OK; it really should hook INT 15h (and move the low DOS memory ceiling BIOS > variable at physical address 0x413) to make itself invisible -- Yep. One of the little things that has not happened yet. > PXELINUX will actually avoid touching the top 128K of memory just in > case a PXE stack doesn't do that, but that's the "right thing" to do. > > When you get the cleanup command (unload stack) then you remove the > hooks, of course. Yep... > > It has been observed before that getting pxelinux running should not > > be too difficult, and no one was opposed. But so far no one has > > volunteered to debug it into operation. > > Besides the missing functions there are two significant stumbling > > blocks. > > 1) The callbacks are currently 32bit only. > > OK... not sure what you mean with callbacks here. Are you referring > to the 32-bit PXE entrypoint, or the 32-bit PXE status callback? I was thinking of anything call back into etherboot as a callback. Sorry it was confusing terminology for the context. > PXELINUX doesn't use the status callbacks, but it does use the 16-bit > PXENV+ or !PXE entrypoints. Right and that is what I meant to say that is not currently implemented, the 16bit entry point. But I think wrappers that call the 32bit entry point should be possible. > > 2) The pxe entry points depend on freebsd headers so the does not > > currently compile on linux. > > <nod> that doesn't sound too hard to fix. > > > Looking through the actual pxe wrappers the code should continue to > > work. But there was at least one location where the fact that > > etherboot can now live above 1MB breaks the code. But for > > the short term that can be worked around by just disabling -DRELOCATION > > though etherboot stomps on to much memory in that case. > > How much memory is "too much"? As far as I know, some actual PXE > stacks will hog memory down to near the 448K mark. PXELINUX only > requires 384K of conventional (low 1M) memory, although I might move > that mark up to 448K in a future version in order to allow the use of > large blocks through the TFTP blksize option. Then with a little bit of care you can probably make it work without -DRELOCATION. And then fix it up to handle etherboot living about 1MB. Before 5.1.x all of the code+data would fit in 48K. In 5.1.x things are not quite as tight because they don't need to be. > I'll see if I have a chance to look at this. If the learning curve > isn't too steep I might be able to make it work. The code is all C and gas assembly. Probably the worst part are the gymnastics etherboot currently uses to make bios calls while keeping all of it's code above 1MB. Basically it sets up a temporary stack pushes the code does the BIOS call, returns to it's normal stack and copies return arguments between the stacks, and forgets the temporary stack. If things look a too confusing you could start with 5.0.9 on the stable branch. The code is not quite as capable but the learning curve is not as steep. Though that is practically the same as compiling 5.1.x without -DRELOCATION. Eric |
|
From: H. P. A. <hp...@zy...> - 2003-04-11 06:21:44
|
Eric W. Biederman wrote: >> >>OK; it really should hook INT 15h (and move the low DOS memory ceiling BIOS >>variable at physical address 0x413) to make itself invisible -- > > Yep. One of the little things that has not happened yet. > This is actually something that's surprisingly painful, mostly because of the sheer number of calls, and the complexity of the E820 memory map. MEMDISK (part of the SYSLINUX package, but actually a standalone program) does it, so you might be able to snarf some code. The code is currently GPL; if that's not OK with you and you want to snarf pieces check with me first. > > Right and that is what I meant to say that is not currently > implemented, the 16bit entry point. But I think wrappers that > call the 32bit entry point should be possible. > I think that's actually how most PXE stacks operate. >>> >>How much memory is "too much"? As far as I know, some actual PXE >>stacks will hog memory down to near the 448K mark. PXELINUX only >>requires 384K of conventional (low 1M) memory, although I might move >>that mark up to 448K in a future version in order to allow the use of >>large blocks through the TFTP blksize option. > > Then with a little bit of care you can probably make it work without > -DRELOCATION. And then fix it up to handle etherboot living about > 1MB. Before 5.1.x all of the code+data would fit in 48K. In 5.1.x > things are not quite as tight because they don't need to be. > If you're occupying less than 128K of conventional then you're actually better than most commercial PXE stacks. This would probably be a lot less painful than hooking INT 15h, although I'm pretty agnostic about it. > > The code is all C and gas assembly. Probably the worst part are the > gymnastics etherboot currently uses to make bios calls while keeping > all of it's code above 1MB. Basically it sets up a temporary stack > pushes the code does the BIOS call, returns to it's normal stack > and copies return arguments between the stacks, and forgets the > temporary stack. > <nod> I have some pretty similar code in SYSLINUX, to support COM32 programs, in MEMDISK (because I wanted to write the MEMDISK installer in C), as well as in the never-completed Genesis project... so I'm familar with the concept :) > If things look a too confusing you could start with 5.0.9 on the > stable branch. The code is not quite as capable but the learning > curve is not as steep. Though that is practically the same as > compiling 5.1.x without -DRELOCATION. Cool. Don't know if/when I would have time to look at it, but I've put it on my list. (If someone else is interested in working on it, though, please do go ahead...) -hpa |
|
From: <ebi...@ln...> - 2003-04-12 02:38:01
|
"H. Peter Anvin" <hp...@zy...> writes: > Eric W. Biederman wrote: > >> > >>OK; it really should hook INT 15h (and move the low DOS memory ceiling BIOS > >>variable at physical address 0x413) to make itself invisible -- > > Yep. One of the little things that has not happened yet. > > > > This is actually something that's surprisingly painful, mostly because of the > sheer number of calls, and the complexity of the E820 memory map. MEMDISK (part > of the SYSLINUX package, but actually a standalone program) does it, so you > might be able to snarf some code. The code is currently GPL; if that's not OK > with you and you want to snarf pieces check with me first. All of etherboot is GPL'd so we should not have a problem there. > > Right and that is what I meant to say that is not currently > > implemented, the 16bit entry point. But I think wrappers that > > call the 32bit entry point should be possible. > > > > I think that's actually how most PXE stacks operate. Just to get the thought recorded what looks to be needed is to build a pxe_stub.S that has the PXEENV structure and the 16bit stubs that call the 32bit code. And something to copy that down to low memory. > >>How much memory is "too much"? As far as I know, some actual PXE > >>stacks will hog memory down to near the 448K mark. PXELINUX only > >>requires 384K of conventional (low 1M) memory, although I might move > >>that mark up to 448K in a future version in order to allow the use of > >>large blocks through the TFTP blksize option. > > Then with a little bit of care you can probably make it work without > > -DRELOCATION. And then fix it up to handle etherboot living about > > 1MB. Before 5.1.x all of the code+data would fit in 48K. In 5.1.x > > things are not quite as tight because they don't need to be. > > > > If you're occupying less than 128K of conventional then you're actually better > than most commercial PXE stacks. Etherboot can still load zImages, so it has been a requirement. > This would probably be a lot less painful than > hooking INT 15h, although I'm pretty agnostic about it. Actually I suspect if INT 15h is just replaced for the memory functions the code should not be very difficult to write. Which should build the corrected map once in C before proceeding. > > The code is all C and gas assembly. Probably the worst part are the > > gymnastics etherboot currently uses to make bios calls while keeping > > all of it's code above 1MB. Basically it sets up a temporary stack > > pushes the code does the BIOS call, returns to it's normal stack > > and copies return arguments between the stacks, and forgets the > > temporary stack. > > <nod> I have some pretty similar code in SYSLINUX, to support COM32 programs, in > > MEMDISK (because I wanted to write the MEMDISK installer in C), as well as in > the never-completed Genesis project... so I'm familar with the > concept :) I still chuckle at the COM32 concept as it looks like going quite a ways to avoid 300 bytes of overhead for an ELF header... Though I can see how it can be useful if space is at a very high premium. > > If things look a too confusing you could start with 5.0.9 on the > > stable branch. The code is not quite as capable but the learning > > curve is not as steep. Though that is practically the same as > > compiling 5.1.x without -DRELOCATION. > > Cool. Don't know if/when I would have time to look at it, but I've put it on my > list. (If someone else is interested in working on it, though, please do go > ahead...) Well we will see how it goes. Currently I am busy porting LinuxBIOS to the Opteron so I won't be able to get at very much stuff for a while. Though once it works it should be possible to test the code by pxe booting etherboot :) Eric |
|
From: H. P. A. <hp...@zy...> - 2003-04-12 03:21:02
|
Eric W. Biederman wrote: > > Just to get the thought recorded what looks to be needed is to > build a pxe_stub.S that has the PXEENV structure and the 16bit stubs > that call the 32bit code. And something to copy that down to low > memory. > Right. Shouldn't be too difficult. > >>>>How much memory is "too much"? As far as I know, some actual PXE >>>>stacks will hog memory down to near the 448K mark. PXELINUX only >>>>requires 384K of conventional (low 1M) memory, although I might move >>>>that mark up to 448K in a future version in order to allow the use of >>>>large blocks through the TFTP blksize option. >>> >>>Then with a little bit of care you can probably make it work without >>>-DRELOCATION. And then fix it up to handle etherboot living about >>>1MB. Before 5.1.x all of the code+data would fit in 48K. In 5.1.x >>>things are not quite as tight because they don't need to be. >>> >> >>If you're occupying less than 128K of conventional then you're actually better >>than most commercial PXE stacks. > > Etherboot can still load zImages, so it has been a requirement. > True, although SYSLINUX deals with this by always loading high, and if the image is a zImage it copies it to low memory immediately before invoking the kernel. > >>This would probably be a lot less painful than >>hooking INT 15h, although I'm pretty agnostic about it. > > Actually I suspect if INT 15h is just replaced for the memory > functions the code should not be very difficult to write. Which > should build the corrected map once in C before proceeding. > That's pretty much the only sane way to do it (and is, indeed, that MEMDISK does.) The pain is actually getting the e820 map routines to work sanely. The C files memdisk/msetup.c and memdisk/e820func.c are the interesting ones within MEMDISK; once you have that you can just call insertrange() to mark memory as a particular type. parse_mem() then produces the non-e820 memory map types. The tricky part was realizing > >>>The code is all C and gas assembly. Probably the worst part are the >>>gymnastics etherboot currently uses to make bios calls while keeping >>>all of it's code above 1MB. Basically it sets up a temporary stack >>>pushes the code does the BIOS call, returns to it's normal stack >>>and copies return arguments between the stacks, and forgets the >>>temporary stack. >> >><nod> I have some pretty similar code in SYSLINUX, to support COM32 programs, in >> >>MEMDISK (because I wanted to write the MEMDISK installer in C), as well as in >>the never-completed Genesis project... so I'm familar with the >>concept :) > > I still chuckle at the COM32 concept as it looks like going quite a > ways to avoid 300 bytes of overhead for an ELF header... Though I can > see how it can be useful if space is at a very high premium. > Remember, the SYSLINUX program itself needs to fit on floppies and take minimum amounts of space. It's 7828 bytes in size at the moment, and the COM32 loader is just "load this file into memory at this address" -- no parsing whatsoever. > > Well we will see how it goes. > > Currently I am busy porting LinuxBIOS to the Opteron so I won't be able to get > at very much stuff for a while. > Why not just run a 32-bit LinuxBIOS on Opteron? Or do you mean to the K8 memory controller and stuff? > > Though once it works it should be possible to test the code by pxe booting > etherboot :) > Gotta love recursion... -hpa |
|
From: <ebi...@ln...> - 2003-04-12 03:41:03
|
"H. Peter Anvin" <hp...@zy...> writes: > Eric W. Biederman wrote: > True, although SYSLINUX deals with this by always loading high, and if the image > > is a zImage it copies it to low memory immediately before invoking the kernel. A nice twist. > >>MEMDISK (because I wanted to write the MEMDISK installer in C), as well as in > >>the never-completed Genesis project... so I'm familar with the > >>concept :) > > I still chuckle at the COM32 concept as it looks like going quite a > > ways to avoid 300 bytes of overhead for an ELF header... Though I can > > see how it can be useful if space is at a very high premium. > > > > Remember, the SYSLINUX program itself needs to fit on floppies and take minimum > amounts of space. It's 7828 bytes in size at the moment, and the COM32 loader > is just "load this file into memory at this address" -- > > no parsing whatsoever. Except when I implement sanity checks there is no parsing with ELF headers either, the structures area all native endian, and wordsize. The difference is load this chunk into memory at this address, and load these other chunks into memory at these other addresses, and finally jump there. I get some real portability and prototyping benefits because it is a standard file format. But when space is at an absolute premium those aren't the most important things. I do see the point of COM32, even if it does not make much sense when you have a few more bytes available, in the circumstances I care about. > > Well we will see how it goes. > > Currently I am busy porting LinuxBIOS to the Opteron so I won't be able to get > > > at very much stuff for a while. > > > > Why not just run a 32-bit LinuxBIOS on Opteron? Or do you mean to the K8 memory > controller and stuff? Mostly the K8 memory controller. I guess a better description would be the K8 chipset :) The extra registers are a very compelling reason to go 64bit before the memory controller is initialized though. For the most part with LinuxBIOS it does not matter as the code is in C anyway. > > Though once it works it should be possible to test the code by pxe booting > > etherboot :) > > > > Gotta love recursion... Especially when there is proper tail recursion. :) I already use that test path with ELF images. And if the recursion stops before I tell it to I know there is something wrong! And it already exist with the mknbi format as well. Eric |
|
From: H. P. A. <hp...@zy...> - 2003-04-12 03:46:30
|
Eric W. Biederman wrote: > > Except when I implement sanity checks there is no parsing with ELF > headers either, the structures area all native endian, and wordsize. > The difference is load this chunk into memory at this address, and > load these other chunks into memory at these other addresses, and > finally jump there. > Right... I have a coarse-grained routine that maps the file at a single address without looking at it at all first. > I get some real portability and prototyping benefits because it is > a standard file format. But when space is at an absolute premium > those aren't the most important things. Right. Note that I usually use standard ELF tools to construct COM32 images, and then just "objcopy -O binary" to finish. Very simple. > > Especially when there is proper tail recursion. :) > > I already use that test path with ELF images. And if the recursion > stops before I tell it to I know there is something wrong! > > And it already exist with the mknbi format as well. > Hmmm.... :/ -hpa |
