nilo-discuss Mailing List for NILO
Brought to you by:
marty_connor,
stefanhajnoczi
Menu
▾
▴
nilo-discuss — General discussion regarding NILO
You can subscribe to this list here.
| 2005 |
Jan
|
Feb
(19) |
Mar
(56) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|---|
|
From: Marty C. <md...@en...> - 2005-03-23 23:13:46
|
This is a reminder message that
nil...@li...
is moving to
gpx...@li...
Please update your rules and pointers.
I have subscribed all current members of nilo-discuss to gpxe-discuss
(you should have received a welcome message to the new list recently).
This message will also be a notification in the nilo-discuss archives.
Thanks for your patience and support during this transition.
Marty
|
|
From: Marty C. <md...@en...> - 2005-03-23 15:42:07
|
On Mar 22, 2005, at 5:19 PM, Marty Connor wrote:
> Now, there will be a little bit of work involved in getting=20
> SourceForge to do some moving of CVS repositories, but I have agreed=20=
> to get that done, since this name change thing is my idea. It's worth=20=
> the effort of a few emails.
This is now done. Anonymous cvs has been updated, and can be had by=20
doing something like:
cvs -d:pserver:ano...@cv...:/cvsroot/gpxe login
=A0
(just hit return at the password prompt)
Then:
cvs -z3 -d:pserver:ano...@cv...:/cvsroot/gpxe co=20=
-P gpxe
For those who want to watch things develop.
> I will also be happy to move everyone on the nilo-discuss mailing list=20=
> to gpxe-discuss, once everything is set up over there, so that won't=20=
> create extra work for people.
I'll be working on this today. I'll leave nilo-discuss in place for a=20=
while after I do the switch to gpxe-discuss, so we can all update our=20
procmail rules and filters. Everyone will probably get a message about=20=
being added by sourceforge mailman, sorry for the noise.
Construction is noisy :)
Marty
|
|
From: Marty C. <md...@en...> - 2005-03-22 22:19:54
|
From the
"A rose by any other name would smell as sweet..."
department.
Michael and I have had a transatlantic phone conversation, and have
come up with what we think is a good plan for continuing development.
To give some background, I've been having some concerns about the
clarity of our vision for NILO, and as we have talked, we seemed to be
coming to a convergence of thought, and this caused me think we should
rename the project to be clearer about what we're doing.
So, I had the good folks of SourceForge create a new project called
gpxe.
Here are two reasons why I think this is a good idea:
First, it better matches what I believe we are working to accomplish,
which is to create a GPL'ed PXE implementation. gpxe will be easy for
people to understand. It follows a pattern of such things as gawk,
gcc, gdb, etc.
Second, gpxe.org/net/com were available. From the standpoint of making
it easy for people to find us, I think it is good to have
easy-to-remember domains. We have nilo.org, but nilo.com and nilo.net
are not available. It's nice to have control of the most obvious
namespace.
Now, there will be a little bit of work involved in getting SourceForge
to do some moving of CVS repositories, but I have agreed to get that
done, since this name change thing is my idea. It's worth the effort
of a few emails.
I will also be happy to move everyone on the nilo-discuss mailing list
to gpxe-discuss, once everything is set up over there, so that won't
create extra work for people.
Alright, let's talk about the "vision thing". I believe our merry band
has come up with some basic things we agree on. If I am mistaken,
please speak up.
1. We will endeavor to follow the PXE specification where possible.
2. Divergences from the specification will be extensions that will only
be
available to loaders by explicit request from the user.
After talking to Michael, I believe we agree that implementing a GPL
PXE is a Good Thing, and that we should work toward that goal as a
first priority.
Michael wants to add a specific extension that allows dynamic module
loading, and for that extension to be in the core. Personally, I am
not particularly interested in module loading, but as long as the
following conditions are met, I think it should be alright as an
extension.
1. The code to load the modules is able to be a compile-time switch,
such that the code can be omitted from the object files, as a way
to save ROM space, and lessen code-path complexity.
2. Even if the code is compiled in, it will only be executed if the
loader
requested by the user is of a particular type. This is to say that
if the file is pxelinux.0, for example, no extensions would be
allowed.
if, however, the file is pxelinux.[z]elf, extensions would be
enabled,
because elf files are understood to require extensions.
Beyond those caveats, it is my hope that we can keep the core code
small, modular, and reasonable uncomplicated. I find a lot of #ifdefs
to be distracting, and I'd really like for our code to be pleasant to
read.
That's pretty much my understanding of where we are now. Please speak
up with any thoughts or corrections. I appreciate your patience with
the startup process. I think it is good to get the foundation and
vision right before trying to build on it, which is why I propose doing
the name change now.
Comments?
Marty
|
|
From: Michael B. <mb...@fe...> - 2005-03-22 16:01:59
|
Just a brief message to outline how I'm currently visualising the architecture, and where the real-mode code and the ELF loader fit into all this: The core code will be the code that is compiled into ROM. The core code will include all the functionality necessary to implement the PXE spec, or as close as is logically feasible given the internal inconsistencies in the spec. In particular, this means that the core code will include a network driver, code to handle DHCP exchanges (including ProxyDHCP, boot server discovery et al), code to handle TFTP transfers and (hopefully) code to handle MTFTP transfers. [1] The core code will include a PXE API interface, at least on i386. [2] The core code will include an ELF loader capable of loading ELF relocatable shared object (.so) files, and a runtime linker capable of linking undefined symbols in the .so files. The core code will include a symbol table for a few well-defined functions in the core code. By this means, .so files can use these core API functions, as well as functions in other .so files. The ABI for the .so files will be the default C ABI used by gcc (cdecl). All non-core code (i.e. code not required for the core functionality) will be moved to a "modules" directory, tweaked to use the ELF API and compiled as .so files. For i386, there will be a compilation option -DKEEP_IT_REAL. If set, the resulting code will restrain itself to living in base memory and will not attempt mode transitions etc. The ELF loader and API will not be compiled in when this option is set; only the PXE API will be available. [1] MTFTP seems to have been designed to mimic TFTP as closely as possible; we may be able to use the same code for both. [2] Yes, I'd like to keep multiple architectures, though I'm quite happy to break support for all of them at present and add them back in as needed. The PXE API is defined only for i386 and the specification is very i386-specific. Etherboot currently makes it easy for other architectures to implement a PXE API if they choose to do so, but there are certain choices that need to be made (such as deciding what to use for the SEGOFF16 datatype) that mean that the end result is, effectively, an Etherboot-specific API anyway. Since there's no standard API on non-i386 platforms, we may as well just say "hey, use the ELF API, since you can just write platform-independent C code and compile it into .so files". Michael |
|
From: Michael B. <mb...@fe...> - 2005-03-22 15:23:51
|
On Mon, 21 Mar 2005, H. Peter Anvin wrote: > > Only one which isn't yet in production. The I/O BAR is in the current > > generation of the product in case it was needed to support PXE. > > Since they used Etherboot for the PXE stack, (and consequently weren't > > restricted to portio, and could get away with essentially reusing the > > Linux driver), the I/O bar is completely unused and will be removed in > > the next rev. > That also means, presumably, they're stuck with the Etherboot subset of > PXE, which would make me guess this is a special-purpose device. From > the looks of it, most vendors are keeping or even adding the I/O bar, > all while cursing the managers at Intel who apparently let a consultant > come up with the spec without any kind of adult supervision. Yes; it's a special-purpose device. PXE is just there as a marketing checkbox. There are some likely applications for this NIC that benefit from having an Etherboot driver and consequently being usable with LinuxBIOS. I can't say much more without violating the NDA. Michael |
|
From: H. P. A. <hp...@zy...> - 2005-03-21 15:11:00
|
Michael Brown wrote: >>Dumb question for Michael: do you happen to know of any cards which are >>known to only have memory apertures? I'd like to disect their factory >>PXE stacks and see how they solve the whole dilemma. That probably sets >>the _de_facto_ standard for what happens next. > > Only one which isn't yet in production. The I/O BAR is in the current > generation of the product in case it was needed to support PXE. Since > they used Etherboot for the PXE stack, (and consequently weren't > restricted to portio, and could get away with essentially reusing the > Linux driver), the I/O bar is completely unused and will be removed in the > next rev. > That also means, presumably, they're stuck with the Etherboot subset of PXE, which would make me guess this is a special-purpose device. From the looks of it, most vendors are keeping or even adding the I/O bar, all while cursing the managers at Intel who apparently let a consultant come up with the spec without any kind of adult supervision. -hpa |
|
From: Michael B. <mb...@fe...> - 2005-03-19 17:04:49
|
> Dumb question for Michael: do you happen to know of any cards which are > known to only have memory apertures? I'd like to disect their factory > PXE stacks and see how they solve the whole dilemma. That probably sets > the _de_facto_ standard for what happens next. Only one which isn't yet in production. The I/O BAR is in the current generation of the product in case it was needed to support PXE. Since they used Etherboot for the PXE stack, (and consequently weren't restricted to portio, and could get away with essentially reusing the Linux driver), the I/O bar is completely unused and will be removed in the next rev. Michael |
|
From: H. P. A. <hp...@zy...> - 2005-03-17 09:45:07
|
Dumb question for Michael: do you happen to know of any cards which are known to only have memory apertures? I'd like to disect their factory PXE stacks and see how they solve the whole dilemma. That probably sets the _de_facto_ standard for what happens next. -hpa |
|
From: H. P. A. <hp...@zy...> - 2005-03-17 09:38:03
|
Michael Brown wrote: > On Wed, 9 Mar 2005, H. Peter Anvin wrote: > >>>>Right. With appropriate descriptors set up they could work in 16-bit >>>>protected mode, though, for supporting the PM entry point. >>> >>>They could, but that wouldn't help the code when it's called via the RM >>>entry point. >> >>Supporting V86 mode entry via the RM entry point is probably futile. > > > If we support "pure real mode" operation, then it should just work in V86 > mode. > For what it's worth, I talked some about this with a couple of guys from Intel, and they said they ended up adding I/O mapped access to the e1000 specifically because PXE was buggered. -hpa |
|
From: Michael B. <mb...@fe...> - 2005-03-10 14:15:53
|
On Wed, 9 Mar 2005, H. Peter Anvin wrote: > >>Right. With appropriate descriptors set up they could work in 16-bit > >>protected mode, though, for supporting the PM entry point. > > They could, but that wouldn't help the code when it's called via the RM > > entry point. > Supporting V86 mode entry via the RM entry point is probably futile. If we support "pure real mode" operation, then it should just work in V86 mode. > > Also, the cards we're talking about are quite likely to need more than > > 64kB of data buffers, in which case we can't fit it into the strict UNDI > > model (64kB code + 64kB data+bss+stack, all in base memory) anyway. We > > have to accept that the PXE specification is actually incompatible with > > some network cards, and plan to carefully violate the spec. > Indeed, although the !PXE structure clearly is designed to accommodate > additional descriptors. But only two of these can simultaneously satisfy the requirements of being located in base memory and being used by the UNDI portion, as I've pointed out before. > (Note: for a lot of cards which *can* have > 64K data buffers, generally > don't *have* to. For example, a device with a 1024-entry descriptor > ring can be filled with only a few entries, on the assumption that > performance is secondary in our application.) The most recent gigabit driver I wrote ended up dropping packets (and so waiting for TFTP retransmits) with fewer than 66kB of buffers. Though it's true that we're not after high performance, having the occasional multi-second pause is not acceptable. Michael |
|
From: H. P. A. <hp...@zy...> - 2005-03-10 01:52:47
|
Michael Brown wrote: >> >>Right. With appropriate descriptors set up they could work in 16-bit >>protected mode, though, for supporting the PM entry point. > > They could, but that wouldn't help the code when it's called via the RM > entry point. > Supporting V86 mode entry via the RM entry point is probably futile. > Also, the cards we're talking about are quite likely to need more than > 64kB of data buffers, in which case we can't fit it into the strict UNDI > model (64kB code + 64kB data+bss+stack, all in base memory) anyway. We > have to accept that the PXE specification is actually incompatible with > some network cards, and plan to carefully violate the spec. Indeed, although the !PXE structure clearly is designed to accommodate additional descriptors. (Note: for a lot of cards which *can* have > 64K data buffers, generally don't *have* to. For example, a device with a 1024-entry descriptor ring can be filled with only a few entries, on the assumption that performance is secondary in our application.) -hpa |
|
From: Michael B. <mb...@fe...> - 2005-03-10 00:32:04
|
On Wed, 9 Mar 2005, H. Peter Anvin wrote: > > The MMIO/portio issue is driver-specific. Drivers that use MMIO will use > > readX/writeX. Since these functions won't work under -DKEEP_IT_REAL, we > > could just make the compiler throw an error. > Right. With appropriate descriptors set up they could work in 16-bit > protected mode, though, for supporting the PM entry point. They could, but that wouldn't help the code when it's called via the RM entry point. Also, the cards we're talking about are quite likely to need more than 64kB of data buffers, in which case we can't fit it into the strict UNDI model (64kB code + 64kB data+bss+stack, all in base memory) anyway. We have to accept that the PXE specification is actually incompatible with some network cards, and plan to carefully violate the spec. > I have no idea how the FreeBSD people expect their things to work with > MMIO-based cards. I suspect they just tried a small number of cards, it > worked, and they thought they were clever. I imagine so, though I'd gladly be proved wrong. Michael |
|
From: H. P. A. <hp...@zy...> - 2005-03-09 19:20:57
|
Michael Brown wrote: > > The MMIO/portio issue is driver-specific. Drivers that use MMIO will use > readX/writeX. Since these functions won't work under -DKEEP_IT_REAL, we > could just make the compiler throw an error. > Right. With appropriate descriptors set up they could work in 16-bit protected mode, though, for supporting the PM entry point. I have no idea how the FreeBSD people expect their things to work with MMIO-based cards. I suspect they just tried a small number of cards, it worked, and they thought they were clever. -hpa |
|
From: Michael B. <mb...@fe...> - 2005-03-09 14:01:49
|
On Tue, 8 Mar 2005, Michael Brown wrote: > Code is imported. There's a script, ebsync.pl, that can be used at any > time to sync across current Etherboot CVS to the EB_CVS branch of NILO > CVS. This does *not* automatically merge changes from the EB_CVS branch > into the NILO HEAD branch; that part must currently be done manually > using something along the lines of > > cvs update -jEB_LAST_SYNC -jEB_CVS > > then resolving all the conflicts, checking in the changes and reassigning > the tag EB_LAST_SYNC with > > cvs tag -r EB_CVS EB_LAST_SYNC > > I'll probably automate this process the first time I need to use it. Quick but important notification: I have fixed a bug that meant that the EB_CVS branch ended up as the default branch for NILO. If you have a CVS checkout of NILO, please update it to HEAD with cvs update -A otherwise your version numbers will get screwed up and you'll have to spend ages fixing them. Michael |
|
From: Michael B. <mb...@fe...> - 2005-03-09 12:33:06
|
On Tue, 8 Mar 2005, Marty Connor wrote: > > Not all drivers *can* be modified to use in/out (i.e. portio) as > > opposed to read/write (i.e. MMIO.) It's a function of the hardware. > So does this mean that any loader being used with these cards would have > to use PM to have any chance of working? In practical terms, yes. > > Perhaps the first thing to do is to print a legible message if > > something tries to call the PM entry point and/or invoke the RM entry > > point in V86 mode. Something asking them to report what they're > > trying to do, and we could evaluate the importance at that point. > Is this what we think FreeBSD is doing when talking to Etherboot? Yes. To be fair: I haven't *tried* FreeBSD's PXE loader in bochs to see what it does, but I can see from the code that it uses V86 mode and people have reported that it does, indeed, fail. > I get the feeling that success may depend on: > - Whether there card uses MMIO or portio > - Whether the loader they intend to use tries to: > "call the PM entry point and/or invoke the RM entry point in V86 > mode" > > Now, since most people probably don't know the answer to either of > these questions, I think we might have to recommend options based on > either scenarios, or most-likely-to-work options (e.g. "Try this first > unless you are planning to boot FREEBSD" or "If you are planning to use > RIS, check this box"). The MMIO/portio issue is driver-specific. Drivers that use MMIO will use readX/writeX. Since these functions won't work under -DKEEP_IT_REAL, we could just make the compiler throw an error. Michael |
|
From: Marty C. <md...@en...> - 2005-03-09 02:12:44
|
On Mar 8, 2005, at 1:10 PM, H. Peter Anvin wrote:
> Not all drivers *can* be modified to use in/out (i.e. portio) as
> opposed to read/write (i.e. MMIO.) It's a function of the hardware.
So does this mean that any loader being used with these cards would
have to use PM to have any chance of working?
> Perhaps the first thing to do is to print a legible message if
> something tries to call the PM entry point and/or invoke the RM entry
> point in V86 mode. Something asking them to report what they're
> trying to do, and we could evaluate the importance at that point.
Is this what we think FreeBSD is doing when talking to Etherboot?
I hope these questions are at least somewhat sensible. I am trying to
anticipate from the perspective of someone trying to figure out what
options they need to select to have their PXE ROM image work.
I get the feeling that success may depend on:
- Whether there card uses MMIO or portio
- Whether the loader they intend to use tries to:
"call the PM entry point and/or invoke the RM entry point in V86
mode"
Now, since most people probably don't know the answer to either of
these questions, I think we might have to recommend options based on
either scenarios, or most-likely-to-work options (e.g. "Try this first
unless you are planning to boot FREEBSD" or "If you are planning to use
RIS, check this box").
I really don't really like the idea of doing this, but until we figure
out what the compatibility gamut of various configurations is, we might
need to.
I have a pristine Intel EEPRO100B and a 3Com 3C905C which I use for
testing. Both have PXE implementations in on-board flash, and are
useful for grossly determining compatibility of stuff, since I suspect
these are two of the more common stacks available.
Is this analysis reasonable? What other factors might we need to
consider when deciding whether people should turn on various
compatibility options. Hopefully if we work with free software PXE
loader developers they will help us support something reasonable.
To work with RIS but not FreeBSD would be a little, umm, strange :)
Marty
|
|
From: H. P. A. <hp...@zy...> - 2005-03-08 21:13:38
|
Michael Brown wrote: > > Sadly not. Dedicated I/O space is a legacy x86 feature; in order to work > in non-x86 machines, hardware needs to support memory-mapped I/O. Since > x86 can also use memory-mapped I/O (unless, of course, you're stuck in > real-mode), there's no real benefit to supporting it. Hence, newer cards > are likely to be MMIO-only. > There are two more reasons for this: a) IOIO is only available in 256-byte chunks due to ISA address space braindamage; b) On x86, IOIO is *SLOW* due mostly to the SMI virtualization support. -hpa |
|
From: Michael B. <mb...@fe...> - 2005-03-08 19:09:07
|
On Tue, 8 Mar 2005, Marty Connor wrote: > >> Drivers we can fix. Older ones are probably simpler anyway, and > >> newer ones probably use inX/outX already. > > Not all drivers *can* be modified to use in/out (i.e. portio) as > > opposed to read/write (i.e. MMIO.) It's a function of the hardware. > Drat. Hopefully these represent old, rarer, and hopefully less-used > cards. Sadly not. Dedicated I/O space is a legacy x86 feature; in order to work in non-x86 machines, hardware needs to support memory-mapped I/O. Since x86 can also use memory-mapped I/O (unless, of course, you're stuck in real-mode), there's no real benefit to supporting it. Hence, newer cards are likely to be MMIO-only. > In order to move things along, I've asked Michael to import code into > the nilo tree whenever he is ready, setting a tag in the Etherboot tree. Code is imported. There's a script, ebsync.pl, that can be used at any time to sync across current Etherboot CVS to the EB_CVS branch of NILO CVS. This does *not* automatically merge changes from the EB_CVS branch into the NILO HEAD branch; that part must currently be done manually using something along the lines of cvs update -jEB_LAST_SYNC -jEB_CVS then resolving all the conflicts, checking in the changes and reassigning the tag EB_LAST_SYNC with cvs tag -r EB_CVS EB_LAST_SYNC I'll probably automate this process the first time I need to use it. Michael |
|
From: H. P. A. <hp...@zy...> - 2005-03-08 18:55:25
|
Marty Connor wrote: > On Mar 8, 2005, at 1:10 PM, H. Peter Anvin wrote: > >> Marty Connor wrote: >> >>> Drivers we can fix. Older ones are probably simpler anyway, and >>> newer ones probably use inX/outX already. >> >> Not all drivers *can* be modified to use in/out (i.e. portio) as >> opposed to read/write (i.e. MMIO.) It's a function of the hardware. > > Drat. Hopefully these represent old, rarer, and hopefully less-used cards. > On the contrary. They represent mostly newer cards. -hpa |
|
From: Marty C. <md...@en...> - 2005-03-08 18:47:33
|
On Mar 8, 2005, at 1:10 PM, H. Peter Anvin wrote: > Marty Connor wrote: >> Drivers we can fix. Older ones are probably simpler anyway, and >> newer ones probably use inX/outX already. > Not all drivers *can* be modified to use in/out (i.e. portio) as > opposed to read/write (i.e. MMIO.) It's a function of the hardware. Drat. Hopefully these represent old, rarer, and hopefully less-used cards. >> If you say so :) Sounds like a somewhat fragile hack, but sometimes >> that's all we've got that gets the job done. > It's not fragile, you just have to put the proper entry point code at > both the 16- and the 32-bit entry point. I have faith. I hope someday to have better understanding. :) >>> I appreciate that all this is quite involved, so I'm going to >>> attempt a >>> brief summary: >>> o We can support genuine real-mode operation, V86-mode operation >>> and the >>> PXE-specified 16-bit protected-mode operation. >>> o Doing this gives us compatibility with FreeBSD and with any >>> as-yet-undiscovered program that uses the PM entry point. >> Are things that do this "broken" or "violating the spec" somehow? >> What I'm getting at, is how hard we should try to get people to fix >> their code to work with ours? I can already hear comments like "well >> it works with my Intel boot ROM on my eepro100... and also with my >> 3C905 MBA Boot ROM..." It would be nice to have some good reasons >> why they should fix it. > Perhaps the first thing to do is to print a legible message if > something tries to call the PM entry point and/or invoke the RM entry > point in V86 mode. Something asking them to report what they're > trying to do, and we could evaluate the importance at that point. Yes, hopefully we can use rom-o-matic.net to encourage people to report their experiences in such a way that they are useful. Hmmm, a convenient web form offering a free iPod mini and a private introduction to the Coors twins comes to mind, but maybe we can appeal to their more altruistic natures :) This is exciting conversation (read: I need to get out more :) I can't wait until we have something to document and test. In order to move things along, I've asked Michael to import code into the nilo tree whenever he is ready, setting a tag in the Etherboot tree. I'm expecting there will be a fair bit of heavy demolition once the low-level interface issues are in place. Hopefully at that point more people will join in the discussion, and maybe help with the hacking. To those quietly lurking: Don't be shy if you have an idea or opinion -- share it! Marty |
|
From: Michael B. <mb...@fe...> - 2005-03-08 18:27:07
|
On Tue, 8 Mar 2005, H. Peter Anvin wrote: > > Even setting up descriptors to cover the high memory used by a card > > doesn't help us in the case of being called in V86 mode, in which case > > we really are constrained to base memory only. > Indeed. I don't see any reason to believe this is a permitted use of > these interfaces, though, on the other hand it clearly works with stock > PXE systems on FreeBSD. I wonder if they require the INT 15h mover API > to exist. I wondered that. They'd presumably have to implement their own version, since the BIOS version would presumably try to do a mode switch, which is forbidden. > > I think that providing a compile-time option to generate 16-bit code > > designed to run in real mode (as I described previously) is viable. > > There are restrictions: most significantly, drivers that use > > memory-mapped I/O (readX/writeX) rather than inX/outX probably won't > > work. Some drivers (e.g. prism2_pci) can probably be modified to use > > inX/outX. Some drivers (e.g. depca.c) seem to use readX/writeX to > > access normal in-memory data structures; I'm not quite sure what's > > going on here. > Well, you could use either the INT 15h AH=87h API, or some clever > descriptor mangling (for the case of 16-bit PM) to support those > operations. It's an option, but for now I think I'll adopt the position that if we assume we're allowed to access high memory, we may as well assume that we can switch to pmode and do it the easy way. > > The PXE protected-mode entry point is actually a hack designed to > > allow 16-bit real-mode code to be called in protected mode. Code > > capable of being run in real mode will work when called via the > > protected-mode entry point, provided that we load %ds etc from the > > !PXE structure on entry. Since our code is 32-bit code coerced into > > 16-bit opcodes using .code16gcc, we're already using %esp as the stack > > register, so supporting the EntryPointESP is a no-op, and we just need > > some glue logic to zero the high word of %esp in order to support the > > (mandatory) EntryPointSP with the same code. > > Actually, you'll have to reload the %ss selector; the default sizes of > stack operations are dependent on the stack segment descriptor. The > easiest way to deal with this is to switch stacks: Good point; I'd forgotten that it depended on %ss. Thinking more about it, we need to have %ss == %ds when we're executing our code anyway, which means we'd have to do a stack switch analogous to the pmode/rmode switch I've currently got implemented. One subtlety is that we then have to make sure that we *don't* pass control to the NBP with our internal stack, otherwise we get screwed at re-entry. > > Since protected-mode operation is so poorly-defined, I strongly > > suspect that no existing programs ever attempt to call a PXE stack in > > protected mode. > > People *do* call the UNDI stack in protected mode. I am not sure if > there is another entry point to the UNDI stack, or how widely it's used. Who does? Michael |
|
From: H. P. A. <hp...@zy...> - 2005-03-08 18:11:06
|
Marty Connor wrote: > > Drivers we can fix. Older ones are probably simpler anyway, and newer > ones probably use inX/outX already. > Not all drivers *can* be modified to use in/out (i.e. portio) as opposed to read/write (i.e. MMIO.) It's a function of the hardware. > > If you say so :) Sounds like a somewhat fragile hack, but sometimes > that's all we've got that gets the job done. > It's not fragile, you just have to put the proper entry point code at both the 16- and the 32-bit entry point. >> I appreciate that all this is quite involved, so I'm going to attempt a >> brief summary: >> >> o We can support genuine real-mode operation, V86-mode operation and the >> PXE-specified 16-bit protected-mode operation. >> o Doing this gives us compatibility with FreeBSD and with any >> as-yet-undiscovered program that uses the PM entry point. > > Are things that do this "broken" or "violating the spec" somehow? > > What I'm getting at, is how hard we should try to get people to fix > their code to work with ours? I can already hear comments like "well it > works with my Intel boot ROM on my eepro100... and also with my 3C905 > MBA Boot ROM..." It would be nice to have some good reasons why they > should fix it. Perhaps the first thing to do is to print a legible message if something tries to call the PM entry point and/or invoke the RM entry point in V86 mode. Something asking them to report what they're trying to do, and we could evaluate the importance at that point. -hpa |
|
From: H. P. A. <hp...@zy...> - 2005-03-08 18:09:00
|
Michael Brown wrote: > > Even setting up descriptors to cover the high memory used by a card > doesn't help us in the case of being called in V86 mode, in which case we > really are constrained to base memory only. > Indeed. I don't see any reason to believe this is a permitted use of these interfaces, though, on the other hand it clearly works with stock PXE systems on FreeBSD. I wonder if they require the INT 15h mover API to exist. > > I think that providing a compile-time option to generate 16-bit code > designed to run in real mode (as I described previously) is viable. > There are restrictions: most significantly, drivers that use memory-mapped > I/O (readX/writeX) rather than inX/outX probably won't work. Some drivers > (e.g. prism2_pci) can probably be modified to use inX/outX. Some drivers > (e.g. depca.c) seem to use readX/writeX to access normal in-memory data > structures; I'm not quite sure what's going on here. > Well, you could use either the INT 15h AH=87h API, or some clever descriptor mangling (for the case of 16-bit PM) to support those operations. > Within these restrictions, it should be perfectly possible to have a > compile-time option to generate 16-bit real-mode code. > > This code would then run properly in V86 mode, since it would never > attempt a mode transition. > > The PXE protected-mode entry point is actually a hack designed to allow > 16-bit real-mode code to be called in protected mode. Code capable of > being run in real mode will work when called via the protected-mode entry > point, provided that we load %ds etc from the !PXE structure on entry. > Since our code is 32-bit code coerced into 16-bit opcodes using > .code16gcc, we're already using %esp as the stack register, so supporting > the EntryPointESP is a no-op, and we just need some glue logic to zero the > high word of %esp in order to support the (mandatory) EntryPointSP with > the same code. Actually, you'll have to reload the %ss selector; the default sizes of stack operations are dependent on the stack segment descriptor. The easiest way to deal with this is to switch stacks: EntryPointESP: pushl %ds pushl %es movw %cs:DataSegment,%ds movw DataSegment,%es movl %esp,SavedStack movw %ss,SavedStack+4 lssl MyStack,%esp call common_code lssl SavedStack,%esp popl %es popl %ds ret > It's worth noting that the specification is incomplete with regard to > protected-mode operation. In particular, it defines a "StatusCallout" > field that must contain a far pointer to a routine to be used for printing > status messages when in protected-mode (since BIOS interrupts will work > only in real mode), but nothing I can find anywhere in the document > describes what the parameters, actions or return value of this > StatusCallout routine actually are. I don't think it has any; it's main purpose seems to be to suppress the default "print dots and watch for the [Esc] key" behaviour of newer PXE ROMs. > Since protected-mode operation is so poorly-defined, I strongly suspect > that no existing programs ever attempt to call a PXE stack in protected > mode. People *do* call the UNDI stack in protected mode. I am not sure if there is another entry point to the UNDI stack, or how widely it's used. > We could always cheat and simply refuse to print status messages to the > video console unless StatusCallout==0 (indicating "use internal printing > routines"). That would probably be as good a cheat as any. > I appreciate that all this is quite involved, so I'm going to attempt a > brief summary: > > o We can support genuine real-mode operation, V86-mode operation and the > PXE-specified 16-bit protected-mode operation. > > o Doing this gives us compatibility with FreeBSD and with any > as-yet-undiscovered program that uses the PM entry point. > > o Not all drivers will work when used in this way. This is a fundamental > limitation and cannot be worked around. > > o We can do this simultaneously with supporting our normal 32-bit > protected-mode operation. (Simultaneously in the sense of "with the > same source code, using compile-time options", rather than "with the > same binary ROM image"). The good part of it is that it lets us be a "full-featured" PXE stack, as opposed to a perennial second-class citizen. Not that the first-class citizens aren't shock-full of bugs and have tons of missing features (like menu support, which is broken in a fair number of the PXE stacks I've tried, just like CD-ROM booting menu support is.) -hpa |
|
From: Marty C. <md...@en...> - 2005-03-08 16:57:56
|
On Mar 8, 2005, at 10:00 AM, Michael Brown wrote: > Too many e-mail threads for me, so I've grouped everything I'm > replying to > in one message. There's an executive summary at the end. Thanks for this. I find it helpful and illuminating. > Currently I know of no clients that attempt to use the protected-mode > entry point, and only FreeBSD that uses the real-mode entry point in > V86 > mode. I think the FreeBSD people might well fix this (assuming it wouldn't break other stuff) if they were informed of what to do. I know this came up on the Etherboot list. Do you recall if any FreeBSD folks spoke up? > Even setting up descriptors to cover the high memory used by a card > doesn't help us in the case of being called in V86 mode, in which case > we > really are constrained to base memory only. This must refer to the FreeBSD folks again. I wonder if they made whatever the proper call would be (assuming this way of calling is a bug on their part) if it is likely to break anything for them. Any idea? > I think that providing a compile-time option to generate 16-bit code > designed to run in real mode (as I described previously) is viable. > There are restrictions: most significantly, drivers that use > memory-mapped > I/O (readX/writeX) rather than inX/outX probably won't work. Some > drivers > (e.g. prism2_pci) can probably be modified to use inX/outX. Some > drivers > (e.g. depca.c) seem to use readX/writeX to access normal in-memory data > structures; I'm not quite sure what's going on here. Drivers we can fix. Older ones are probably simpler anyway, and newer ones probably use inX/outX already. > Within these restrictions, it should be perfectly possible to have a > compile-time option to generate 16-bit real-mode code. > This code would then run properly in V86 mode, since it would never > attempt a mode transition. Let's hope the gcc compiler continues to support this. I guess there's no other really viable option at the moment. > The PXE protected-mode entry point is actually a hack designed to allow > 16-bit real-mode code to be called in protected mode. Code capable of > being run in real mode will work when called via the protected-mode > entry > point, provided that we load %ds etc from the !PXE structure on entry. > Since our code is 32-bit code coerced into 16-bit opcodes using > ..code16gcc, we're already using %esp as the stack register, so > supporting > the EntryPointESP is a no-op, and we just need some glue logic to zero > the > high word of %esp in order to support the (mandatory) EntryPointSP with > the same code. If you say so :) Sounds like a somewhat fragile hack, but sometimes that's all we've got that gets the job done. > I appreciate that all this is quite involved, so I'm going to attempt a > brief summary: > > o We can support genuine real-mode operation, V86-mode operation and > the > PXE-specified 16-bit protected-mode operation. > o Doing this gives us compatibility with FreeBSD and with any > as-yet-undiscovered program that uses the PM entry point. Are things that do this "broken" or "violating the spec" somehow? What I'm getting at, is how hard we should try to get people to fix their code to work with ours? I can already hear comments like "well it works with my Intel boot ROM on my eepro100... and also with my 3C905 MBA Boot ROM..." It would be nice to have some good reasons why they should fix it. The pain of supporting undesirable behaviors is, err, tedious at times. > o Not all drivers will work when used in this way. This is a > fundamental > limitation and cannot be worked around. This makes sense, and hopefully we can document where this is likely (or known) to occur. > o We can do this simultaneously with supporting our normal 32-bit > protected-mode operation. (Simultaneously in the sense of "with the > same source code, using compile-time options", rather than "with the > same binary ROM image"). Are we really talking about two different parts/products here? Are we talking about one that works with most loaders, and another that works with a few broken legacy loaders that didn't follow the spec very well? If that is what we have to face, then so be it. I just want to know how to explain to people on rom-o-matic.net and in our documentation which ROM images they probably want. This may only be something we can figure out empirically, like so many things. It's unsatisfying to have to have the compile-time option, such that there will be NILO ROMs out there that will be more compatible with legacy loaders, and other NILO ROMs that will be required to use newer cards. So does this mean that FreeBSD attempting to use a gigabit card may simply fail? That would suck. Of course if this happens we could explain to the FreeBSD people why this is, and they could fix their code. It would be nice if the fix also worked for the legacy ROMs as well. As ugly as this may be at a low-level, I think it's worth the attempt to make a free PXE stack, and I have a feeling we'll learn a lot from the experience. Sounds like it's time to import some code into that lonely nilo cvs directory, eh, Michael? :) Marty |
|
From: Michael B. <mb...@fe...> - 2005-03-08 15:01:41
|
Too many e-mail threads for me, so I've grouped everything I'm replying to in one message. There's an executive summary at the end. On Mon, 7 Mar 2005, Marty Connor wrote: > This real mode entry point seems to be and important issue. > > Could someone please explain what bad things might happen if we don't > get his right? (what might break, and how?). If we leave things as they currently are, then we're stuck with two main restrictions: 1. We cannot support the protected-mode entry point, for reasons previously described in excruciating detail. 2. We cannot support clients (NBPs, external BC etc.) that calls the real-mode entry point in V86 mode rather than real mode. Currently I know of no clients that attempt to use the protected-mode entry point, and only FreeBSD that uses the real-mode entry point in V86 mode. On Tue, 8 Mar 2005, Marty Connor wrote: > Perhaps some useful questions to ask are: > > - In what ways and to what degrees is our PXE implementation > compliant with the PXE specification v2.1? > > - Where do we diverge from the specification, and why? I can't give a definitive answer without going through the spec in detail, which I don't have time to do. Where we diverge, it's generally either because complying would have meant major structural changes to Etherboot (e.g. splitting BC and UNDI components) or because we have no known users of a particular feature and no way to test it (e.g. TFTM). In other places, we diverge in order to provide additional robustness: for example, you can call our initialisation routines in virtually any order, or even skip them completely and have the card automatically be initialised the first time a packet is transmitted or received. > - How and in what ways is our UNDI interface interoperable with PXE > loaders? The only known client that uses the UNDI interface is Etherboot itself, using its UNDI driver. It works with this client, though I accept that since I wrote both bits of code, that doesn't necessarily say anything about compliance with the spec. :) On Tue, 8 Mar 2005, H. Peter Anvin wrote: > > Indeed; according to *that* page we are allowed an arbitrary number of > > segments. However, according to page 92 (PDF page 94), the UNDI > > loader is allowed only one code and one data segment. This is a hard > > limit, since there are only two 16-bit fields in the UNDI loader > > parameter structure to specify the segments. Just one of the many > > entertaining inconsistencies in PXE. > > Page 92 only applies if you're building a modular ROM (BC+BUSD+UNDI). We claim to implement an UNDI ROM, so that clients can load us from ROM and use our UNDI interface. (OK, so we actually drag in our base code as well, but it never gets called.) Sufficiently sophisticated BIOSes should be able to load our UNDI driver and use us to communicate over the network. I have never encountered a BIOS that actually does this, but it's meant to be possible. If we go over two segments, then we lose the ability to support this. On Tue, 8 Mar 2005, H. Peter Anvin wrote: > A major problem is that the PXE spec appears to have been designed by an > infinite number of monkeys on LSD. After reading it over and over again > I'm convinced that Michael is right, there is no sane way to make it > compatible with 32-bit protected mode. This means, pretty much, that > the only possible way to make it compliant with the spec is to use > 16-bit code everywhere, and set up descriptors to cover the high memory > that whatever card uses. This is a painful way to program, and means > diverging drivers from Linux and Etherboot, which is a major lose. Even setting up descriptors to cover the high memory used by a card doesn't help us in the case of being called in V86 mode, in which case we really are constrained to base memory only. I think that providing a compile-time option to generate 16-bit code designed to run in real mode (as I described previously) is viable. There are restrictions: most significantly, drivers that use memory-mapped I/O (readX/writeX) rather than inX/outX probably won't work. Some drivers (e.g. prism2_pci) can probably be modified to use inX/outX. Some drivers (e.g. depca.c) seem to use readX/writeX to access normal in-memory data structures; I'm not quite sure what's going on here. Within these restrictions, it should be perfectly possible to have a compile-time option to generate 16-bit real-mode code. This code would then run properly in V86 mode, since it would never attempt a mode transition. The PXE protected-mode entry point is actually a hack designed to allow 16-bit real-mode code to be called in protected mode. Code capable of being run in real mode will work when called via the protected-mode entry point, provided that we load %ds etc from the !PXE structure on entry. Since our code is 32-bit code coerced into 16-bit opcodes using .code16gcc, we're already using %esp as the stack register, so supporting the EntryPointESP is a no-op, and we just need some glue logic to zero the high word of %esp in order to support the (mandatory) EntryPointSP with the same code. It's worth noting that the specification is incomplete with regard to protected-mode operation. In particular, it defines a "StatusCallout" field that must contain a far pointer to a routine to be used for printing status messages when in protected-mode (since BIOS interrupts will work only in real mode), but nothing I can find anywhere in the document describes what the parameters, actions or return value of this StatusCallout routine actually are. Since protected-mode operation is so poorly-defined, I strongly suspect that no existing programs ever attempt to call a PXE stack in protected mode. We could always cheat and simply refuse to print status messages to the video console unless StatusCallout==0 (indicating "use internal printing routines"). I appreciate that all this is quite involved, so I'm going to attempt a brief summary: o We can support genuine real-mode operation, V86-mode operation and the PXE-specified 16-bit protected-mode operation. o Doing this gives us compatibility with FreeBSD and with any as-yet-undiscovered program that uses the PM entry point. o Not all drivers will work when used in this way. This is a fundamental limitation and cannot be worked around. o We can do this simultaneously with supporting our normal 32-bit protected-mode operation. (Simultaneously in the sense of "with the same source code, using compile-time options", rather than "with the same binary ROM image"). Michael |
