etherboot-developers — Discussion of developer issues

Re: [Etherboot-developers] Q: GPL license clarification.

[<ke...@us...>]

>> Such a clarification seems reasonable to me.
>
>I agree.  If world domination is a goal, bios vendors need to be comfortable 
>that they are not going to accidentally GPL their code.
> 
>> The GPL does place certain other responsibilities on a vendor wishing 
>> to distribute software that has been placed under the GPL. The 
>> provision about making the source code available comes to mind. It is 
>> not difficult to fulfill, but adds some overhead to using GPL software.
>
>Really, this is the most important point.  If a vendor makes use of etherboot
> with some modifications to suit their needs and does not make those changes 
>available, there will be problems.  As long as one can download the source an
>d reproduce the compiled eb rom that is being distributed the vendor should b
>e OK.

I'm in violent agreement with this thread. As it happens, the docs need
updating for 5.2. If someone would like to draft the appropriate
paragraphs and run it past this list first I'll be happy to insert it
into the 5.2 user guide. If you're game to edit Docbook please feel free
to do that too.

Essentially the words we need are:

Clarification of user rights and responsibilities conferred by the GPL
as applied to Etherboot, to wit: Etherboot is a separate work from the
BIOS; and that if they modify Etherboot, they have to distribute source
with binary (and gain cred with the OSS world as a result). And a
disclaimer that they should seek legal advice of their own re the GPL
since WANLs.

Re: [Etherboot-developers] Memory allocation

[<ebi...@ln...>]

> > > > > O.k.  One issue on my wishlist is to have a stub that loads etherboot
> > > > > into high memory (>1MB) from the rom initialization code, and then just
> > > > > leaves a very small stub in real mode.  Given that I have seen several
> > > > > BIOS's start acting erratically because of their real mode storage for
> > > > > options roms was used up.  This could make etherboot more reliable.
> > > > Could be interesting; PMM is potentially in charge of high memory at the 
> > > > time the initialisation code is called (assuming the BIOS supports PMM), 
> > > > but anything it allocates is lost before the system starts booting.  I'm 
> > > > not sure how you'd work around that problem.
> > > Me either.  The PXE spec seems to require it to some extent.  But just
> > > skimming it I don't see how.
> > 
> > It's not very clear.  PXE spec seems to state that the PXE ROM should move 
> > itself into upper memory using PMM, implying that it stays there ready for 
> > when the system boots.  However, PMM spec states that this is impossible; 
> > anything allocated by PMM is for POST-time only and will be destroyed at 
> > boot time.
> > 
> > To make it even more confusing, a disassembly of an Intel PXE ROM shows 
> > debug messages that mention PMM, but AFAICT the code never makes any PMM 
> > API calls.  (At least, the PMM signature, "$PMM", doesn't appear anywhere 
> > within the ROM and this is the only way to locate PMM services within the 
> > BIOS.)
> 
> Interesting...
> 
> Perhaps the guys who wrote the pxe spec were confused.

I have just had an interesting conversation and I now have
a reasonable explanation why BIOS code has problems with
protected mode.  The problem is that code run in vm86 mode frequently
makes use of BIOS services.  And if the BIOS services are implemented 
in protected mode vm86 mode does not work.

Which would explain why there are no BIOS interfaces for handling
this problem.  

For the case of making etherboot UMB friendly I would suggest we look
at copying the compressing image to low memory, in loader.S and
reserving that memory as appropriate.

Eric

Re: Re: [Etherboot-developers] Q: GPL license clarification.

[Vasil V. <vas...@sy...>]

> > Now where was I... oh yes, debugging PXE chaining...
> >
> I am getting a laptop without a floppy drive so i may have some call for this...

I had the same case with Windows preinstalled. I wanted to keep windows, but
to repartition I had to use parted which runs under Linux. So, I used PXE
chaining to get an image with parted, repartitioned the disk and here you go.
Then installed using kickstart, again netbooted. I will have to create a mini
HOWTO on this.

Vasil

Re: Re: [Etherboot-developers] Q: GPL license clarification.

[Timothy L. <tl...@ro...>]

> Such a clarification seems reasonable to me.

I agree.  If world domination is a goal, bios vendors need to be comfortable that they are not going to accidentally GPL their code.
 
> The GPL does place certain other responsibilities on a vendor wishing 
> to distribute software that has been placed under the GPL. The 
> provision about making the source code available comes to mind. It is 
> not difficult to fulfill, but adds some overhead to using GPL software.

Really, this is the most important point.  If a vendor makes use of etherboot with some modifications to suit their needs and does not make those changes available, there will be problems.  As long as one can download the source and reproduce the compiled eb rom that is being distributed the vendor should be OK.
[snip]

> Now, one could argue that Etherboot, because it can stand alone and run 
> on bare metal is separable, and thus could be inserted into a BIOS chip 
> as a storage medium, and because of the mechanism the BIOS uses to scan 
> for extension ROMs, remains a separate program, and thus does not 
> require the BIOS vendor to distribute the entire BIOS under the GPL.
> 
> Because the BIOS extension mechanism was created to allow vendors to 
> use code that may in fact be unknown to the BIOS manufacturer when the 
> BIOS is produced, as long as they follow the PNP BIOS specification, I 
> think one could reasonably argue that Etherboot fits within the spirit 
> of the GPL with regard to aggregation.

If the vendor's implentation follows this standard it would fit nicely.  Essentially this is what happens when people add eb to the bios or to the NIC's rom.  
 
> In fairness, I must admit to seeing the other side of the argument, 
> that hinges on the phrase "combined so that they become effectively two 
> parts of one program". the BIOS extension method does, at run-time, 
> basically make the two pieces of code virtually a single unit. It's 
> just that when you're at bare-metal, _any_ use of other code would 
> probably look like that because it's the most efficient way to do it.

I disagree.  It has the appearance of a single unit but it is not.  However, one of the things that the vendor's should consider is making it possible to upgrade eb separately from the rest of the bios.  It would help ensure that the appearance reflects reality.

> So, I think we are safe (and reasonable) in our interpretation. If 
> someone makes the argument that being a BIOS extension make us 
> "effectively two parts of one program" at run-time there would be a lot 
> of very unhappy vendors.

And at least one unhappy developer;-)  I for one would like to see etherboot become a standard for netbooting.  Unhappy vendor's are not going to make that happen.
 
> I will put in the standard disclaimer about not being an attorney, and 
> though I believe that  my analysis is reasonable, it should not be, and 
> is not intended to be relied upon in a legal context. One should seek 
> "legally" sanctioned advice in this case. :)

Me too...
 
> Now where was I... oh yes, debugging PXE chaining...
> 
I am getting a laptop without a floppy drive so i may have some call for this...

Tim

Re: [Etherboot-developers] UNDI driver

[Vasil V. <vas...@sy...>]

On Fri, 30 May 2003, Michael Brown wrote:

> On Thu, 29 May 2003, Georg Baum wrote:
> > I did, but it did not work (cvs from today). The PXE implementation is Intel
> > Boot Agent Version 4.0.22, the NIC is an onboard eepro100. Here are the
> > messages:
> > PXE loader for etherboot
> > 0x27EK low memory
> >  PXE    unloaded
> > ROM segment 0x000 length 0x000 reloc 0x00020000
> > Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
> > Relocating _text from: [00007e70,00017b80) to [076f02f0,07700000)
> > Boot from (N)etwork (D)isk (F)loppy or from (L)ocal? N
> > Probing pci nic...
> > [UNDI]Hunting for PnP BIOS...found $PnP at f000:6e20...ok
> > Hunting for pixies...found !PXE at 0009de0...ok
> > UNDI API call 0x0000 failed with status 0x006a
>
> PXE chaining is failing.  I will try to sort out the pxeloader code so
> that it leaves the !PXE structure in a valid state (unless one of Peter or
> Vasil has already done this).  This looks like the same problem Marty was
> experiencing.

I am bit rusty on this and don't have much time right now to try it, but
wouldn't it be enough to define PXELOADER_KEEP_UNDI? The only thing it does is
actually not to reduce the the memory as to free the UNDI memory.

Alternatively, if "leaving !PXE structure in a valid state" means we don't
stop the the UNDI, then just don't include the whole of pxeprefix.c .

I didn't quite follow what the UNDI driver requires, but once this is clear it
shouldn't be hard to just skip whichever steps shouldn't be done. I guess that
the whole of pxeprefix shouldn't be there.

Gread job, Michael. Now, I can just specify to a vendor that I want PXE
support and not what the NIC is. If the NIC has a driver in etherboot, then
great. If not, then I can still use etherboot.

In general, etherboot is turning into a great infrastructure.

Vasil

Re: [Etherboot-developers] UNDI driver

[Michael B. <mb...@fe...>]

On Thu, 29 May 2003, Marty Connor wrote:
> > I suspect this means that PXENV_UNDI_SET_STATION_ADDRESS isn't properly
> > implemented.  You can try commenting out the call as follows:
> >  int eb_pxenv_undi_set_station_address ( void ) {
> > -       return undi_call ( PXENV_UNDI_SET_STATION_ADDRESS );
> > +       return 1; /* undi_call ( PXENV_UNDI_SET_STATION_ADDRESS ); */
> >  }
> > PXE spec requires that we make this call, so it may be a case of "make 
> > the
> > call, ignore failures and continue".
> Success! With this patch both the 3Com and the Intel cards boot from 
> floppy using the UNDI driver.

Excellent.  I have committed an update that fixes this permanently; it 
will simply ignore failures from PXENV_UNDI_SET_STATION_ADDRESS and 
suppress the API call failed message.

> > I am off to Manchester for the next two days, but I'll take a look at 
> > the
> > weekend.
> Thanks for replying before you left.
> Have a nice trip!

Am still in Manchester at the moment (isn't ssh wonderful? :) - will get 
on to pxeprefix stuff tomorrow, unless anyone else wants to sort it out.

Michael

Re: [Etherboot-developers] UNDI driver

[Michael B. <mb...@fe...>]

On Thu, 29 May 2003, Georg Baum wrote:
> I did, but it did not work (cvs from today). The PXE implementation is Intel 
> Boot Agent Version 4.0.22, the NIC is an onboard eepro100. Here are the 
> messages:
> PXE loader for etherboot
> 0x27EK low memory
>  PXE    unloaded
> ROM segment 0x000 length 0x000 reloc 0x00020000
> Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
> Relocating _text from: [00007e70,00017b80) to [076f02f0,07700000)
> Boot from (N)etwork (D)isk (F)loppy or from (L)ocal? N
> Probing pci nic...
> [UNDI]Hunting for PnP BIOS...found $PnP at f000:6e20...ok
> Hunting for pixies...found !PXE at 0009de0...ok
> UNDI API call 0x0000 failed with status 0x006a

PXE chaining is failing.  I will try to sort out the pxeloader code so
that it leaves the !PXE structure in a valid state (unless one of Peter or
Vasil has already done this).  This looks like the same problem Marty was 
experiencing.

> Hunting for pixies...none found
> Hunting for ROMs...found 55AA at 000e0000...PCI:8086:1031...ok
> ROM contains IBA 4.0.22 Slot 0140 by Intel Corporation
> Located UNDI ROM supporting revision 2.1.0
> Installing UNDI driver code to 9d00:0000, data at 9380:0000
> UNDI driver created a pixie at 9d00:0070...ok
> API 9d00:0106 St 8c40:0800 UD 9380:94b0 UC 9d00:2050 BD 0000:3930 BC 
> 0000:5f86

And here we see one of the nice features of the driver at work: even 
though it failed to PXE-chain, it's trying to reload the PXE driver from 
ROM as a fallback strategy.  :)

> Initialized UNDI NIC with IO 0x7000, IRQ 11, MAC 00:00:E2:8C:5F:43
> NDIS type DIX+802.3 interface at 100 Mbps
> Searching for server (DHCP)...
> ..Me: 192.168.0.3, Server: 192.168.0.1
> Loading 
> 192.168.0.1:/tftpboot/lts/vmlinuz-2.4.19-ltsp-1-mknbi-1.2.7-10...(NBI)segment 
> [00092800, 000093A00) overlaps used basemem [00093000, 000A0000)
> error: not a valid image
> Unable to load file.
> <sleep>
> Note that the kernel image _is_ valid. After the sleep, it tries again with 
> different results and finally hangs after printing the "API 9d00:0106..." 
> line, but I think this is only a side-effect and the real error occurs 
> earlier.

The "not a valid image" message is misleading: the real error is the 
previous line talking about a segment overlap.  The image you are trying 
to load wants to be loaded at a location that would tread on the UNDI 
driver code.

Can you try enabling DEBUG_BASMEM in arch/i386/firmware/pcbios/basemem.c?  
This should print out diagnostic messages about base memory allocation, 
which will help us see what is going on.  It's possible (likely) that the 
!PXE structure from the first attempt is not being freed properly and so 
the second time round it loads it much lower in base memory than it would 
normally.  The alternative is that your UNDI driver is huge, in which case 
you will need to create an image that doesn't need to tread on that area 
of memory.

> Then I tried chaining from Etherboot, and the result is:
> ROM segment 0x1000 length 0x8000 reloc 0x00020000
> Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
> Relocating _text from: [0000241b0,00033ec0) to [076f02f0,07700000)
> Boot from (N)etwork (D)isk (F)loppy or from (L)ocal? N
> Probing pci nic...
> [UNDI]Hunting for PnP BIOS...found $PnP at f000:6e20...ok
> Hunting for pixies...none found
> Hunting for ROMs...found 55AA at 000e0000...PCI:8086:3577...not me 
> (8086:1031)
> ...none found
> Probing isa nic...
> <sleep>

It's not seeing the PXE ROM.  Do you have LAN enabled as part of the BIOS 
boot sequence?  I have found that some BIOSes (or possibly some PXE ROMs) 
will hide the PXE ROM if LAN is not listed as a BIOS boot device.

Michael

Re: [Etherboot-developers] Q: GPL license clarification.

[Marty C. <md...@et...>]

On Wednesday, May 28, 2003, at 04:24  AM, Eric W. Biederman wrote:
> Both the Linux kernel and dosemu have similar provisions in
> their preamble to clarify the intention of the developers.
> I have no wish to actually modify the license just to make
> it perfectly clear to BIOS vendors that etherboot is not
> a danger to them.

Such a clarification seems reasonable to me.

The GPL does place certain other responsibilities on a vendor wishing 
to distribute software that has been placed under the GPL. The 
provision about making the source code available comes to mind. It is 
not difficult to fulfill, but adds some overhead to using GPL software.

I have noted that the FSF has made efforts of late to inform business 
users that Free Software terms are not necessarily incompatible with 
commercial interests.

Here is a useful link on the FSF site:

     http://www.fsf.org/licenses/gpl-faq.html#GPLInProprietarySystem

It says in part:

...
However, in many cases you can distribute the GPL-covered software 
alongside your proprietary system. To do this validly, you must make 
sure that the free and non-free programs communicate at arms length, 
that they are not combined in a way that would make them effectively a 
single program.

The difference between this and "incorporating" the GPL-covered 
software is partly a matter of substance and partly form. The 
substantive part is this: if the two programs are combined so that they 
become effectively two parts of one program, then you can't treat them 
as two separate programs. So the GPL has to cover the whole thing.
...

Now, one could argue that Etherboot, because it can stand alone and run 
on bare metal is separable, and thus could be inserted into a BIOS chip 
as a storage medium, and because of the mechanism the BIOS uses to scan 
for extension ROMs, remains a separate program, and thus does not 
require the BIOS vendor to distribute the entire BIOS under the GPL.

Because the BIOS extension mechanism was created to allow vendors to 
use code that may in fact be unknown to the BIOS manufacturer when the 
BIOS is produced, as long as they follow the PNP BIOS specification, I 
think one could reasonably argue that Etherboot fits within the spirit 
of the GPL with regard to aggregation.

In fairness, I must admit to seeing the other side of the argument, 
that hinges on the phrase "combined so that they become effectively two 
parts of one program". the BIOS extension method does, at run-time, 
basically make the two pieces of code virtually a single unit. It's 
just that when you're at bare-metal, _any_ use of other code would 
probably look like that because it's the most efficient way to do it.

That being said, I am going to suggest that Adaptec and 3Com who have 
extension BIOSes that are sometimes on their cards, and sometimes put 
into the motherboard flash storage unit to be loaded by the BIOS at 
run-time, retain their license terms.

So, I think we are safe (and reasonable) in our interpretation. If 
someone makes the argument that being a BIOS extension make us 
"effectively two parts of one program" at run-time there would be a lot 
of very unhappy vendors.

I will put in the standard disclaimer about not being an attorney, and 
though I believe that  my analysis is reasonable, it should not be, and 
is not intended to be relied upon in a legal context. One should seek 
"legally" sanctioned advice in this case. :)

Now where was I... oh yes, debugging PXE chaining...

Marty


-- 
     Try: http://rom-o-matic.net/ to make Etherboot images instantly.

    Name: Marty Connor
US Mail: Entity Cyber, Inc.; P.O. Box 391827;
          Cambridge, MA 02139; USA
   Voice: (617) 491-6935; Fax: (617) 491-7046
   Email: md...@et...
     Web: http://www.etherboot.org/

Re: [Etherboot-developers] UNDI driver

[Georg B. <gb...@us...>]

Am Samstag, 24. Mai 2003 21:46 schrieb Michael Brown:
> I haven't yet got around to testing chaining from PXE, but
>
>   make bin/undi.zpxe
>
> will give the appropriate test image, should anyone wish to try it out.

I did, but it did not work (cvs from today). The PXE implementation is Intel 
Boot Agent Version 4.0.22, the NIC is an onboard eepro100. Here are the 
messages:

PXE loader for etherboot
0x27EK low memory
 PXE    unloaded
ROM segment 0x000 length 0x000 reloc 0x00020000
Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
Relocating _text from: [00007e70,00017b80) to [076f02f0,07700000)
Boot from (N)etwork (D)isk (F)loppy or from (L)ocal? N

Probing pci nic...
[UNDI]Hunting for PnP BIOS...found $PnP at f000:6e20...ok
Hunting for pixies...found !PXE at 0009de0...ok
UNDI API call 0x0000 failed with status 0x006a
Hunting for pixies...none found
Hunting for ROMs...found 55AA at 000e0000...PCI:8086:1031...ok
ROM contains IBA 4.0.22 Slot 0140 by Intel Corporation
Located UNDI ROM supporting revision 2.1.0
Installing UNDI driver code to 9d00:0000, data at 9380:0000
UNDI driver created a pixie at 9d00:0070...ok
API 9d00:0106 St 8c40:0800 UD 9380:94b0 UC 9d00:2050 BD 0000:3930 BC 
0000:5f86
Initialized UNDI NIC with IO 0x7000, IRQ 11, MAC 00:00:E2:8C:5F:43
NDIS type DIX+802.3 interface at 100 Mbps
Searching for server (DHCP)...
..Me: 192.168.0.3, Server: 192.168.0.1
Loading 
192.168.0.1:/tftpboot/lts/vmlinuz-2.4.19-ltsp-1-mknbi-1.2.7-10...(NBI)segment 
[00092800, 000093A00) overlaps used basemem [00093000, 000A0000)
error: not a valid image
Unable to load file.
<sleep>


Note that the kernel image _is_ valid. After the sleep, it tries again with 
different results and finally hangs after printing the "API 9d00:0106..." 
line, but I think this is only a side-effect and the real error occurs 
earlier.

Then I tried chaining from Etherboot, and the result is:


ROM segment 0x1000 length 0x8000 reloc 0x00020000
Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
Relocating _text from: [0000241b0,00033ec0) to [076f02f0,07700000)
Boot from (N)etwork (D)isk (F)loppy or from (L)ocal? N

Probing pci nic...
[UNDI]Hunting for PnP BIOS...found $PnP at f000:6e20...ok
Hunting for pixies...none found
Hunting for ROMs...found 55AA at 000e0000...PCI:8086:3577...not me 
(8086:1031)
...none found
Probing isa nic...
<sleep>


Michael, I hope this is of help to you. Please tell if I should do anything 
further to track the problem down.


Georg

Re: [Etherboot-developers] UNDI driver

[Marty C. <md...@et...>]

On Wednesday, May 28, 2003, at 09:24  PM, Michael Brown wrote:
> I suspect this means that PXENV_UNDI_SET_STATION_ADDRESS isn't properly
> implemented.  You can try commenting out the call as follows:
>  int eb_pxenv_undi_set_station_address ( void ) {
> -       return undi_call ( PXENV_UNDI_SET_STATION_ADDRESS );
> +       return 1; /* undi_call ( PXENV_UNDI_SET_STATION_ADDRESS ); */
>  }
> PXE spec requires that we make this call, so it may be a case of "make 
> the
> call, ignore failures and continue".

Success! With this patch both the 3Com and the Intel cards boot from 
floppy using the UNDI driver.

>> Chaining didn't go so well for me. I wasn't able to get it to work.
>> PXE loader for etherboot
>> 0x27FK low memory
>>   PXENV+ unloaded
>> ROM segment 0x0000 length 0x0000 reloc 0x00020000
>> Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
>> Relocating _text from: [0000c070,0001b780) to [07eb08f0,07ec0000)
>> Boot from (N)etwork (D)isk (F)loppy or from (L)ocal?
>> Probing pci nic...
>> [UNDI]Hunting for PnP BIOS... found $PnP at f000:2c30...ok
>> Hunting for pixies...found !PXE at 0009d7a0...ok
>> It hangs here.
> Hmmm, it looks as though the PXE loader is unloading the stack but 
> leaving
> the empty shell of the !PXE structure resident in memory, so that we 
> pick
> it up on a scan and try to use memory that has already been freed.
> Two-stage fix:
> 1. Possibly perform further validity checks on the !PXE structure,
>    although we're already checking the signature and checksum.
> 2. PXE loader should trash the UNDI code segment if it unloads the PXE
>    stack.

I think we're pretty close on this, and with some debugging we should 
be able to get it going.

Vasil and Peter, could you possibly make a version of (or patch for) 
pxeprefix.S for 5.1.8 that would implement strategy 2 above? I'm going 
to try to invalidate the checksum, but my assembler is rusty, and it 
will take a me a little while to get back up to speed. Many thanks for 
your help.

> You might get it to work if you define PXELOADER_KEEP_UNDI when 
> compiling
> pxeprefix.S; this should prevent the PXE loader from unloading the PXE
> stack.  I haven't tested this, just an idea.

I tried this, and it got a bit farther on some cards, but on others it 
hung.
I suspect the Etherboot UNDI driver is getting confused by the pieces 
of PXE and UNDI structure hanging around in memory. The other thing 
that comes to mind is so other machine state that is confusing at 
startup for Etherboot such as register contents or hooked INTs.

> I am off to Manchester for the next two days, but I'll take a look at 
> the
> weekend.

Thanks for replying before you left.
Have a nice trip!

Marty

-- 
     Try: http://rom-o-matic.net/ to make Etherboot images instantly.

    Name: Marty Connor
US Mail: Entity Cyber, Inc.; P.O. Box 391827;
          Cambridge, MA 02139; USA
   Voice: (617) 491-6935; Fax: (617) 491-7046
   Email: md...@et...
     Web: http://www.etherboot.org/

Re: [Etherboot-developers] Memory allocation

[<ke...@us...>]

I have to say you guys have done an amazing amount while I was away, I
have yet to catch up all with the postings. Maybe I should go away
again. :-)

>The only piece I am certain is missing are gatea20_unset calls in
>5.1.x because of problems with relocation.  But it looks like you have
>found the solution to that.  So we should be able to add them back in
>as appropriate.

Yup, A20 couldn't be unset while running above 1M or the code would get
gated out of existence. This is in the path that jumps to real mode
images i.e. NBI. So I temporarily disabled the call but it remains to be
fixed.  I like Michael's solution of using an even megabyte.

>While we are thinking about it the a20 code needs to move out of
>misc.c as it is highly arch dependent.

That too, yes.

Re: [Etherboot-developers] Memory allocation

[<ebi...@ln...>]

Michael Brown <mb...@fe...> writes:

> Weren't we restricted to 64KB until recently?  If we're going to exceed 
> 1MB in 5.2, we've got some serious code bloat!  :)

make bin/etherboot.img tends in this direction.
That is etherboot with all of the drivers compiled in.  
We still have a ways to go but we are not far off at the moment.

My basic point was that it is not hard to force the important
bits below 1MB if we need to.

> > > 2. Call gateA20_set() immediately after doing any UNDI API call.  Thanks 
> > >    to (1), I can do this cleanly in C instead of assembler. 
> > This is definitely a solution I like.
> > On the same toon there is some outstanding gatea20 work that was don in 5.0.x
> > that has never made it into 5.1.x.  Roughly it was disabling and reenabling
> > gatea20 around the transitions to/from real mode.
> 
> Where is this code?  Grepping for a20/A20 in 5.0/src doesn't find it.

I know there is something outstanding.  All I remember for certain
is that there was some a20 bug fixing in 5.0.x right at the start of
5.1.x that did get picked up in 5.1.x.

The only piece I am certain is missing are gatea20_unset calls in
5.1.x because of problems with relocation.  But it looks like you have
found the solution to that.  So we should be able to add them back in
as appropriate.

While we are thinking about it the a20 code needs to move out of
misc.c as it is highly arch dependent.

> > > > O.k.  One issue on my wishlist is to have a stub that loads etherboot
> > > > into high memory (>1MB) from the rom initialization code, and then just
> > > > leaves a very small stub in real mode.  Given that I have seen several
> > > > BIOS's start acting erratically because of their real mode storage for
> > > > options roms was used up.  This could make etherboot more reliable.
> > > Could be interesting; PMM is potentially in charge of high memory at the 
> > > time the initialisation code is called (assuming the BIOS supports PMM), 
> > > but anything it allocates is lost before the system starts booting.  I'm 
> > > not sure how you'd work around that problem.
> > Me either.  The PXE spec seems to require it to some extent.  But just
> > skimming it I don't see how.
> 
> It's not very clear.  PXE spec seems to state that the PXE ROM should move 
> itself into upper memory using PMM, implying that it stays there ready for 
> when the system boots.  However, PMM spec states that this is impossible; 
> anything allocated by PMM is for POST-time only and will be destroyed at 
> boot time.
> 
> To make it even more confusing, a disassembly of an Intel PXE ROM shows 
> debug messages that mention PMM, but AFAICT the code never makes any PMM 
> API calls.  (At least, the PMM signature, "$PMM", doesn't appear anywhere 
> within the ROM and this is the only way to locate PMM services within the 
> BIOS.)

Interesting...

Perhaps the guys who wrote the pxe spec were confused.

Eric

Re: [Etherboot-developers] UNDI driver

[Michael B. <mb...@fe...>]

> [UNDI]Hunting for PnP BIOS...found $PnP at f000:2c30...ok
> Hunting for pixies...none found
> Hunting for ROMs...found 55AA at 000cc000...PCI:8086:1229...ok
> ROM contains Intel UNDI, PXE-2.0 (build 067) by Intel Corporation
> Located UNDI ROM supporting revision 2.1.0
> Installing UNDI driver code to 9d80:0000, data at 9a80:0000
> UNDI driver created a pixie at 9d80:0070...ok
> API 9d80:0106 St 9430:0800 UD 9a80:4d30 UC 9d80:1e70 BD 0000:37c0 BC  
> 0000:563a
> UNDI API call 0x000a failed with status 0x0000
> <snip> 
> Looks like an UNDI call failed. Please let me know if I can help debug  
> this one further.

Call 0x000a is PXENV_UNDI_SET_STATION_ADDRESS (you can find the numbers in 
arch/i386/include/pxe.h; I left textual descriptions of the API numbers 
out of the code to save space).  Interestingly, status 0x0000 is 
PXENV_STATUS_SUCCESS, so there's a bug in the Intel code if it's returning 
exit = PXENV_EXIT_FAILURE with status = PXENV_STATUS_SUCCESS.

I suspect this means that PXENV_UNDI_SET_STATION_ADDRESS isn't properly 
implemented.  You can try commenting out the call as follows:



--- arch/i386/drivers/net/undi.c        24 May 2003 20:16:25 -0000      1.12
+++ arch/i386/drivers/net/undi.c        29 May 2003 01:17:30 -0000
@@ -443,7 +443,7 @@
 }

 int eb_pxenv_undi_set_station_address ( void ) {
-       return undi_call ( PXENV_UNDI_SET_STATION_ADDRESS );
+       return 1; /* undi_call ( PXENV_UNDI_SET_STATION_ADDRESS ); */
 }

 int eb_pxenv_undi_get_information ( void ) {



PXE spec requires that we make this call, so it may be a case of "make the 
call, ignore failures and continue".

> Chaining didn't go so well for me. I wasn't able to get it to work.  
> Here's a relevant part of my dhcpd.conf file:
> # 3COM PXE Card
>      host ws137 {
>          hardware ethernet     00:01:02:59:03:d5;
>          fixed-address         192.168.2.137;
>          if        substring (option vendor-class-identifier, 0, 9) =  
> "PXEClient" {
>              filename "/undi.zpxe";
>          } else if substring (option vendor-class-identifier, 0, 9) =  
> "Etherboot" {
>              filename              "/lts/vmlinuz-2.4.19-ltsp-1";
>              option vendor-encapsulated-options  
> 3c:09:45:74:68:65:72:62:6f:6f:74:ff;
>          }
>      }
> Here's the output using the 3Com Card:
> Managed PC Boot Agent (MBA) v4.00
> (C) Copyright 1999 Lanworks Technologies Co. a subsidiary of 3Com  
> Corporation
> All rights reserved.
> Pre-boot eXecution Environment (PXE) v2.00
> (C) Copyright 1999 Intel Corporation.
> (C) Copyright 1999 Lanworks Technologies Co. a subsidiary of 3Com  
> Corporation
> All rights reserved.
> DHCP MAC ADDR: 00 01 02 59 03 D5
> CLIENT IP: 192.168.2.137  MASK: 255.255.255.0  DHCP IP: 192.167.2.37
> GATEWAY IP: 192.168.2.15
> PXE loader for etherboot
> 0x27FK low memory
>   PXENV+ unloaded
> ROM segment 0x0000 length 0x0000 reloc 0x00020000
> Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
> Relocating _text from: [0000c070,0001b780) to [07eb08f0,07ec0000)
> Boot from (N)etwork (D)isk (F)loppy or from (L)ocal?
> Probing pci nic...
> [UNDI]Hunting for PnP BIOS... found $PnP at f000:2c30...ok
> Hunting for pixies...found !PXE at 0009d7a0...ok
> It hangs here.
> Please let me know how I can help debug this. It would be way cool to  
> make this work such that any PXE card could use Etherboot just by  
> chaining the Etherboot PXE loader.

Hmmm, it looks as though the PXE loader is unloading the stack but leaving 
the empty shell of the !PXE structure resident in memory, so that we pick 
it up on a scan and try to use memory that has already been freed.  
Two-stage fix:

1. Possibly perform further validity checks on the !PXE structure,
   although we're already checking the signature and checksum.

2. PXE loader should trash the UNDI code segment if it unloads the PXE 
   stack.

You might get it to work if you define PXELOADER_KEEP_UNDI when compiling 
pxeprefix.S; this should prevent the PXE loader from unloading the PXE 
stack.  I haven't tested this, just an idea.

I am off to Manchester for the next two days, but I'll take a look at the 
weekend.

> P.S. The screen texts above might have minor typos as I copied them by  
> hand. I tried my best to check the numbers to give you the best  
> information I could.

Many thanks; it's always useful to get detailed bug reports.

Michael

Re: [Etherboot-developers] UNDI driver

[Marty C. <md...@et...>]

On Saturday, May 24, 2003, at 03:46  PM, Michael Brown wrote:
> The i386 UNDI driver in Etherboot 5.1 CVS should now be functioning
> correctly.  Please test and give feedback.
>   make bin/undi.zfd0
> will give you a bootable floppy that should be able to detect any PXE
> ROMs, load the UNDI drivers and use them to boot as per normal.

OK, I have put together a test system, and have two Ethernet cards (an  
Intel EEPRO100, and a 3Com 3C905C-TX-M) with PXE, and an HP branded  
thin client with an on-board VIA 6102 chipset and Intel PXE 2.2 in BIOS  
to test with.

I got the latest copy of Etherboot to my Redhat 8.0 system with:

     $ cvs -d:pserver:ano...@cv...:/cvsroot/etherboot  
login

followed by:

     $ cvs -d:pserver:ano...@cv...:/cvsroot/etherboot  
checkout etherboot

to download it.

I then did:

     $ cd etherboot/etherboot-5.1/src/
     $ make bin/undi.zfd0

to create the test floppy.

My test system is an Intel CA810E motherboard with what I believe is a  
Phoenix derived BIOS (not sure of that).

My first test went well:

NIC: 3C905-TX-M
Boot METHOD: FLOPPY
OUTPUT:
Loading ROM image...............
ROM segment 0x0000 length 0x0000 reloc 0x00020000
Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
Boot from (N)etwork (D)disk (F)loppy or from (L)ocal?
Probing pci nic...
[UNDI]Hunting for PnP BIOS...found $PnP at f000:2c30...ok
Hunting for pixies...none found
Hunting for ROMs...found 55AA at 000cc000...PCI:10b7:9200...ok
ROM contains MBA UNDI by 3Com
Located UNDI ROM supporting revision 2.1.0
Installing UNDI driver code to 9d00:0000, data at 99c0:0000
UNDI driver created a pixie at 9d00:0060...ok
API 9d00:00f6 St 0000:0000 UD 9c40:3284 UC 9d00:24c0 BD 0000:0000 BC  
0000:0000
Initialized UNDI NIC with IO 0xdc00, IRQ 11, MAC 00:01:02:59:03:D5
NDIS type DIX+802.3 interface at 100Mbps
Searching for server (DHCP)...
..Me: 192.168.2.137, Server: 192.168.2.37, Gateway 192.168.2.15
Loading 192.168.2.37:/lts/vmlinuz-2.4.19-ltsp-1  
...(NBI)........................
........................................................................ 
........
........................................................................ 
........
......done
mknbi-1.2-7/first32.c (GPL)
Top of ramdisk is 0X07EC0000
Ramdisk at 0X07E0D000, size 0X000B3000
Uncompressing Linux... Ok, booting the kernel.

Works!

My next test was to try the same thing with the other card:

NIC: EEPRO100
Boot METHOD: FLOPPY
OUTPUT:
Loading ROM image...............
ROM segment 0x0000 length 0x0000 reloc 0x00020000
Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
Boot from (N)etwork (D)disk (F)loppy or from (L)ocal?
Probing pci nic...
[UNDI]Hunting for PnP BIOS...found $PnP at f000:2c30...ok
Hunting for pixies...none found
Hunting for ROMs...found 55AA at 000cc000...PCI:8086:1229...ok
ROM contains Intel UNDI, PXE-2.0 (build 067) by Intel Corporation
Located UNDI ROM supporting revision 2.1.0
Installing UNDI driver code to 9d80:0000, data at 9a80:0000
UNDI driver created a pixie at 9d80:0070...ok
API 9d80:0106 St 9430:0800 UD 9a80:4d30 UC 9d80:1e70 BD 0000:37c0 BC  
0000:563a
UNDI API call 0x000a failed with status 0x0000
Hunting for ROMS...found 55AA at 000c0000...PCI:8086:7125...not me  
(8086:1229)
...none found
Probing isa nic...
<sleep>
Boot from (N)etwork (D)isk (F)loppy or from (L)ocal?

Looks like an UNDI call failed. Please let me know if I can help debug  
this one further.

> I haven't yet got around to testing chaining from PXE, but
>   make bin/undi.zpxe
> will give the appropriate test image, should anyone wish to try it out.

Chaining didn't go so well for me. I wasn't able to get it to work.  
Here's a relevant part of my dhcpd.conf file:

# 3COM PXE Card
     host ws137 {
         hardware ethernet     00:01:02:59:03:d5;
         fixed-address         192.168.2.137;
         if        substring (option vendor-class-identifier, 0, 9) =  
"PXEClient" {
             filename "/undi.zpxe";
         } else if substring (option vendor-class-identifier, 0, 9) =  
"Etherboot" {
             filename              "/lts/vmlinuz-2.4.19-ltsp-1";
             option vendor-encapsulated-options  
3c:09:45:74:68:65:72:62:6f:6f:74:ff;
         }
     }

Here's the output using the 3Com Card:

Managed PC Boot Agent (MBA) v4.00
(C) Copyright 1999 Lanworks Technologies Co. a subsidiary of 3Com  
Corporation
All rights reserved.

Pre-boot eXecution Environment (PXE) v2.00
(C) Copyright 1999 Intel Corporation.
(C) Copyright 1999 Lanworks Technologies Co. a subsidiary of 3Com  
Corporation
All rights reserved.

DHCP MAC ADDR: 00 01 02 59 03 D5
CLIENT IP: 192.168.2.137  MASK: 255.255.255.0  DHCP IP: 192.167.2.37
GATEWAY IP: 192.168.2.15
PXE loader for etherboot
0x27FK low memory
  PXENV+ unloaded
ROM segment 0x0000 length 0x0000 reloc 0x00020000
Etherboot 5.1.8 (GPL) Tagged ELF for [UNDI]
Relocating _text from: [0000c070,0001b780) to [07eb08f0,07ec0000)
Boot from (N)etwork (D)isk (F)loppy or from (L)ocal?
Probing pci nic...
[UNDI]Hunting for PnP BIOS... found $PnP at f000:2c30...ok
Hunting for pixies...found !PXE at 0009d7a0...ok

It hangs here.

Please let me know how I can help debug this. It would be way cool to  
make this work such that any PXE card could use Etherboot just by  
chaining the Etherboot PXE loader.

> There are no known issues for systems containing a single NIC and  
> single
> PXE ROM at this time.  (Multiple NICs / multiple PXE ROMs may have
> problems; please let me know.)
> Please do give feedback; this driver has to work with any PXE
> implementation, and I have only Intel's to play with.
> Michael

We might consider sending a message to the etherboot-users list once  
things are a little farther along asking for testers. Once they see how  
easy it is to get a cvs version and make the required test images, we  
might get a few people to give it a whirl.

Thanks for your efforts, Michael. This UNDI driver could be a useful  
addition for people who   have PXE and use it to boot Etherboot as well  
as some other cases we haven't even thought of yet.

Marty

P.S. The screen texts above might have minor typos as I copied them by  
hand. I tried my best to check the numbers to give you the best  
information I could.

-- 
     Try: http://rom-o-matic.net/ to make Etherboot images instantly.

    Name: Marty Connor
US Mail: Entity Cyber, Inc.; P.O. Box 391827;
          Cambridge, MA 02139; USA
   Voice: (617) 491-6935; Fax: (617) 491-7046
   Email: md...@et...
     Web: http://www.etherboot.org/

Re: [Etherboot-developers] Memory allocation

[Michael B. <mb...@fe...>]

On Wed, 28 May 2003, Eric W. Biederman wrote:
> > I didn't.  The UNDI driver (i.e. the one in the PXE ROM) plays about with
> > it whenever it gets bored and feels like causing some extra debugging
> > work.  I have a two-stage solution to this problem:
> > 1. Ensure Etherboot text is always within an even megabyte, so that 
> >    _real_call returns successfully and we make it all the way back to the 
> >    C code even if something nasty happens to A20.  (Hey, half your 
> >    memory's just disappeared?  No problem! :)
> That is a little limiting (etherboot text && stack must be in 1MB) but
> in practice this does not look like a problem.  Especially as special
> linker sections can force the issue if we exceed 1MB.

Weren't we restricted to 64KB until recently?  If we're going to exceed 
1MB in 5.2, we've got some serious code bloat!  :)

> > 2. Call gateA20_set() immediately after doing any UNDI API call.  Thanks 
> >    to (1), I can do this cleanly in C instead of assembler. 
> This is definitely a solution I like.
> On the same toon there is some outstanding gatea20 work that was don in 5.0.x
> that has never made it into 5.1.x.  Roughly it was disabling and reenabling
> gatea20 around the transitions to/from real mode.

Where is this code?  Grepping for a20/A20 in 5.0/src doesn't find it.

> > > O.k.  One issue on my wishlist is to have a stub that loads etherboot
> > > into high memory (>1MB) from the rom initialization code, and then just
> > > leaves a very small stub in real mode.  Given that I have seen several
> > > BIOS's start acting erratically because of their real mode storage for
> > > options roms was used up.  This could make etherboot more reliable.
> > Could be interesting; PMM is potentially in charge of high memory at the 
> > time the initialisation code is called (assuming the BIOS supports PMM), 
> > but anything it allocates is lost before the system starts booting.  I'm 
> > not sure how you'd work around that problem.
> Me either.  The PXE spec seems to require it to some extent.  But just
> skimming it I don't see how.

It's not very clear.  PXE spec seems to state that the PXE ROM should move 
itself into upper memory using PMM, implying that it stays there ready for 
when the system boots.  However, PMM spec states that this is impossible; 
anything allocated by PMM is for POST-time only and will be destroyed at 
boot time.

To make it even more confusing, a disassembly of an Intel PXE ROM shows 
debug messages that mention PMM, but AFAICT the code never makes any PMM 
API calls.  (At least, the PMM signature, "$PMM", doesn't appear anywhere 
within the ROM and this is the only way to locate PMM services within the 
BIOS.)

Michael

Re: [Etherboot-developers] Memory allocation

[<ebi...@ln...>]

Michael Brown <mb...@fe...> writes:

> > > > > 4. How do we avoid treading on other entities?  (In particular: the 
> > > > > Intel UNDI driver seems to install something just above the 1MB mark.
> 
> > > > >    prep_segment() then obliterates this, because it can't tell that the
> > > > >    region is in use).
> > > > Very good question.  It looks like we need to rescan things (at least
> > > > in the UNDI driver case.
> > > UNDI driver was a false alert; it turned out to be the status of the A20 
> > > line.  I thought I must be treading on something UNDI-related when I 
> > > loaded a segment in the [1MB,2MB) range, but actually I was simply 
> > > vapourising the contents of base memory thanks to A20.  Have you ever 
> > > tried doing memset ( phys_to_virt(0), '!', 1<<20 );  ?  :-)
> > How did you fail to enable the a20 line?
> 
> I didn't.  The UNDI driver (i.e. the one in the PXE ROM) plays about with
> it whenever it gets bored and feels like causing some extra debugging
> work.  I have a two-stage solution to this problem:
> 
> 1. Ensure Etherboot text is always within an even megabyte, so that 
>    _real_call returns successfully and we make it all the way back to the 
>    C code even if something nasty happens to A20.  (Hey, half your 
>    memory's just disappeared?  No problem! :)

That is a little limiting (etherboot text && stack must be in 1MB) but
in practice this does not look like a problem.  Especially as special
linker sections can force the issue if we exceed 1MB.
 
> 2. Call gateA20_set() immediately after doing any UNDI API call.  Thanks 
>    to (1), I can do this cleanly in C instead of assembler. 

This is definitely a solution I like.

On the same toon there is some outstanding gatea20 work that was don in 5.0.x
that has never made it into 5.1.x.  Roughly it was disabling and reenabling
gatea20 around the transitions to/from real mode.
 
> > > I've looked into PMM a little more and we can, I think, ignore it; it is 
> > > to do with allocating memory above 1MB but only during POST.  As soon as 
> > > Int 19 happens (i.e. before we get control), any memory allocated via PMM 
> > > is freed and PMM ceases to be available.
> > O.k.  One issue on my wishlist is to have a stub that loads etherboot
> > into high memory (>1MB) from the rom initialization code, and then just
> > leaves a very small stub in real mode.  Given that I have seen several
> > BIOS's start acting erratically because of their real mode storage for
> > options roms was used up.  This could make etherboot more reliable.
> 
> Could be interesting; PMM is potentially in charge of high memory at the 
> time the initialisation code is called (assuming the BIOS supports PMM), 
> but anything it allocates is lost before the system starts booting.  I'm 
> not sure how you'd work around that problem.

Me either.  The PXE spec seems to require it to some extent.  But just
skimming it I don't see how.
 
> [ Some working real mode stub code would, however, come in handy for the 
>   case of PXE-on-Etherboot; we would need to provide a real-mode entry 
>   point. ]

There are 2 cases of real mode stubs.
Callbacks PXE-on-Etherboot.
The initial boot, and transitioning from a compressed image in an
    UMB to wherever we will finally land.

I can multiplex them all through the standard entry point in main
easily.  But I don't currently see how to reduce the UMB footprint.
And the trend for new NICs  (tg3 e1000) is that they require larger
drivers.

Eric

Re: [Etherboot-developers] Memory allocation

[Michael B. <mb...@fe...>]

> > > > 4. How do we avoid treading on other entities?  (In particular: the 
> > > >    Intel UNDI driver seems to install something just above the 1MB mark.  
> > > >    prep_segment() then obliterates this, because it can't tell that the
> > > >    region is in use).
> > > Very good question.  It looks like we need to rescan things (at least
> > > in the UNDI driver case.
> > UNDI driver was a false alert; it turned out to be the status of the A20 
> > line.  I thought I must be treading on something UNDI-related when I 
> > loaded a segment in the [1MB,2MB) range, but actually I was simply 
> > vapourising the contents of base memory thanks to A20.  Have you ever 
> > tried doing memset ( phys_to_virt(0), '!', 1<<20 );  ?  :-)
> How did you fail to enable the a20 line?

I didn't.  The UNDI driver (i.e. the one in the PXE ROM) plays about with
it whenever it gets bored and feels like causing some extra debugging
work.  I have a two-stage solution to this problem:

1. Ensure Etherboot text is always within an even megabyte, so that 
   _real_call returns successfully and we make it all the way back to the 
   C code even if something nasty happens to A20.  (Hey, half your 
   memory's just disappeared?  No problem! :)

2. Call gateA20_set() immediately after doing any UNDI API call.  Thanks 
   to (1), I can do this cleanly in C instead of assembler. 

> > I've looked into PMM a little more and we can, I think, ignore it; it is 
> > to do with allocating memory above 1MB but only during POST.  As soon as 
> > Int 19 happens (i.e. before we get control), any memory allocated via PMM 
> > is freed and PMM ceases to be available.
> O.k.  One issue on my wishlist is to have a stub that loads etherboot
> into high memory (>1MB) from the rom initialization code, and then just
> leaves a very small stub in real mode.  Given that I have seen several
> BIOS's start acting erratically because of their real mode storage for
> options roms was used up.  This could make etherboot more reliable.

Could be interesting; PMM is potentially in charge of high memory at the 
time the initialisation code is called (assuming the BIOS supports PMM), 
but anything it allocates is lost before the system starts booting.  I'm 
not sure how you'd work around that problem.

[ Some working real mode stub code would, however, come in handy for the 
  case of PXE-on-Etherboot; we would need to provide a real-mode entry 
  point. ]

Michael

Re: [Etherboot-developers] Memory allocation

[<ebi...@ln...>]

Michael Brown <mb...@fe...> writes:

> > > 3. Etherboot real-mode stack.  This is always at the top of free base 
> > >    memory (as defined by 40:13), provided that Etherboot is the only thing 
> > >    allocating base memory.
> > >    NB. Other code may allocate base memory by changing the counter at
> > >    40:13.  We should really recalculate the real-mode stack top each time
> > >    we make a _real_call, based on the current contents of 40:13.  Is there 
> > >    any reason why we can't do this?  (I'm thinking of LinuxBIOS in 
> > >    particular; does this use the same 40:13 counter?)
> > LinuxBIOS does not use 40:13 at the present time.  In fact the only
> > interface to the outside world (Besides the simple LinuxBIOS table)
> > visible when running etherboot is etherboot itself.
> > Care needs to be take because I'm not certain 40:13 is reliable.  I
> > know there are several cases that do BIOS calls to get this
> > information.  And the linux kernel explicitly does not read it with
> > a comment about it being unreliable.
> 
> The PXE spec actually instructs you to use 40:13, so I'm comfortable doing 
> it in the case of the Etherboot UNDI driver; if 40:13 doesn't work then 
> the PXE driver would probably fail anyway.
> 
> 40:13 is used only by routines in basemem.c.  However, I did modify 
> pcbios/memsizes.c to make a call to adjust_real_mode_stack() in basemem.c 
> in order to centralise the management of the real mode stack location.  If 
> you think 40:13 might not be reliable then you may want to revert this 
> change.

It is quite possible there is a strong difference between what is reliable
by the time the OS starts, and what is reliable beforehand.  Unless we
have reason to suspect I would suggest leaving it in.
 
> > > 5. Allocated base memory, from 640K down to [40:13]k.  Includes the BIOS 
> > >    EPDA.
> > > The interaction of these five areas with prep_segment() has the following 
> > > problems:
> > > a. No checks are made for (3).  prep_segment will happily trash the 
> > >    real-mode stack and _real_call will happily walk all over an area that
> > >    prep_segment has prepared.
> > Correct.  That was an after thought.
> 
> What is the size of the real-mode stack, and shall we add it in as another 
> check in prep_segment?

At most 64K. As for a check that is a good question.  Ideally we should be
able to avoid the areas claimed by prep_segment.  But that would require
remember things.

> > > 4. How do we avoid treading on other entities?  (In particular: the 
> > >    Intel UNDI driver seems to install something just above the 1MB mark.  
> > >    prep_segment() then obliterates this, because it can't tell that the
> > >    region is in use).
> > Very good question.  It looks like we need to rescan things (at least
> > in the UNDI driver case.
> 
> UNDI driver was a false alert; it turned out to be the status of the A20 
> line.  I thought I must be treading on something UNDI-related when I 
> loaded a segment in the [1MB,2MB) range, but actually I was simply 
> vapourising the contents of base memory thanks to A20.  Have you ever 
> tried doing memset ( phys_to_virt(0), '!', 1<<20 );  ?  :-)

How did you fail to enable the a20 line?
 
> > > 5. Would it take into account the BIOS PMM, if it exists?
> > Possibly.  I don't recall what that is...
> > If it is BIOS services for allocating memory above 1MB then that would
> > be a good idea.
> 
> I've looked into PMM a little more and we can, I think, ignore it; it is 
> to do with allocating memory above 1MB but only during POST.  As soon as 
> Int 19 happens (i.e. before we get control), any memory allocated via PMM 
> is freed and PMM ceases to be available.

O.k.  One issue on my wishlist is to have a stub that loads etherboot
into high memory (>1MB) from the rom initialization code, and then
just leaves a very small stub in real mode.  Given that I have seen
several BIOS's start acting erratically because of their real mode
storage for options roms was used up.  This could make etherboot more 
reliable.

Eric

Re: [Etherboot-developers] Memory allocation

[Michael B. <mb...@fe...>]

> > 2. Etherboot protected-mode stack.  I think this is within the Etherboot 
> >    text, somebody please confirm/correct me on this.
> Actually a better way of looking at Etherboot text and the stack is:
> Etherboot text+data+bss  They are all of a predetermined fixed size,
> and the protected mode stack is allocated in the bss.  The bss is important
> because it takes no space in the etherboot image but it is allocated
> statically at build time.
> static char buf[255];  or any other static variable without an initializer
> will be allocated in the bss.

Thanks, that's handy to know.

> > 3. Etherboot real-mode stack.  This is always at the top of free base 
> >    memory (as defined by 40:13), provided that Etherboot is the only thing 
> >    allocating base memory.
> >    NB. Other code may allocate base memory by changing the counter at
> >    40:13.  We should really recalculate the real-mode stack top each time
> >    we make a _real_call, based on the current contents of 40:13.  Is there 
> >    any reason why we can't do this?  (I'm thinking of LinuxBIOS in 
> >    particular; does this use the same 40:13 counter?)
> LinuxBIOS does not use 40:13 at the present time.  In fact the only
> interface to the outside world (Besides the simple LinuxBIOS table)
> visible when running etherboot is etherboot itself.
> Care needs to be take because I'm not certain 40:13 is reliable.  I
> know there are several cases that do BIOS calls to get this
> information.  And the linux kernel explicitly does not read it with
> a comment about it being unreliable.

The PXE spec actually instructs you to use 40:13, so I'm comfortable doing 
it in the case of the Etherboot UNDI driver; if 40:13 doesn't work then 
the PXE driver would probably fail anyway.

40:13 is used only by routines in basemem.c.  However, I did modify 
pcbios/memsizes.c to make a call to adjust_real_mode_stack() in basemem.c 
in order to centralise the management of the real mode stack location.  If 
you think 40:13 might not be reliable then you may want to revert this 
change.

> > 5. Allocated base memory, from 640K down to [40:13]k.  Includes the BIOS 
> >    EPDA.
> > The interaction of these five areas with prep_segment() has the following 
> > problems:
> > a. No checks are made for (3).  prep_segment will happily trash the 
> >    real-mode stack and _real_call will happily walk all over an area that
> >    prep_segment has prepared.
> Correct.  That was an after thought.

What is the size of the real-mode stack, and shall we add it in as another 
check in prep_segment?

> > 4. How do we avoid treading on other entities?  (In particular: the 
> >    Intel UNDI driver seems to install something just above the 1MB mark.  
> >    prep_segment() then obliterates this, because it can't tell that the
> >    region is in use).
> Very good question.  It looks like we need to rescan things (at least
> in the UNDI driver case.

UNDI driver was a false alert; it turned out to be the status of the A20 
line.  I thought I must be treading on something UNDI-related when I 
loaded a segment in the [1MB,2MB) range, but actually I was simply 
vapourising the contents of base memory thanks to A20.  Have you ever 
tried doing memset ( phys_to_virt(0), '!', 1<<20 );  ?  :-)

> > 5. Would it take into account the BIOS PMM, if it exists?
> Possibly.  I don't recall what that is...
> If it is BIOS services for allocating memory above 1MB then that would
> be a good idea.

I've looked into PMM a little more and we can, I think, ignore it; it is 
to do with allocating memory above 1MB but only during POST.  As soon as 
Int 19 happens (i.e. before we get control), any memory allocated via PMM 
is freed and PMM ceases to be available.

Michael

Re: [Etherboot-developers] Q: GPL license clarification.

[<ebi...@ln...>]

Markus Gutschke <ma...@gu...> writes:

> Ken Yap wrote:
> >>Would everyone be amenable to some additional language
> >>in the GPL license clarifying that simple aggregation of
> >>etherboot in the same rom as another BIOS.  Does not constitute
> >>linkage and does not slurp the vendors ROM under the GPL.
> >>
> >>I don't think anyone has intended that but I have had at
> >>least one nervous BIOS vendor say something about it.
> > Do you mean some clarification that putting Etherboot in the ROM does
> > not suck in the BIOS under the GPL? I think this paragraph in COPYING is
> > the relevant one:
> > In addition, mere aggregation of another work not based on the Program
> > with the Program (or with a work based on the Program) on a volume of
> > a storage or distribution medium does not bring the other work under
> > the scope of this License.
> 
> [ Long legal discussion ahead; if it bores you, then the summary is:
>    Go ahead! I see no problem with bundling etherboot with proprietary
>    software in the same ROM. ]
> 
> 
> I believe this is the same issue that Linus keeps mentioning on the kernel
> mailing list every so often.

Either interpretation is valid.

> if
> the BIOS vendor in question was really concerned about the legality of bundling
> etherboot with the BIOS, I would recommend they make a tool available that does
> exactly this: upgrade the etherboot implementation independently of the system
> BIOS.

That is what they are currently working on, and it is certainly 
a convenient feature to have.  However it is my perception that this
is unnecessary, and I would like for the perception to be that
making such an arrangement is nothing more than a means of
covering of being doubly certain no problems will result.

> 
> If it makes anybody feel better, I would have no problem with adding a clause to
> 
> the license saying that it is OK to aggregate etherboot together with
> proprietary software in the same ROM as long as the standard BIOS interface for
> communicating between etherboot and the other software is retained. In fact, if
> my legal analysis is correct (and please note that I am not a lawyer) this
> statement just reiterates what the law says anyway. On the other hand, if my
> analysis is wrong and both pieces of code are derived works of each other, than
> we could only grant this license extension if we managed to track down everybody
> 
> who ever substantially contributed to etherboot -- now, that could be tricky.

I only recommend adding text to the preamble to clarify the intent.
Not to change the terms and conditions of the license.

Mostly this is aimed at reducing fear uncertainty and doubt with
respect to etherboot.

Both the Linux kernel and dosemu have similar provisions in
their preamble to clarify the intention of the developers.

I have no wish to actually modify the license just to make
it perfectly clear to BIOS vendors that etherboot is not
a danger to them.

Eric

Re: [Etherboot-developers] Q: GPL license clarification.

[Markus G. <ma...@gu...>]

Ken Yap wrote:
>>Would everyone be amenable to some additional language
>>in the GPL license clarifying that simple aggregation of
>>etherboot in the same rom as another BIOS.  Does not constitute
>>linkage and does not slurp the vendors ROM under the GPL.
>>
>>I don't think anyone has intended that but I have had at
>>least one nervous BIOS vendor say something about it.
> 
> Do you mean some clarification that putting Etherboot in the ROM does
> not suck in the BIOS under the GPL? I think this paragraph in COPYING is
> the relevant one:
> 
> In addition, mere aggregation of another work not based on the Program
> with the Program (or with a work based on the Program) on a volume of
> a storage or distribution medium does not bring the other work under
> the scope of this License.

[ Long legal discussion ahead; if it bores you, then the summary is:
   Go ahead! I see no problem with bundling etherboot with proprietary
   software in the same ROM. ]


I believe this is the same issue that Linus keeps mentioning on the 
kernel mailing list every so often.

If both pieces of code depend on each other and can not be used 
seperately, then one would be considered a derived work of the other. In 
this case, authors of both works could claim copyright on the combined 
work and it could only be licensed under conditions both of them agree to.

On the other hand, if each piece of code can be used completely 
independently, then each author only holds rights to the component they 
have written, and there is (probably) no way they could use a license 
for one piece of code to enforce restrictions on the other; e.g. it is 
probably not possible for a BIOS vendor to say that users cannot run 
etherboot on a machine that includes their BIOS; any attempt to enforce 
a restriction like this would likely be ruled null and void by the 
courts of most countries, as it would be considered in violation of 
fair-use rights.

While lawyers can argue this point forever, and while there does not 
seem to be much precedent for deciding this problem, I would think it is 
fair to assume that a good argument could be made for considering both 
etherboot and the BIOS to be seperate entities that are merely 
aggregated together.

There are two factors that lead me to this conclusion:

  a) there is a well-defined interface that both programs implement for 
interacting with each other. It is provably possible to combine each one 
of the two pieces of code with any one of multiple different 
implementations of the other component (i.e. etherboot can work with all 
  current PC BIOSs, and PC BIOS implementations can call ROM extensions 
other than just etherboot).

  b) it is possible to remove any one of the two components from the 
system ROM and to upgrade them independently without affecting the other 
one. In fact, if the BIOS vendor in question was really concerned about 
the legality of bundling etherboot with the BIOS, I would recommend they 
make a tool available that does exactly this: upgrade the etherboot 
implementation independently of the system BIOS.

If it makes anybody feel better, I would have no problem with adding a 
clause to the license saying that it is OK to aggregate etherboot 
together with proprietary software in the same ROM as long as the 
standard BIOS interface for communicating between etherboot and the 
other software is retained. In fact, if my legal analysis is correct 
(and please note that I am not a lawyer) this statement just reiterates 
what the law says anyway. On the other hand, if my analysis is wrong and 
both pieces of code are derived works of each other, than we could only 
grant this license extension if we managed to track down everybody who 
ever substantially contributed to etherboot -- now, that could be tricky.


Markus

Re: [Etherboot-developers] Q: GPL license clarification.

[<ke...@us...>]

>Would everyone be amenable to some additional language
>in the GPL license clarifying that simple aggregation of
>etherboot in the same rom as another BIOS.  Does not constitute
>linkage and does not slurp the vendors ROM under the GPL.
>
>I don't think anyone has intended that but I have had at
>least one nervous BIOS vendor say something about it.

Do you mean some clarification that putting Etherboot in the ROM does
not suck in the BIOS under the GPL? I think this paragraph in COPYING is
the relevant one:

In addition, mere aggregation of another work not based on the Program
with the Program (or with a work based on the Program) on a volume of
a storage or distribution medium does not bring the other work under
the scope of this License.

[Etherboot-developers] Q: GPL license clarification.

[<ebi...@ln...>]

Would everyone be amenable to some additional language
in the GPL license clarifying that simple aggregation of
etherboot in the same rom as another BIOS.  Does not constitute
linkage and does not slurp the vendors ROM under the GPL.

I don't think anyone has intended that but I have had at
least one nervous BIOS vendor say something about it.

Eric

Re: [Etherboot-developers] Memory allocation

[<ebi...@ln...>]

Michael Brown <mb...@fe...> writes:

> Memory allocation within Etherboot seems to suffer from several problems 
> at the moment.  As far as I can tell, we have the following memory ranges 
> being used:
> 
> 1. Etherboot text.  This will be at RELOCADDR (=0x20000) if -DRELOCATE is
>    not defined, otherwise somewhere towards the top of available RAM.  
>    (Will always be within an even megabyte, i.e. A20=0).
> 
> 2. Etherboot protected-mode stack.  I think this is within the Etherboot 
>    text, somebody please confirm/correct me on this.

Actually a better way of looking at Etherboot text and the stack is:
Etherboot text+data+bss  They are all of a predetermined fixed size,
and the protected mode stack is allocated in the bss.  The bss is important
because it takes no space in the etherboot image but it is allocated
statically at build time.

static char buf[255];  or any other static variable without an initializer
will be allocated in the bss.

> 3. Etherboot real-mode stack.  This is always at the top of free base 
>    memory (as defined by 40:13), provided that Etherboot is the only thing 
>    allocating base memory.
> 
>    NB. Other code may allocate base memory by changing the counter at
>    40:13.  We should really recalculate the real-mode stack top each time
>    we make a _real_call, based on the current contents of 40:13.  Is there 
>    any reason why we can't do this?  (I'm thinking of LinuxBIOS in 
>    particular; does this use the same 40:13 counter?)

LinuxBIOS does not use 40:13 at the present time.  In fact the only
interface to the outside world (Besides the simple LinuxBIOS table)
visible when running etherboot is etherboot itself.

Care needs to be take because I'm not certain 40:13 is reliable.  I
know there are several cases that do BIOS calls to get this
information.  And the linux kernel explicitly does not read it with
a comment about it being unreliable.
 
> 4. Etherboot heap.  This gets placed in the largest contiguous block in 
>    memory.  At the moment, nothing uses it.

The multicast protocol drivers use it.  For most things static
allocation is sufficient and when static allocation works it is safer.
Since packets can come in a random order during multicast reception we
may not have enough information unless we allocate a static buffer for
the file we are downloading.

> 5. Allocated base memory, from 640K down to [40:13]k.  Includes the BIOS 
>    EPDA.
> 
> The interaction of these five areas with prep_segment() has the following 
> problems:
> 
> a. No checks are made for (3).  prep_segment will happily trash the 
>    real-mode stack and _real_call will happily walk all over an area that
>    prep_segment has prepared.

Correct.  That was an after thought.
 
> b. If heap (4) or base memory (5) is allocated after the first call to 
>    prep_segment, it's possible that it will allocate memory that 
>    prep_segment() has already checked is free and prepared.

Correct.  So far all of the heap allocation is just for the multicast
case when we cannot incrementally load the image.  So it all happens
during processing of the first packet.

> In addition, there seems to be an assortment of other mechanisms that 
> programs may use to deal with memory allocation: BIOS PMM, Int 15,87, 
> E820-map reading a la Etherboot etc.  Currently Etherboot makes no attempt 
> to handle or cooperate with any of these; if anything other than Etherboot 
> is allocating memory then there's a high chance of a collision.
> 
> MEMDISK contains routines that, AFAICT, allow us to fake the E820 map, so 
> that we could mark as reserved any regions that we have allocated.  
> Questions:
> 
> 1. Would this be sufficient to prevent other entities treading on 
>    Etherboot?  (I'm guessing yes).

Likely.

> 2. Do we have to do all our memory allocation and E820 map modification 
>    *before* we allow any other entity that might allocate memory to 
>    execute?  (I'm guessing yes).

That would be tricky.  The current assumption is that etherboot owns
the machine.  We may need to rescan the map.

Given that in the worst case we don't know where a packet needs to go
into memory until we read it.  I would say the general etherboot case
is that it needs to do most of the allocation first.

> 3. What if another entity is also faking the E820 map?  (I'm guessing that
>    this is not a problem as long as the other entity also does (2)). 
> 
> 4. How do we avoid treading on other entities?  (In particular: the 
>    Intel UNDI driver seems to install something just above the 1MB mark.  
>    prep_segment() then obliterates this, because it can't tell that the
>    region is in use).

Very good question.  It looks like we need to rescan things (at least
in the UNDI driver case.

> 5. Would it take into account the BIOS PMM, if it exists?

Possibly.  I don't recall what that is...
If it is BIOS services for allocating memory above 1MB then that would
be a good idea.
 
> 6. How do we tidy up before passing control to the loaded program?  We 
>    don't always want to free everything; if it's a menu program then we 
>    need to keep Etherboot "hidden".

This is the biggest reason I dislike callbacks, there is a lot of
weird cooperation needed to get everything working.

> Comments, suggestions, ideas welcome.  This is currently the only thing 
> stopping the UNDI driver from working.  It needs to be sorted out before 
> Etherboot can reliably co-exist with other boot ROMs (e.g. PXE ROMs are 
> likely to install things into extended memory) and it's a prerequisite of 
> PXE-on-Etherboot.


I hope this helps.

Eric

Re: [Etherboot-developers] Bug in DHCP request packet format

[<ke...@us...>]

>In etherboot 5.1.8 NIC.C, this change:
>-------------------------------
>@@ -782,7 +782,7 @@
>  /* Append machine_info to end, in encapsulated option */
>  bp_vend = ip.bp.bp_vend + sizeof rfc1533_cookie + sizeof dhcpdiscover;
>  memcpy(bp_vend, dhcp_machine_info, DHCP_MACHINE_INFO_SIZE);
>- bp_vend += sizeof DHCP_MACHINE_INFO_SIZE;
>+ bp_vend += DHCP_MACHINE_INFO_SIZE;
>  *bp_vend++ = RFC1533_END;
> #endif /* NO_DHCP_SUPPORT */
>-------------------------------
>will correct a bug in the creation of the DHCP Request packet that
>effectively truncates dhcp_machine_info to first 4 bytes (I checked the
>assembly produced by the compiler).
>
>This bug seems to only affect the 5.1.x chain.

Many thanks. Committed to CVS.