etherboot-developers — Discussion of developer issues

[Etherboot-developers] Bug in DHCP request packet format

[Robb M. <ma...@ac...>]

In etherboot 5.1.8 NIC.C, this change:
-------------------------------
@@ -782,7 +782,7 @@
  /* Append machine_info to end, in encapsulated option */
  bp_vend = ip.bp.bp_vend + sizeof rfc1533_cookie + sizeof dhcpdiscover;
  memcpy(bp_vend, dhcp_machine_info, DHCP_MACHINE_INFO_SIZE);
- bp_vend += sizeof DHCP_MACHINE_INFO_SIZE;
+ bp_vend += DHCP_MACHINE_INFO_SIZE;
  *bp_vend++ = RFC1533_END;
 #endif /* NO_DHCP_SUPPORT */
-------------------------------
will correct a bug in the creation of the DHCP Request packet that
effectively truncates dhcp_machine_info to first 4 bytes (I checked the
assembly produced by the compiler).

This bug seems to only affect the 5.1.x chain.

Cheers,
Robb Main.

Re: [Etherboot-developers] UNDI driver

[Marty C. <md...@et...>]

On Saturday, May 24, 2003, at 03:46  PM, Michael Brown wrote:
> The i386 UNDI driver in Etherboot 5.1 CVS should now be functioning
> correctly.  Please test and give feedback.

This is excellent news, Michael! Congratulations.

>   make bin/undi.zfd0
> will give you a bootable floppy that should be able to detect any PXE
> ROMs, load the UNDI drivers and use them to boot as per normal.
> I haven't yet got around to testing chaining from PXE, but
>   make bin/undi.zpxe
> will give the appropriate test image, should anyone wish to try it out.
> There are no known issues for systems containing a single NIC and 
> single
> PXE ROM at this time.  (Multiple NICs / multiple PXE ROMs may have
> problems; please let me know.)

I am building the appropriate test environment this morning and hope to 
begin testing later today. I'll report back with my results.

> Please do give feedback; this driver has to work with any PXE
> implementation, and I have only Intel's to play with.
> Michael

I have a 3Com 3C905C-xxx with PXE in flash that I use especially for 
occasions like this. I should send you one :)

Thanks for doing this.

Marty


-- 
     Try: http://rom-o-matic.net/ to make Etherboot images instantly.

    Name: Marty Connor
US Mail: Entity Cyber, Inc.; P.O. Box 391827;
          Cambridge, MA 02139; USA
   Voice: (617) 491-6935; Fax: (617) 491-7046
   Email: md...@et...
     Web: http://www.etherboot.org/

[Etherboot-developers] UNDI driver

[Michael B. <mb...@fe...>]

The i386 UNDI driver in Etherboot 5.1 CVS should now be functioning
correctly.  Please test and give feedback.

  make bin/undi.zfd0

will give you a bootable floppy that should be able to detect any PXE 
ROMs, load the UNDI drivers and use them to boot as per normal.

I haven't yet got around to testing chaining from PXE, but

  make bin/undi.zpxe

will give the appropriate test image, should anyone wish to try it out.

There are no known issues for systems containing a single NIC and single 
PXE ROM at this time.  (Multiple NICs / multiple PXE ROMs may have 
problems; please let me know.)

Please do give feedback; this driver has to work with any PXE 
implementation, and I have only Intel's to play with.

Michael

[Etherboot-developers] Memory allocation

[Michael B. <mb...@fe...>]

Memory allocation within Etherboot seems to suffer from several problems 
at the moment.  As far as I can tell, we have the following memory ranges 
being used:

1. Etherboot text.  This will be at RELOCADDR (=0x20000) if -DRELOCATE is
   not defined, otherwise somewhere towards the top of available RAM.  
   (Will always be within an even megabyte, i.e. A20=0).

2. Etherboot protected-mode stack.  I think this is within the Etherboot 
   text, somebody please confirm/correct me on this.

3. Etherboot real-mode stack.  This is always at the top of free base 
   memory (as defined by 40:13), provided that Etherboot is the only thing 
   allocating base memory.

   NB. Other code may allocate base memory by changing the counter at
   40:13.  We should really recalculate the real-mode stack top each time
   we make a _real_call, based on the current contents of 40:13.  Is there 
   any reason why we can't do this?  (I'm thinking of LinuxBIOS in 
   particular; does this use the same 40:13 counter?)

4. Etherboot heap.  This gets placed in the largest contiguous block in 
   memory.  At the moment, nothing uses it.

5. Allocated base memory, from 640K down to [40:13]k.  Includes the BIOS 
   EPDA.

The interaction of these five areas with prep_segment() has the following 
problems:

a. No checks are made for (3).  prep_segment will happily trash the 
   real-mode stack and _real_call will happily walk all over an area that
   prep_segment has prepared.

b. If heap (4) or base memory (5) is allocated after the first call to 
   prep_segment, it's possible that it will allocate memory that 
   prep_segment() has already checked is free and prepared.


In addition, there seems to be an assortment of other mechanisms that 
programs may use to deal with memory allocation: BIOS PMM, Int 15,87, 
E820-map reading a la Etherboot etc.  Currently Etherboot makes no attempt 
to handle or cooperate with any of these; if anything other than Etherboot 
is allocating memory then there's a high chance of a collision.

MEMDISK contains routines that, AFAICT, allow us to fake the E820 map, so 
that we could mark as reserved any regions that we have allocated.  
Questions:

1. Would this be sufficient to prevent other entities treading on 
   Etherboot?  (I'm guessing yes).

2. Do we have to do all our memory allocation and E820 map modification 
   *before* we allow any other entity that might allocate memory to 
   execute?  (I'm guessing yes).

3. What if another entity is also faking the E820 map?  (I'm guessing that
   this is not a problem as long as the other entity also does (2)). 

4. How do we avoid treading on other entities?  (In particular: the 
   Intel UNDI driver seems to install something just above the 1MB mark.  
   prep_segment() then obliterates this, because it can't tell that the
   region is in use).

5. Would it take into account the BIOS PMM, if it exists?

6. How do we tidy up before passing control to the loaded program?  We 
   don't always want to free everything; if it's a menu program then we 
   need to keep Etherboot "hidden".


Comments, suggestions, ideas welcome.  This is currently the only thing 
stopping the UNDI driver from working.  It needs to be sorted out before 
Etherboot can reliably co-exist with other boot ROMs (e.g. PXE ROMs are 
likely to install things into extended memory) and it's a prerequisite of 
PXE-on-Etherboot.

Michael

Re: [Etherboot-developers] Relocation problems

[<ebi...@ln...>]

Michael Brown <mb...@fe...> writes:

> > As far as A20 status goes, I'm thinking of modifying relocate.c so that it 
> > relocates Etherboot to a location with A20=0, so that Etherboot remains 
> > accessible even if something screws up the A20 status.  Does this sound 
> > like a good idea?
> 
> I've implemented this now and it's checked in.
> 
> Rather nicely, it's "solved" my problem; I can now make the UNDI API calls
> even from a relocated Etherboot.  However, I suspect that this is
> coincidence; the relocation target is now a megabyte away from where it
> was previously and so it's probably now just happens not to be in the
> place that the UNDI driver clobbers with its int 15,87 call.

O.k.  It looks like we need to hook the int 15 memory size calls then
so it looks like the memory etherboot is using is reserved.

Eric

Re: [Etherboot-developers] Relocation problems

[Michael B. <mb...@fe...>]

> As far as A20 status goes, I'm thinking of modifying relocate.c so that it 
> relocates Etherboot to a location with A20=0, so that Etherboot remains 
> accessible even if something screws up the A20 status.  Does this sound 
> like a good idea?

I've implemented this now and it's checked in.

Rather nicely, it's "solved" my problem; I can now make the UNDI API calls
even from a relocated Etherboot.  However, I suspect that this is
coincidence; the relocation target is now a megabyte away from where it
was previously and so it's probably now just happens not to be in the
place that the UNDI driver clobbers with its int 15,87 call.

Michael

Re: [Etherboot-developers] Relocation problems

[Michael B. <mb...@fe...>]

> There is a potential for the status of the A20 line to be
> significant.
> <snip>
> It is possible the undi driver is allocating high memory with
> some BIOS call and stomping etherboot.

Thanks for this; I've found a few int 15,24 calls (enable/disable A20) and 
some int 15,87 calls (move block to/from extended memory) in the UNDI ROM.  
This looks to be the most likely culprit.

As far as A20 status goes, I'm thinking of modifying relocate.c so that it 
relocates Etherboot to a location with A20=0, so that Etherboot remains 
accessible even if something screws up the A20 status.  Does this sound 
like a good idea?

> But my gut reaction is that something somewhere is missing
> a virt_to_phys or a phys_to_virt..

I don't think this can be the case; these are non-identity mappings even
without -DRELOCATE.

> But in general when I am faced with these things I take my
> serial debugging prints statements and instrument the code and find
> out exactly where it is failing so I can do something about
> the failure.
> My debug print statements are attached below.  The expand inline
> and increase the code size but by doing so have a very minimal
> register impact and not stack impact so can be used in all
> kinds of situations.  The code works find as either 16bit or 32bit
> code.  

Thanks.  I think it would be handy to have those available all the time; 
why not add them in as a .h file in Etherboot?

Michael

Re: [Etherboot-developers] pxe

[<ebi...@ln...>]

Michael Brown <mb...@fe...> writes:

> On Thu, 22 May 2003, Danny Braniss wrote:
> > > There is now a copy of pxe.h from FreeBSD in Etherboot CVS, under 
> > > arch/i386/include.  I used it when writing the Etherboot UNDI driver.  
> > > It's been edited slightly; I corrected several typos and one alignment 
> > > bug.  If this is the only header required, then we've now got it.
> > hum, i think i sent out a message this morning, but it could have been
> > before coffee, so if this is a repost - sorry
> 
> Did you send it to etherboot-developers-admin instead of 
> etherboot-developers?
> 
> > whith some minor twiks i managed to compile under FreeBSD, but there is
> > a minor unresolved:
> > 	main.o(.text+0x1459): undefined reference to `initsp'
> > and i found:
> > 	./src/pcbios.S:.globl initsp
> > i think some initialization is missing, but since my knowledeg of x86
> > assembler is zero, i have no idea how to fix this

Wrong track.  initsp was deleted.  How the .globl initsp survived
is beyond me.

> You seem to be using the 5.0 branch - is this deliberate?  (The BSD header 
> I was referring to has been placed into the 5.1 branch.)

One branch or the other should not matter too much, in this
regard.

The two pieces of inline assembly in the pxe code in 5.1 are
both incorrect.  For an old enough version of 5.0 the code
works.  In particular 5.0.x one version before the inclusion
of the code.

Etherboot does not use inline assembly for this kind of thing
and that is what is causing the problem.  Instead these hooks
are abstracted into subroutines, and just written in assembler.

The important part being the code is written as separate subroutines
with a clearly defined purpose.

So two things are needed.
1) A routine that will start a pxe image with the proper arguments.
   To replace this junk of assembly:
   Something similar to xstart16 is needed, 
   or possibly xstart16 can be generalized.
                
                __asm__ __volatile__    (
                        "movb $1, pxeemu_nbp_active\n"
                        "call _prot_to_real\n"
                        ".code16\n"
                        "movw %0, %%ax\n"                    
                        "movw %%ax, %%es\n"
                        "movl %1, %%ebx\n"
                        "ljmp $0x0,$0x7c00\n"
                        ".code32\n"
                        : : "i" (RELOC >> 4), 
                        "g" (((vm_offset_t)&pxenv) - RELOC));
  
2) A routine to generate a thunk that can be used to
   reenter etherboot in 32bit mode, and transfer
   control to the pxeemu handler.  And eventually we will 
   need code to generate 16bit thunks as well.  That
   will allow us to replace this bit of assembly.

               /* Switch Stacks */
                __asm__("xorl %%eax, %%eax\n"
                        "movw initsp, %%ax\n"
                        "addl %0, %%eax\n"
                        "movl %%eax, %%esp\n"
                        : : "i" (RELOC));

Basically those two pieces of inline assembly are a maintenance
disaster because: 
- What they are trying to do is not well abstracted.
- They are not doing things in the usual etherboot style.
- The code cannot even be compiled on anything except FreeBSD.
  With the appropriate magic headers.

For 5.1 2 additional pieces need to happen:
- The pxenv structure needs to moved lower in memory if
  it is to work with -DRELOCATE enabled.
- Etherboot needs to hook int 0x15 and reserve the memory
  etherboot is living in.  This may just be a cleanliness
  issue.

Eric

Re: [Etherboot-developers] Relocation problems

[<ebi...@ln...>]

Michael Brown <mb...@fe...> writes:

> I'm having big problems getting the UNDI driver to work with relocation.  
> An UNDI API call that works fine when the Etherboot code is in base memory 
> fails when called with the *exact* same parameters (i.e. all the real-mode 
> segment:offset addresses are the same) if the Etherboot code has been 
> relocated out of base memory.  This is the very first UNDI API call: the 
> one that installs the UNDI driver into base memory (the base memory having 
> been allocated using allot_base_memory()).


> Within the "10: .code16" section of a real-mode call, are there any 
> registers (other than %ss and %sp) that need to be preserved in order for 
> the return from real mode to function correctly?

There should not be but it is possible there is some architecture
state we depend on.  To a certain mild extent we can even
tolerate changes in %sp so long as we return to that stack.
 
> Any other ideas very welcome; I've run out.

If I had the time I would look but baring that:

I have seen alignment issues depending on the relocation address
chosen.

There is a potential for the status of the A20 line to be
significant.

I once saw an instruction that was invalid on a 486 but
worked on later processors.

It is possible the undi driver is allocating high memory with
some BIOS call and stomping etherboot.

But my gut reaction is that something somewhere is missing
a virt_to_phys or a phys_to_virt..

But in general when I am faced with these things I take my
serial debugging prints statements and instrument the code and find
out exactly where it is failing so I can do something about
the failure.

My debug print statements are attached below.  The expand inline
and increase the code size but by doing so have a very minimal
register impact and not stack impact so can be used in all
kinds of situations.  The code works find as either 16bit or 32bit
code.  

Note: I have this bad habit of not believing I have registers
when I am coding in assembly.  

It will be very interesting to find out if the undi initialization
even attempts to return. 

Eric


#if 1
/* Base Address */
#define TTYS0_BASE	0x3f8
/* Data */
#define TTYS0_RBR (TTYS0_BASE+0x00)
#define TTYS0_TBR (TTYS0_BASE+0x00)
/* Control */
#define TTYS0_IER (TTYS0_BASE+0x01)
#define TTYS0_IIR (TTYS0_BASE+0x02)
#define TTYS0_FCR (TTYS0_BASE+0x02)
#define TTYS0_LCR (TTYS0_BASE+0x03)
#define TTYS0_MCR (TTYS0_BASE+0x04)

#define TTYS0_DLL (TTYS0_BASE+0x00)
#define TTYS0_DLM (TTYS0_BASE+0x01)
/* Status */
#define TTYS0_LSR (TTYS0_BASE+0x05)
#define TTYS0_MSR (TTYS0_BASE+0x06)
#define TTYS0_SCR (TTYS0_BASE+0x07)

#define TTYS0_BAUD 9600
#define TTYS0_DIV  (115200/TTYS0_BAUD)
#define TTYS0_DIV_LO	(TTYS0_DIV&0xFF)
#define TTYS0_DIV_HI	((TTYS0_DIV >> 8)&0xFF)

#if ((115200%TTYS0_BAUD) != 0)
#error Bad ttyS0 baud rate
#endif

#define TTYS0_INIT	\
	/* disable interrupts */	\
	movb	$0x00, %al		; \
	movw	$TTYS0_IER, %dx		; \
	outb	%al, %dx		; \
					; \
	/* enable fifos */		\
	movb	$0x01, %al		; \
	movw	$TTYS0_FCR, %dx		; \
	outb	%al, %dx		; \
					; \
	/* Set Baud Rate Divisor to TTYS0_BAUD */	\
	movw	$TTYS0_LCR, %dx		; \
	movb	$0x83, %al		; \
	outb	%al, %dx		; \
					; \
	movw	$TTYS0_DLL, %dx		; \
	movb	$TTYS0_DIV_LO, %al	; \
	outb	%al, %dx		; \
					; \
	movw	$TTYS0_DLM, %dx		; \
	movb	$TTYS0_DIV_HI, %al	; \
	outb	%al, %dx		; \
					; \
	movw	$TTYS0_LCR, %dx		; \
	movb	$0x03, %al		; \
	outb	%al, %dx
	

	/* uses:	ax, dx */
#define TTYS0_TX_AL		\
	mov	%al, %ah	; \
9:	mov	$TTYS0_LSR, %dx	; \
	inb	%dx, %al	; \
	test	$0x20, %al	; \
	je	9b		; \
	mov	$TTYS0_TBR, %dx	; \
	mov	%ah, %al	; \
	outb	%al, %dx

	/* uses:	ax, dx */
#define TTYS0_TX_CHAR(byte)	\
	mov	byte, %al	; \
	TTYS0_TX_AL

#define TTYS0_TX_HEX32(lword)	\
	mov	lword, %eax	; \
	shr	$28, %eax	; \
	add	$'0', %al	; \
	cmp	$'9', %al	; \
	jle	9f		; \
	add	$39, %al	; \
9:				; \
	TTYS0_TX_AL		; \
				; \
	mov	lword, %eax	; \
	shr	$24, %eax	; \
	and	$0x0f, %al	; \
	add	$'0', %al	; \
	cmp	$'9', %al	; \
	jle	9f		; \
	add	$39, %al	; \
9:				; \
	TTYS0_TX_AL		; \
				; \
	mov	lword, %eax	; \
	shr	$20, %eax	; \
	and	$0x0f, %al	; \
	add	$'0', %al	; \
	cmp	$'9', %al	; \
	jle	9f		; \
	add	$39, %al	; \
9:				; \
	TTYS0_TX_AL		; \
				; \
	mov	lword, %eax	; \
	shr	$16, %eax	; \
	and	$0x0f, %al	; \
	add	$'0', %al	; \
	cmp	$'9', %al	; \
	jle	9f		; \
	add	$39, %al	; \
9:				; \
	TTYS0_TX_AL		; \
				; \
	mov	lword, %eax	; \
	shr	$12, %eax	; \
	and	$0x0f, %al	; \
	add	$'0', %al	; \
	cmp	$'9', %al	; \
	jle	9f		; \
	add	$39, %al	; \
9:				; \
	TTYS0_TX_AL		; \
				; \
	mov	lword, %eax	; \
	shr	$8, %eax	; \
	and	$0x0f, %al	; \
	add	$'0', %al	; \
	cmp	$'9', %al	; \
	jle	9f		; \
	add	$39, %al	; \
9:				; \
	TTYS0_TX_AL		; \
				; \
	mov	lword, %eax	; \
	shr	$4, %eax	; \
	and	$0x0f, %al	; \
	add	$'0', %al	; \
	cmp	$'9', %al	; \
	jle	9f		; \
	add	$39, %al	; \
9:				; \
	TTYS0_TX_AL		; \
				; \
	mov	lword, %eax	; \
	and	$0x0f, %al	; \
	add	$'0', %al	; \
	cmp	$'9', %al	; \
	jle	9f		; \
	add	$39, %al	; \
9:				; \
	TTYS0_TX_AL


#define DEBUG(x) TTYS0_TX_CHAR($x) ;  TTYS0_TX_CHAR($'\r') ;  TTYS0_TX_CHAR($'\n')
#define DEBUG_TX_HEX32(x) TTYS0_TX_HEX32(x); TTYS0_TX_CHAR($'\r') ;  TTYS0_TX_CHAR($'\n')
#endif

[Etherboot-developers] Relocation problems

[Michael B. <mb...@fe...>]

I'm having big problems getting the UNDI driver to work with relocation.  
An UNDI API call that works fine when the Etherboot code is in base memory 
fails when called with the *exact* same parameters (i.e. all the real-mode 
segment:offset addresses are the same) if the Etherboot code has been 
relocated out of base memory.  This is the very first UNDI API call: the 
one that installs the UNDI driver into base memory (the base memory having 
been allocated using allot_base_memory()).

The problem is not intrinsic to -DRELOCATE; I have tried compiling with 
-DRELOCATE and an edited version of core/relocate.c that forces the 
relocation target address to be the original address (i.e. no move).  This 
works, proving that there is no problematic code included by -DRELOCATE.  
The Etherboot UNDI driver itself is -DRELOCATE-agnostic; the only call 
that cares about relocation (undi_transmit()) sorts itself out at run-time 
rather than compile-time and, in any case, it's not even getting that far.

I am certain that the correct parameters are being passed to the UNDI API 
routine.

Within the "10: .code16" section of a real-mode call, are there any 
registers (other than %ss and %sp) that need to be preserved in order for 
the return from real mode to function correctly?

Any other ideas very welcome; I've run out.

Michael

Re: [Etherboot-developers] pxe

[Michael B. <mb...@fe...>]

On Thu, 22 May 2003 prl...@sy... wrote:
> Thinking aloud, how feasible would it be to make a quasi interrupt
> driven driver, on top of the (existing) polling drivers, which could be
> exported as UNDI?  I can't help noting that this *would* be emulation as
> I understand the term. :-) I do realise that this would be an extremely
> pointless project if NBP actually needs it - it is not a practical
> suggestion, but once the PXE skeleton is in place and stable I can
> imagine that someone might try.

Two main obstacles that I can see:

1. The hardware interrupts need to be enabled.  This requires all the 
   drivers to be modified.

2. poll() needs to be capable of being called as part of an interrupt 
   service routine.

We don't actually need to implement ISRs, 8259A-handling code etc.; under 
the UNDI model the responsibilities are divided as follows:

UNDI driver: enable the interrupt on the NIC, pass back the IRQ number to 
the protocol driver ("protocol driver" in PXE terminology meaning the 
program that is using the UNDI driver).

Protocol driver: install an ISR for the specified IRQ and enable it.

Protocol driver: handle interrupts.  Call the UNDI driver's "ISR"  
routine to determine whether or not this interrupt has originated from our
NIC.  If so, set a flag and exit the interrupt handler.

Protocol driver: notice the flag set by the interrupt handler, call the 
UNDI driver's "ISR" routine again to read out the actual data.


In practice, we might be able to get away with the following modifications 
to all drivers:

1. In probe(): enable interrupts on the NIC, pass back the IRQ (probably
   via an extra member in the "dev" structure).

2. Split poll() into two routines:

a) poll(); a routine that *very quickly* determines whether or not there 
   is a packet waiting and does nothing else.  Must be able to be called 
   as part of an interrupt service routine.

b) receive(): receive the actual data.


This wouldn't allow us to implement the full UNDI API (with calls like 
"set MAC address", "get link speed" etc.), but it would be enough to cover 
all the calls that are essential for initialisation and transmitting / 
receiving data.  We could always hack round the other ones (e.g. return 
success for "set MAC address" as long as it's being set to it's current 
value anyway).

It's starting to look easier each time I think about it.

One interesting case would be PXEAPI on top of Etherboot on top of UNDI.  
This would involve having two !PXE structures in memory, which could 
really confuse a NBP.  I imagine we'd have to corrupt the UNDI driver's 
!PXE structure so that it still functioned but wouldn't be detected.

Michael

Re: [Etherboot-developers] pxe

[<prl...@sy...>]

> It's unlikely that Etherboot will ever implement the full PXE API.  
> Implementing the TFTP and UDP layer calls would be relatively 
> straightforward, but implementing the UNDI API calls would require a 
> significant restructuring of the internal Etherboot driver API.  This 
> would offer *no* benefits other than being able to implement the full PXE 
> API, and would have significant downsides in that Etherboot drivers would 
> have to become a lot more complicated.  (For a start, they'd have to use 
> interrupts instead of polling.)

> PXE emulation, in the sense of implementing part of the PXE API, is 
> probably a better goal than a full PXE implementation.

You guys seem to use the word "emulate" in an odd way.

http://dictionary.reference.com/search?q=emulate

I agree with the logic of what you are saying, Michael, just noting that
"emulation" seems like the wrong word for "partial implementation".

A partial implementation would be fine, I'm sure: NBPs are likely use
the higher level APIs, as Eric pointed out in an earlier thread. Almost
anyone else will be using the "features" of PXE (unlike Etherboot with a
UNDI driver, which is trying to *replace* them).

Intel might take a dim view of using "PXE" for something incomplete,
though, even if it is effective.  How about PSKE? Or POXE? PIKI?

Thinking aloud, how feasible would it be to make a quasi interrupt
driven driver, on top of the (existing) polling drivers, which could be
exported as UNDI?  I can't help noting that this *would* be emulation as
I understand the term. :-) I do realise that this would be an extremely
pointless project if NBP actually needs it - it is not a practical
suggestion, but once the PXE skeleton is in place and stable I can
imagine that someone might try.

Re: [Etherboot-developers] pxe

[Michael B. <mb...@fe...>]

On Thu, 22 May 2003, Danny Braniss wrote:
> > There is now a copy of pxe.h from FreeBSD in Etherboot CVS, under 
> > arch/i386/include.  I used it when writing the Etherboot UNDI driver.  
> > It's been edited slightly; I corrected several typos and one alignment 
> > bug.  If this is the only header required, then we've now got it.
> hum, i think i sent out a message this morning, but it could have been
> before coffee, so if this is a repost - sorry

Did you send it to etherboot-developers-admin instead of 
etherboot-developers?

> whith some minor twiks i managed to compile under FreeBSD, but there is
> a minor unresolved:
> 	main.o(.text+0x1459): undefined reference to `initsp'
> and i found:
> 	./src/pcbios.S:.globl initsp
> i think some initialization is missing, but since my knowledeg of x86
> assembler is zero, i have no idea how to fix this

You seem to be using the 5.0 branch - is this deliberate?  (The BSD header 
I was referring to has been placed into the 5.1 branch.)

Michael

Re: [Etherboot-developers] pxe

[Michael B. <mb...@fe...>]

> > The "E" in PXE means Environment; it is an execution environment
> > defined by Intel's spec. When Etherboot provides a PXE environment, it
> > really will BE a PXE implementation - not an "emulation". Code really
> > will have been booted by a "PXE boot-rom", namely Etherboot; it does
> > not have to be written by Intel to be PXE.
> Given the holes in the implementation calling it emulation is
> probably a fair description.  But then again given the holes
> in the PXE spec/practice we might qualify as implementing it.

It's unlikely that Etherboot will ever implement the full PXE API.  
Implementing the TFTP and UDP layer calls would be relatively 
straightforward, but implementing the UNDI API calls would require a 
significant restructuring of the internal Etherboot driver API.  This 
would offer *no* benefits other than being able to implement the full PXE 
API, and would have significant downsides in that Etherboot drivers would 
have to become a lot more complicated.  (For a start, they'd have to use 
interrupts instead of polling.)

PXE emulation, in the sense of implementing part of the PXE API, is 
probably a better goal than a full PXE implementation.

> The code degraded before it made it into a production release.
> There may be a point in the cvs archive where the code works though.
> The core of the code looks o.k. but there are a few places where
> the code is likely to still have issues.
> The biggest problem is that the code depends on bsd specific headers
> and will never compile under anything else.  If you could fix
> that the code is more likely to be maintained.

There is now a copy of pxe.h from FreeBSD in Etherboot CVS, under 
arch/i386/include.  I used it when writing the Etherboot UNDI driver.  
It's been edited slightly; I corrected several typos and one alignment 
bug.  If this is the only header required, then we've now got it.

Michael

Re: [Etherboot-developers] pxe

[H. P. A. <hp...@zy...>]

prl...@sy... wrote:
>>I believe that is also the case for FreeBSD.  Though I am not
>>terribly fond of the way FreeBSD support is currently built.  There
>>is no future proofing in there.
> 
> 
> Open source should not depend on PXE.
> 
> 
>>Given the holes in the implementation calling it emulation is
>>probably a fair description.  But then again given the holes
>>in the PXE spec/practice we might qualify as implementing it.
> 
> 
> The implementation may be incomplete - but one *emulates* something
> else. My point was that Etherboot would not "emulate" PXE or be somehow
> "artifical" or suboptimal - quite the opposite...
> 
> Bochs, for instance, does not "emulate" BIOS calls - it implements them,
> just as Award does. It may frequently sit on top of emulated "hardware",
> but that's different - put it on ADLO and LinuxBIOS and I think it is
> fair to say that there is no "emulation" going on, or at least no more
> than a "normal" BIOS on the mobo.
> 

The best phrasing would be that Etherboot would implement a PXE subset. 
  Eventually it might implement the full spec, but that is a large effort.

	-hpa

Re: [Etherboot-developers] pxe

[<prl...@sy...>]

> I believe that is also the case for FreeBSD.  Though I am not
> terribly fond of the way FreeBSD support is currently built.  There
> is no future proofing in there.

Open source should not depend on PXE.

> Given the holes in the implementation calling it emulation is
> probably a fair description.  But then again given the holes
> in the PXE spec/practice we might qualify as implementing it.

The implementation may be incomplete - but one *emulates* something
else. My point was that Etherboot would not "emulate" PXE or be somehow
"artifical" or suboptimal - quite the opposite...

Bochs, for instance, does not "emulate" BIOS calls - it implements them,
just as Award does. It may frequently sit on top of emulated "hardware",
but that's different - put it on ADLO and LinuxBIOS and I think it is
fair to say that there is no "emulation" going on, or at least no more
than a "normal" BIOS on the mobo.

Re: [Etherboot-developers] pxe

[<ebi...@ln...>]

prl...@sy... writes:

> If you have a real need for PXE, that's fine, but there is a common
> misconception that "PXE" is another term for "network boot". Not so -
> if you just want Linux, PXE is just unnecessary complication.

I believe that is also the case for FreeBSD.  Though I am not
terribly fond of the way FreeBSD support is currently built.  There
is no future proofing in there.
 
> > 	im interesetd in etherboot setting the env. so that the loaded
> > program thinks it got loaded by a pxe boot-rom, actualy a minimum
> > pxe emulation is great - just open/close/udpread/uptwrite - not all
> > the jazz of wfm.
> 
> The "E" in PXE means Environment; it is an execution environment
> defined by Intel's spec. When Etherboot provides a PXE environment, it
> really will BE a PXE implementation - not an "emulation". Code really
> will have been booted by a "PXE boot-rom", namely Etherboot; it does
> not have to be written by Intel to be PXE.

Given the holes in the implementation calling it emulation is
probably a fair description.  But then again given the holes
in the PXE spec/practice we might qualify as implementing it.

> I was trying to make the point that in addition to the desirable
> feature of supporting NBPs on top of Etherboot, a PXE boot rom can
> invoke Etherboot as an NBP (and soon use UNDI - note the separate
> thread on Michael Brown's progress). Just be aware that references to
> PXE in the context of Etherboot may not be want you want.
> 
> I *believe* that the existing code for FreeBSD was designed to do
> exactly what you are describing, but it may have degraded in recent
> versions if no-one has stressed it. 

The code degraded before it made it into a production release.
There may be a point in the cvs archive where the code works though.
The core of the code looks o.k. but there are a few places where
the code is likely to still have issues.

The biggest problem is that the code depends on bsd specific headers
and will never compile under anything else.  If you could fix
that the code is more likely to be maintained.

Eric

Re: [Etherboot-developers] pxe

[<prl...@sy...>]

> ahh, but i did mean pxeboot :-)

> I'm very much interested in booting via PXE, and FreeBSD specially
> (hence pxeboot), and could put in some time, but my x86 assembly
> skils are close do zero, but i have some resources at my disposal.

OK - it seemed you were talking about a machine that ran Linux, which
is the main focus of Etherboot. Just bear in mind that the initial
target for development of PXE on Etherboot will probably be pxelinux.

If you have a real need for PXE, that's fine, but there is a common
misconception that "PXE" is another term for "network boot". Not so -
if you just want Linux, PXE is just unnecessary complication.

> 	im interesetd in etherboot setting the env. so that the loaded
> program thinks it got loaded by a pxe boot-rom, actualy a minimum
> pxe emulation is great - just open/close/udpread/uptwrite - not all
> the jazz of wfm.

The "E" in PXE means Environment; it is an execution environment
defined by Intel's spec. When Etherboot provides a PXE environment, it
really will BE a PXE implementation - not an "emulation". Code really
will have been booted by a "PXE boot-rom", namely Etherboot; it does
not have to be written by Intel to be PXE.

I was trying to make the point that in addition to the desirable
feature of supporting NBPs on top of Etherboot, a PXE boot rom can
invoke Etherboot as an NBP (and soon use UNDI - note the separate
thread on Michael Brown's progress). Just be aware that references to
PXE in the context of Etherboot may not be want you want.

I *believe* that the existing code for FreeBSD was designed to do
exactly what you are describing, but it may have degraded in recent
versions if no-one has stressed it. If you have having problems,
consider reverting to an earlier version of Etherboot at the point
where the FreeBSD stuff was added, since I believe that was reported
as working: look through list archives to track what happened. I don't
know if the original contributor of that code is still on the list,
but you could try contacting him.

> i can put the etherboot image on it's root partition (it has a micro
> disc + some flash memory), or it can boot from a floppy, i'll check
> that asap, just to make sure it's loaded ok. btw, how - if at all -
> can i use lilo to boot etherboot?

Get a LILO Etherboot image from rom-o-matic, put it somewhere on a
partition LILO where can see it and add an entry in lilo.conf.  It's a
very convenient way to prototype boot environments.

Re: [Etherboot-developers] UNDI driver

[Michael B. <mb...@fe...>]

> 1. Add arch/i386/firmware/pcbios/basemem.c, #ifdeffed with PCBIOS, 
>    containing routines to allocate base memory in the BIOS-expected way 
>    (i.e. by simply updating the value at 40:13).

This is now in place.  I had great fun tracking down why my allocated 
memory was being trodden on every time I did a printf().  Eventually 
discovered that get_memsizes() resets the position of the real mode stack 
to be the top of free base memory, which meant that my allocated base 
memory was being written all over as soon as _real_call got called.  Now 
fixed in that I adjust real_mode_stack each time I allocate or free 
memory.

Question: is there any reason why _real_call can't just calculate the 
stack base each time (by reading 40:13)?  (This would work with PCBIOS, 
but I don't know about LinuxBIOS.)

Michael

[Etherboot-developers] manufacturer of garments and bags in China

[Rachel <se...@pu...>]

Dear Sir or Madam,  

I have the pleasure to know your esteemed corp. 

We are a manufacturer & exporter of garments and bags in Quanzhou, China.

I think we can cooperate and supply you with garments as you need.  

The following is some introductions about our company.

       Set up: 1988

       Type: manufacturer & exporter

       Product: knitted garments and bags

       Employees: 1300 persons ( garments factory: 500      bags factory: 800)

       Product data:       

                                  product (main items)                      capacity(/year)

                                  brief                                    2,000,000dzs    

                                  baby body                                1,800,000dzs

                                  boxer short                                200,000dzs

                                  pajama                                      50,000dzs

                                  soft bag                                 1,500,000pcs

                                  hard bag                                   500,000pcs

Mimn order:  300dzs for garments

Payment:  irrevocable L/C at sight

Bank: BANK OF CHINA


Our garment factory mainly specialize in Lady's and men's underwear,

children's wear, baby's wear, pajama, boxer shorts, T-shirt, etc. 

The materials we often use are cotton, T/C, Polyester, Polyamide, 

Elasthan, and Polyamide. Our products are design with PAD system, 

produced with advanced equipment, processed in highly quality control 

system with seasoned workmanship and high efficiency. Our main market 

is Europe, Australia, Japan and America. We also accept the orders designed 

and required by costumers. You can see some pictures of our samples through 

our web http://www.senwer.com. (For more pictures in your interesting, 

pls kindly contact us directly). 
 

Our bag factory was founded in 1988, too. We produce all kinds of bags, 

including suitcase, backpack, travel bag, shoulder bag, sport bag, trolley, 

camera bag, tote bag, school bag, computer case, luggage,waist bag, notecase, etc. 

And the goods have met a great favor in the Europe countries, Australia 

and America because of their good quality, beautiful design and competitive price.


Thank you very much. Hope you will give us an opportunity to do 

business together and we will try our level best to fulfill your present 

requirement. Should you therefore need any more details for your 

clarification, pls do not hesitate to contact us. And you are welcome 

to visit our factories.


With best regards


Rachel Wang

Mob:0086-13960286700
E-mail:ra...@se...


Jason Chen

Mob:0086-13959893400
E-mail: jas...@se...


Vicki Wang

Mob:0086-13960228599
E-mail: vi...@se...


-----------------------------------------------------------------------------

SENWER GARMENTS CO., LTD.

ADD: Room F202, Fugui Renjia Building, Liuguan Road, Quanzhou, Fujian, China.

Tel: 0086-595-2506700    Fax: 0086-595-2563400    P.C.:362000

Http://www.senwer.com    E-mail: se...@pu...

-----------------------------------------------------------------------------

Re: [Etherboot-developers] pxe

[<prl...@sy...>]

> 	From the docs it's not clear, and from the sources, im still trying
> to compile under FreeBSD, I have some way to go :-), so before I loose
> myself, the short Q:
> 	can/could etherboot be used to boot pxeboot? 
> the long way:
> 	I have this Matchbox PC that someone dropped on my table, and
> it runs Linux, and im trying to make it boot over the network, but
> the NIC (Network Interface Chip :-), does not have PXE boot, it'a a CS89x0.
> 	I see some emulate pxe for freebsd stuff but there seems to be
> some parts missing.

I don't know pxeboot (do you mean pxelinux?)

The direct answer for almost any PXE Network Boostrap Program (NBP) on
top of Etherboot is "not yet".  Someone contributed code for a specific
FreeBSD case, so don't waste your time unless you are also using
FreeBSD.  There is considerable motivation for completing enough PXE
support for common NBPs (NTLOADER, pxelinux, bpbatch), so it seems
likely to happen soon. If you'd like to help, that would be great.

Note that Etherboot-on-PXE is being completed just now - don't confuse
this with PXE-on-Etherboot.

But you don't need to bother with PXE if the only OS you need is Linux.
Etherboot is designed to boot Linux directly; PXE is NOT required as
long as you have something to boot Etherboot from, such as a NIC ROM, a
modular BIOS, a disk (or disk like entity). So, if you plan to getting
this PC to network boot at all (PXE or not) you need some way to get
Etherboot on it; get an image with the cs89x0.o driver from rom-o-matic
in whatever form suits you.

[Etherboot-developers] pxe

[Danny B. <da...@cs...>]

hi all,
	From the docs it's not clear, and from the sources, im still trying
to compile under FreeBSD, I have some way to go :-), so before I loose
myself, the short Q:
	can/could etherboot be used to boot pxeboot? 
the long way:
	I have this Matchbox PC that someone dropped on my table, and
it runs Linux, and im trying to make it boot over the network, but
the NIC (Network Interface Chip :-), does not have PXE boot, it'a a CS89x0.
	I see some emulate pxe for freebsd stuff but there seems to be
some parts missing.

	thanks,
		danny

Re: [Etherboot-developers] UNDI driver

[Michael B. <mb...@fe...>]

> In etherboot we currently keep track of which memory is available for
> allocation.  See the e820 structure and it's kin.  We refuse
> to step on any memory that is listed as used.
> <snip>
> We currently have routines to allocate memory in a general fashion
> but the heap is placed above 1M.   So we call allocate bounce
> buffers but the below 1M case still needs to be handled.  It should
> be relatively straight forward however.
> <snip>
> The important point is that low addresses especially those below
> 1M are precious because the loaded image can count on them being
> available.

An image loading via PXE would not be able to count on them being
available; PXE drivers expect to be placed at the top of base memory.  By
using up base memory in the case of the Etherboot UNDI driver, we're doing
no worse than PXE does.  We don't have a choice about loading the UNDI
driver under 1M; some of the API calls have to be made in real mode.

I'm proposing the following solution, based on discussion thus far:

1. Add arch/i386/firmware/pcbios/basemem.c, #ifdeffed with PCBIOS, 
   containing routines to allocate base memory in the BIOS-expected way 
   (i.e. by simply updating the value at 40:13).

2. Add a check in prep_segment() in core/osloader.c, also #ifdeffed with
   PCBIOS, that will check that the segment being prepared doesn't overlap 
   the BIOS allocated low memory area.

3. The Etherboot UNDI driver will allocate space for the UNDI driver's 
   code and data segments in base memory, which is where it expects to be 
   put.  These have to go under 1M anyway; they are used in real mode.

4. The Etherboot UNDI driver will also allocate ETH_FRAME_LEN bytes of 
   base memory for the real-mode transmit buffer, plus a few tens of bytes 
   for structures like the UNDI API call parameter structure and the UNDI 
   transmit buffer descriptor.

Total base memory usage will be around 2K + UNDI driver data segment + 
UNDI driver code segment.

> > It's tiny; just a small wrapper around real_call.  I pass in a structure
> > that contains the address of the UNDI API routine and 3 words to push onto
> > the stack before making the API call.  _undi_call just switches to real 
> > mode, pushes the parameters onto the stack, jumps to the specified 
> > address, cleans up the stack and returns.  It's not far away from being a 
> > generic "call any real-mode routine" wrapper.  (It might be handy to have 
> > one of these, thinking about it.)
> 1) A generic "call any real-mode routine" wrappers suffers from 2
> things.  That handling the segment:offset addressing is a pain.
> That there are different calling conventions in the dos world.
> 2) real_call is as close to fully generic as you can get and not
> address those issues.
> 3) a generic interface for the calling convention used in the undi
> code would be fine.  Then you can make cute little stubs for the
> undi entry points in your driver.

(3) is what we've got at the moment.  For the sake of elegance, I'm 
proposing to split the tiny _undi_call routine into a separate 
undi_wrapper.S file and modify the Makefiles in some way to accommodate 
this.

Final comments / suggestions welcome; coding will probably happen this 
evening (UK time).

Michael

Re: [Etherboot-developers] UNDI driver

[Marty C. <md...@et...>]

On Monday, May 19, 2003, at 01:45  PM, Peter Lister wrote:
> It's annoying that neither Vasil nor I will get the glory, but I'm
> pleased that someone has finally worked up the steam. Well done,
> Michael.

Yes, Well done, Michael!

And don't worry, Peter and Vasil there's plenty of glory still to be 
had! :)

Michael's work should be taken much as the code you and Vasil did - it 
is an excellent facility that allows others to build other excellent 
hacks on top of it.

One of my current desires is that we get PXELINUX to work with 
Etherboot for LinuxWorld in August. Not because I need it personally, 
but because it will help a lot of people, and will demonstrate how a 
small Free Software project can create production-quality software that 
is useful in many environments.

It's really a lot of fun to do this kind of thing, and to work with 
people who care about the code :)

Thanks to everyone for making it so.

Marty

-- 
     Try: http://rom-o-matic.net/ to make Etherboot images instantly.

    Name: Marty Connor
US Mail: Entity Cyber, Inc.; P.O. Box 391827;
          Cambridge, MA 02139; USA
   Voice: (617) 491-6935; Fax: (617) 491-7046
   Email: md...@et...
     Web: http://www.etherboot.org/

Re: [Etherboot-developers] UNDI driver

[<ebi...@ln...>]

Michael Brown <mb...@fe...> writes:

> > > Question 1: Is there currently any method for allocating base memory?  I
> > > need space for three structures:
> > Memory should be taken off of the TOP of the 640K chunk of memory.
> > That is the only way ancient DOS memory reporting APIs can cope.
> > And in etherboot that should not be hard to report.  We might want to
> > think of bounce buffers to hold data that will eventually go there.
> 
> OK, but am I right in thinking that there is currently no mechanism for 
> doing this in Etherboot?  I remember HPA mentioning code in MEMDISK that 
> could be ripped out for this purpose.

There are 2 cases of interest.
1) Code using etherboot as a library
2) Etherboot using an UNDI driver.

Unless an UNDI driver is allowed to allocate memory for itself
we don't need the code in MEMDISK.  That code is only needed
if we are not going to sever as a library.

In etherboot we currently keep track of which memory is available for
allocation.  See the e820 structure and it's kin.  We refuse
to step on any memory that is listed as used.

As for bounce buffers it would simply take a specialized memcpy to
use in osloader.c  

We currently have routines to allocate memory in a general fashion
but the heap is placed above 1M.   So we call allocate bounce
buffers but the below 1M case still needs to be handled.  It should
be relatively straight forward however.

> > Losing 128K of memory will have some interesting affects on mknbi and
> > friends.
> 
> It won't always be 128K; the UNDI driver specifies how much space it needs 
> for its code and data segments, so we can allocate only what's necessary.

Agreed.  I am just thinking we will need to make certain it works when
we allocate 128K.

The important point is that low addresses especially those below
1M are precious because the loaded image can count on them being
available.
 
> > For the temporary can you just use real_call and a buffer allocated on
> > the stack?  It has provisions for coping with data left on the return
> > stack.   This does not solve the issue but it means we only need the
> > temporary stack.
> 
> The largest chunk of data I need is the whole transmit buffer; I need to 
> make this available to the real-mode UNDI API transmit call, which AFAIK 
> means copying it into base memory.  Is there enough room for this on the 
> stack and, if so, can you give me some pointers on how it works? 

Ok.  I have something that works for returning data in real_call
but I don't have a parameter for passing extra data on the stack.
That probably would not be too hard but I can certainly see the
value in just allocating a buffer from the top of low memory.

Since we are already allocating relatively fixed buffers it should
not be hard to allocate one more.

> I'll 
> need to know the addresses of the structures before going into the 
> assembler code, otherwise my assembler wrappers will have to be 
> significantly more complex.
> 
> > > Question 2: I have an assembly routine _undi_call that is a wrapper around 
> > > the real-mode UNDI API calls.  
> > Does _undi_call use real_call?
> 
> Yes.  I've checked in the code to pcbios.S temporarily, so that (a) people 
> can build and test the driver and (b) you can see exactly what the wrapper 
> currently does.
> 
> Please note that I've basically learnt x86 assembler over the past four 
> days, so there's bound to be a much simpler way of doing it.
> 
> > The big question is how small is undi_call?  If it is small enough
> > placing it pcbios.S may make sense.  As it could be packed into some
> > of the space usually used for alignment or what not.  But I do see
> > the point, if it has size putting it in every driver is probably
> > worse.
> 
> It's tiny; just a small wrapper around real_call.  I pass in a structure
> that contains the address of the UNDI API routine and 3 words to push onto
> the stack before making the API call.  _undi_call just switches to real 
> mode, pushes the parameters onto the stack, jumps to the specified 
> address, cleans up the stack and returns.  It's not far away from being a 
> generic "call any real-mode routine" wrapper.  (It might be handy to have 
> one of these, thinking about it.)

1) A generic "call any real-mode routine" wrappers suffers from 2
things.  That handling the segment:offset addressing is a pain.
That there are different calling conventions in the dos world.

2) real_call is as close to fully generic as you can get and not
address those issues.

3) a generic interface for the calling convention used in the undi
code would be fine.  Then you can make cute little stubs for the
undi entry points in your driver.

Hopefully I have pointed you enough in the correct direction for
a while.

Eric