etherboot-developers — Discussion of developer issues

[Etherboot-developers] Heads up.. 5.1.2+ won't build...

[Eric W B. <ebi...@ln...>]

Looking earlier today I noticed that I had removed all referenced
to _prot_to_real and _real_to_prot except two.  The references
that are used when etherboot starts up.

Just so no expects the latest cvs code to work flawlessly on
in a non LinuxBIOS configuration for the next couple of days...

Eric

Re: [Etherboot-developers] Q: Non-Standard printf?

[<ke...@us...>]

>> You mean it doesn't pay attention to the width specifier?
>Primarily.  Though I think I have seen a few other anomolies as well.

You should have seen it before the big rework circa 4.7.x. It used %b to
print uchar, %x to print short, and %X to print int. Klaus and I made it
conform to the glibc standard of %hhx, %hx and %x and the # modifier at
least. Also it printed into a fixed sized buffer so I was always worried
someone would print a long string. Now it doesn't matter how long a
string it's passed.

It made things a bit more familiar for people coming from mainstream
printf.  Direct compatibility with other sources wasn't a concern then,
although I probably should have made it handle width specifiers so that
porting from Linux drivers would have been slightly easier.

>> That was never
>> implemented, for space reasons. If you want to have better compatibility
>> without using up much code space you could get the printf to ignore the
>> width specifier.

Simplest would be to skip [0-9 .] after seeing %.

Re: [Etherboot-developers] Q: Non-Standard printf?

[Eric W B. <ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >I'm trying to understand why the printf in etherboot is non-standard.
> >If this is just something that happened or if this that has a good reason
> >behind it.
> >As I share code between projects my biggest bug is that code like:
> >printf("%04x\n", foo);  
> >Fails and I get really strange debugging output.
> 
> You mean it doesn't pay attention to the width specifier?
Primarily.  Though I think I have seen a few other anomolies as well.

> That was never
> implemented, for space reasons. If you want to have better compatibility
> without using up much code space you could get the printf to ignore the
> width specifier.

I will look.  Given that this is a code space thing.  I will look very carefully
before touching it.

Eric

Re: [Etherboot-developers] Q: Non-Standard printf?

[<ke...@us...>]

>I'm trying to understand why the printf in etherboot is non-standard.
>If this is just something that happened or if this that has a good reason
>behind it.
>As I share code between projects my biggest bug is that code like:
>printf("%04x\n", foo);  
>Fails and I get really strange debugging output.

You mean it doesn't pay attention to the width specifier? That was never
implemented, for space reasons. If you want to have better compatibility
without using up much code space you could get the printf to ignore the
width specifier.

Re: [Etherboot-developers] MOTD options?

[<ke...@us...>]

>> No. The menu options aren't needed either.
>
>Cool. Code I can just delete.
>
>I thought the code was left there to make the menu programs easier,
>to write.

Wait a sec. Should have had coffee first before replying. If they are
using the external menu, they will need to request those options. It
only takes up a bit of table space anyway.

>And the ANSIESC code in pcibios.S looks like it can also go.

That can go since nothing calls in within Etherboot.

[Etherboot-developers] Q: Non-Standard printf?

[Eric W B. <ebi...@ln...>]

I'm trying to understand why the printf in etherboot is non-standard.
If this is just something that happened or if this that has a good reason
behind it.

As I share code between projects my biggest bug is that code like:
printf("%04x\n", foo);  
Fails and I get really strange debugging output.

Not that I mind to much but it is sad to have code that works the first
time, to be bad simply because of debugging messages :)

Eric

Re: [Etherboot-developers] MOTD options?

[Eric W B. <ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >In 5.1.2 we are requesting MOTD options but are never using
> >them anyway.  Is this something we still want to request?
> 
> No. The menu options aren't needed either.

Cool. Code I can just delete.

I thought the code was left there to make the menu programs easier,
to write.

And the ANSIESC code in pcibios.S looks like it can also go.

Eric

Re: [Etherboot-developers] MOTD options?

[<ke...@us...>]

>In 5.1.2 we are requesting MOTD options but are never using
>them anyway.  Is this something we still want to request?

No. The menu options aren't needed either.

[Etherboot-developers] MOTD options?

[Eric W B. <ebi...@ln...>]

In 5.1.2 we are requesting MOTD options but are never using
them anyway.  Is this something we still want to request?

Eric

Re: [Etherboot-developers] Etherboot thoughts....

[Eric W B. <ebi...@ln...>]

Markus Gutschke <ma...@gu...> writes:

> Eric W Biederman wrote:
> > Tomorrow is a holiday here so I will only get to it if I start
> > going stir crazy.
> 
> $ calendar
> 24 Jul*	Pioneer Day in Utah
> 24 Jul*	Scopes Monkey Trial, 1925
> 24 Jul*	Simon Bolivar's Day in Ecuador and Venezuela
> 24 Jul*	Tu B'Av (15th Av; minor festival)
> 24 Jul*	The ring comes to Bilbo (LOTR)
>          ^^^^^^^^^^^^^^^^^^^^^^^
> 
> Ah, this must be it ;-) Now we know where Eric lives -- it's Middle Earth. That
> would certainly explain all the magic he performs.

You figured me out.  Now what am I to do?

I know check in the code....

The first pass at relocation is done.

I suspect most of the drivers will require a little bit of fixup,
and I haven't tested it under non-LinuxBIOS conditions.  But everything
__should__ work.

The changes to the BIOS calls are always in effect.  But the code
won't actually relocate itself unlses -DRELOCATION is specified.

Eric

Re: Re[2]: [Etherboot-developers] Q: does xstart work?

[Eric W B. <ebi...@ln...>]

Anselm Martin Hoffmeister <an...@ho...> writes:

> Hello Eric, list,
> 
> EWB> Probably the fun part is keeping track of which areas have been allocated
> EWB> to the partially loaded image.
>  
> >> I'm not so concerned about the text segment size of Etherboot when
> >> Etherboot is converted to run high. In the current version the main
> >> concern is the 48kB limitation. I think there are few machines that
> >> can't spare a couple of hundred kB from the top of memory during the
> >> image loading phase.  One is not supposed to load memory to brim anyway,
> >> Linux will need room to expand when it gains control.
> 
> EWB> Except when you have a ramdisk.  But a sane implemenation will load
> EWB> the ramdisk low, and then push it up higher if required.  Though I seem
> EWB> to like the insane varieties.
> 
> As I see it, Eric, you used the last room for just beyond land's end
> for SLAM downloaded data stuff. BTW, I copied that idea, so the last
> [image-block-num]*[image-block-size] bytes of memory are used for that
> purpose in TFTM too. No problem to spare the last say 300kB, but one
> must not forget!

Agreed.  That has been a FIXME in my code for a while.  Getting 
etherboot to relocate itself was the interesting part of this adventure.
When I get the code checked in there will be a lot of small little
fixes that need to propogate through etherboot to fix this.

Unless the multicast file has always loads into a fixed offset in ram
there is a tradeoff between scaleability and memory usage.  And I really 
don't see a practical way to fix that.  In the normal case of
increasing offsets going to increasingly higher addresses in ram the
the memory overhead actually becomes a constant.  This is because you
can overwrite memory that you have already loaded to it's final
location.

Eric

Re[2]: [Etherboot-developers] Q: does xstart work?

[Anselm M. H. <an...@ho...>]

Hello Eric, list,

EWB> Probably the fun part is keeping track of which areas have been allocated
EWB> to the partially loaded image.
 
>> I'm not so concerned about the text segment size of Etherboot when
>> Etherboot is converted to run high. In the current version the main
>> concern is the 48kB limitation. I think there are few machines that
>> can't spare a couple of hundred kB from the top of memory during the
>> image loading phase.  One is not supposed to load memory to brim anyway,
>> Linux will need room to expand when it gains control.

EWB> Except when you have a ramdisk.  But a sane implemenation will load
EWB> the ramdisk low, and then push it up higher if required.  Though I seem
EWB> to like the insane varieties.

As I see it, Eric, you used the last room for just beyond land's end
for SLAM downloaded data stuff. BTW, I copied that idea, so the last
[image-block-num]*[image-block-size] bytes of memory are used for that
purpose in TFTM too. No problem to spare the last say 300kB, but one
must not forget!

Best regards,
 Anselm                            mailto:an...@ho...

Re: [Etherboot-developers] Etherboot 5.0.7rc2 at rom-o-matic

[Marty C. <ma...@et...>]

On Tue, 16 Jul 2002 12:58:22 -0600 Eric W Biederman <ebi...@ln...>
wrote:
>Backporting my small changes from 5.1.x looks like.  
>Actually just include "stdint.h" instead of <stdint.h> in
>linuxbios_tables.h
>However we work it the fewer external headers we include by accident
>or on purpose the more reproducible the build of etherboot will be.

I patched this in 5.0.7rc2 on rom-o-matic.net.

Marty

-- 
    Try: http://rom-o-matic.net/ to make Etherboot images instantly.

   Name: Marty Connor
US Mail: Entity Cyber, Inc.; P.O. Box 391827; 
         Cambridge, MA 02139; USA
  Voice: (617) 491-6935; Fax: (617) 491-7046 
  Email: ma...@et...
    Web: http://www.etherboot.org/

Re: [Etherboot-developers] Etherboot thoughts....

[Markus G. <ma...@gu...>]

Eric W Biederman wrote:
> Tomorrow is a holiday here so I will only get to it if I start
> going stir crazy.

$ calendar
24 Jul*	Pioneer Day in Utah
24 Jul*	Scopes Monkey Trial, 1925
24 Jul*	Simon Bolivar's Day in Ecuador and Venezuela
24 Jul*	Tu B'Av (15th Av; minor festival)
24 Jul*	The ring comes to Bilbo (LOTR)
         ^^^^^^^^^^^^^^^^^^^^^^^

Ah, this must be it ;-) Now we know where Eric lives -- it's Middle Earth. That 
would certainly explain all the magic he performs.


Markus


-- 
Markus Gutschke
3637 Fillmore Street #106
San Francisco, CA 94123-1600
+1-415-567-8449
ma...@gu...

[Etherboot-developers] Etherboot thoughts....

[Eric W B. <ebi...@ln...>]

The freebsd pxeemu code is broken in 5.0.6.

In pxeemu_entry there is the snippet of code:

		/* Switch Stacks */
		__asm__("xorl %%eax, %%eax\n"
			"movw initsp, %%ax\n"
			"addl %0, %%eax\n"
			"movl %%eax, %%esp\n"
			: : "i" (RELOC));

Which references a local variable in start32.S which
because it was local was actually renamed: initial_regs + 12....

So for those people wanting PXE the freebsd code is a place to
start but it could really use some touch ups to even comppile ;(

Beyond that I've got my work to date in good pretty good shape to
start checking it in (on the development branch, of course).
Tomorrow is a holiday here so I will only get to it if I start
going stir crazy.

The topics look like

virt_to_bus, virt_to_phys, and ioremap for portability and
relocatibility.

Code to build a compressesed LinuxBIOS romimage.

Unplugged cable detection for the eepro100.

Renamed xstart xstart16, and added xstart32.  So we have a common
function between osloader.c and the external programs.  This allows
us to tear down virtual addresses when we are relocated.

Improved debugging code for detecting LinuxBIOS bugs.

The LinuxBIOS stdint compile fix.

Replacement of _real_call instead _prot_to_real && _real_to_prot,
in the assembly code, to allow for relocation.

Reversion of hacks in the pci code.  Deleting membase and not fixing
all of the drivers that use it is unacceptible.  I have fixed the
fa311.c driver to call pcbios_read_config_dword(... PCI_RESOURCE_1 ..)
for itself.

Relocate.c to actually relocate LinuxBIOS.

Fixes to the rtc timer code to:
A) Print the cpu Mhz so buggy implementations of timer2 may be
   detected.
B) Use a smaller value that actually works with the AMD766
   southbridge.
C) Derive TICKS_PER_MS from a more precise definition of
   CLOCK_TICK_RATE

I've got to go or I would check it in now...

Eric

Re: [Etherboot-developers] Q: does xstart work?

[<ke...@us...>]

>> Also the segment descriptor array.
>
>Hmm. I don't quite follow you there.  I only need 6 bytes for the
>global descriptor pointer, and then I can use a global descriptor
>table > 1MB.  That part is working fine.

The image is passed two arguments, a pointer to the tagged image header,
and a pointer to the bootp structure. The image header is needed for the
trampoline segment first-{linux,*dos}.S (in mknbi) to locate the
parameter string, among other things. You could drop support for
first-linux.S, but it would still be needed for first-*dos.S.

>I agree the uncompressed size is not a big deal.  The compressed size
>continues to concern me.  Even with LinuxBIOS I am loading it in
>a rom chip.  A much bigger one than etherboot traditionally lives in
>true, but still a rom.   And the primary reason I am working with
>etherboot is that it is small, and my other alternatives are not.  So
>I don't want to mess that up. 

Don't worry, I'm a memory skinflint too. :-)

Re: [Etherboot-developers] Q: does xstart work?

[Eric W B. <ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >xstart16 still needs a little work to get the bootp data structure below
> >1MB.  But othewise it should work fine.
> >
> >The open issues are:
> >- How to copy/move bootp low for 16bit code.
> 
> Also the segment descriptor array.

Hmm. I don't quite follow you there.  I only need 6 bytes for the
global descriptor pointer, and then I can use a global descriptor
table > 1MB.  That part is working fine.
 
> Maybe this ties in with the memory allocator routines. Ask it for a
> chunk of memory below 1MB and memcpy the bootp and segment descriptor
> structures before calling xstart. You could also ask the memory
> allocator for a chunk for use as stack.

Probably the fun part is keeping track of which areas have been allocated
to the partially loaded image.
 
> I'm not so concerned about the text segment size of Etherboot when
> Etherboot is converted to run high. In the current version the main
> concern is the 48kB limitation. I think there are few machines that
> can't spare a couple of hundred kB from the top of memory during the
> image loading phase.  One is not supposed to load memory to brim anyway,
> Linux will need room to expand when it gains control.

Except when you have a ramdisk.  But a sane implemenation will load
the ramdisk low, and then push it up higher if required.  Though I seem
to like the insane varieties.

I agree the uncompressed size is not a big deal.  The compressed size
continues to concern me.  Even with LinuxBIOS I am loading it in
a rom chip.  A much bigger one than etherboot traditionally lives in
true, but still a rom.   And the primary reason I am working with
etherboot is that it is small, and my other alternatives are not.  So
I don't want to mess that up. 

As far as stack size.  I really only care there except memory < 1MB is
a limited resource in high demand.  What clicked for me a little while 
ago is that I don't have to keep anything permantnely below 1MB.  I
can just push all of my code on the real mode stack.  And I worry a
little if I am using someone else's stack that I will overflow it.
The code is tight enough now that overflowing stacks is no longer an
issue.  

Though it was quite fun reducing the footprint from 300 bytes to 80 :)

Mostly I did it to combat my small observation that whenever I touch
etherboot in a nontrivial the core gets bigger.  

O.k. I am so tired I am rambling.  After I get some sleep I will check
do a little cleanup and see what parts of my code I can check in.

Eric

Re: [Etherboot-developers] Q: does xstart work?

[<ke...@us...>]

>xstart16 still needs a little work to get the bootp data structure below
>1MB.  But othewise it should work fine.
>
>The open issues are:
>- How to copy/move bootp low for 16bit code.

Also the segment descriptor array.

Maybe this ties in with the memory allocator routines. Ask it for a
chunk of memory below 1MB and memcpy the bootp and segment descriptor
structures before calling xstart. You could also ask the memory
allocator for a chunk for use as stack.

I'm not so concerned about the text segment size of Etherboot when
Etherboot is converted to run high. In the current version the main
concern is the 48kB limitation. I think there are few machines that
can't spare a couple of hundred kB from the top of memory during the
image loading phase.  One is not supposed to load memory to brim anyway,
Linux will need room to expand when it gains control.

Re: [Etherboot-developers] Q: does xstart work?

[Eric W B. <ebi...@ln...>]

Eric W Biederman <ebi...@ln...> writes:

> ke...@us... (Ken Yap) writes:
> 
> > >>	ADDR32	LJMPI(_execaddr-_start)
> > 
> > Incidentally if you feel motivated to convert this bit of data
> > segment-modifying code to a push onto the stack followed by a ret,
> > please do.
> 
> I will or something similiar, accessing the 32bit data segment
> from 16bit is a fairly bad thing to do.


O.k. I have all of the routines converted.
xstart16 still needs a little work to get the bootp data structure below
1MB.  But othewise it should work fine.

The open issues are:
- How to copy/move bootp low for 16bit code.
- How to properly allocate the real mode stack.
    I am thinking I can just use the initial stack and push
    the code on there.

For reference the trampoline _real_call pushes on the real mode stack
is just 80 bytes.  80 bytes shouldn't overflow even a small stack.
The worst cases is meme820 which allocates a 280 byte buffer for the
memory map.

Eric


	.globl	xstart16
xstart16:
	pushl	%ebp
	movl	%esp,%ebp
	pushl	%ebx
	pushl	%esi
	pushl	%edi
	movl	8(%ebp),%edx
	movl	12(%ebp),%ebx

	/* FIXME handle the bootp record */
	movl	16(%ebp),%ecx	/* bootp record (32bit pointer) */
	shll	$12,%ecx	/* convert to segment:offset form */
	shrw	$12,%cx

	pushl	$ 10f
	pushl	$ 20f - 10f
	call	_real_call
	.section ".text16"
10:	.code16
	popw	%ax		/* get the return ip addr */
	pushl	%ecx		/* bootp record */
	pushl	%ebx		/* file header */
	pushw	%cs		/* Setup the far return address */
	pushw	%ax		
	pushl	%edx		/* Setup the far address to call */
	lret			/* Back into the routine I'm calling */
20:	.code32
	.previous
	
	popl	%edi
	popl	%esi
	popl	%ebx
	popl	%ebp
	ret

Re: [Etherboot-developers] Q: does xstart work?

[Eric W B. <ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >>	ADDR32	LJMPI(_execaddr-_start)
> 
> Incidentally if you feel motivated to convert this bit of data
> segment-modifying code to a push onto the stack followed by a ret,
> please do.

I will or something similiar, accessing the 32bit data segment
from 16bit is a fairly bad thing to do.

After staring at the problem of relocating the part of etherboot
that must run in 16bit real mode I have finally come up with
a clean design, that will be easy to convert to.

- The 16bit code will normally live above 1MB.
- The 16bit code will be pushed on the 16bit stack just before use.
- _prot_to_real && _real_to_prot will be replaced by _real_call
  which preservs all registers except %esp, and %eax. _real_call
  pushes the 16bit code onto the real mode stack.

An example from my test code.
	pushl	$ 1f
	pushl	$ 2f - 1f
	call	_real_call
1:
	.code16
	movl	$0x88880000, %ebp
	shll	$16, %ebx
	shll	$16, %ecx
	shll	$16, %edx
	shll	$16, %esi
	shll	$16, %edi
	ret
	.code32
2:


Eric

Re: [Etherboot-developers] Q: does xstart work?

[<ke...@us...>]

>>	ADDR32	LJMPI(_execaddr-_start)

Incidentally if you feel motivated to convert this bit of data
segment-modifying code to a push onto the stack followed by a ret,
please do.

Re: [Etherboot-developers] Q: does xstart work?

[<ke...@us...>]

>> But it's in 32-bit PM at that point.
>
>Hmm.  The LJMPI sure looks like it is in 32PM to me....
>xstart:
>	pushl	%ebp
>	movl	%esp,%ebp
>	pushl	%ebx
>	pushl	%esi
>	pushl	%edi
>	movl	8(%ebp),%eax
>	movl	%eax,_execaddr
>	movl	12(%ebp),%ebx
>	movl	16(%ebp),%ecx	/* bootp record (32bit pointer) */
>	shll	$12,%ecx	/* convert to segment:offset form */
>	shrw	$12,%cx
>	call	_prot_to_real
>	.code16
>	pushl	%ecx		/* bootp record */
>	pushl	%ebx		/* file header */
>	movl	$((RELOC<<12)+(1f-RELOC)),%eax
>	pushl	%eax
>	ADDR32	LJMPI(_execaddr-_start)
>1:
>	addw	$8,%sp		/* XXX or is this 10 in case of a 16bit "ret" 
>*/
>	DATA32 call	_real_to_prot
>	.code32

No, you were right, it's in 16-bit RM at that point and so interprets
the 32-bit word as segment:offset. For a moment I was worried that I was
generating non-compliant tagged images in mknbi. I don't know what
ADDR32 does in .code16 but nothing harmful apparently. Maybe it's needed
for the indirection address.

Re: [Etherboot-developers] Q: does xstart work?

[Eric W B. <ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >> > _execaddr is filled in with a linear address.
> >
> >My best guess is that the ADDR32 prefix is just ignored somewhere.
> >As a linear address doesn't make any sense in 16bit real mode.
> 
> But it's in 32-bit PM at that point.

Hmm.  The LJMPI sure looks like it is in 32PM to me....
xstart:
	pushl	%ebp
	movl	%esp,%ebp
	pushl	%ebx
	pushl	%esi
	pushl	%edi
	movl	8(%ebp),%eax
	movl	%eax,_execaddr
	movl	12(%ebp),%ebx
	movl	16(%ebp),%ecx	/* bootp record (32bit pointer) */
	shll	$12,%ecx	/* convert to segment:offset form */
	shrw	$12,%cx
	call	_prot_to_real
	.code16
	pushl	%ecx		/* bootp record */
	pushl	%ebx		/* file header */
	movl	$((RELOC<<12)+(1f-RELOC)),%eax
	pushl	%eax
	ADDR32	LJMPI(_execaddr-_start)
1:
	addw	$8,%sp		/* XXX or is this 10 in case of a 16bit "ret" */
	DATA32 call	_real_to_prot
	.code32
	popl	%edi
	popl	%esi
	popl	%ebx
	popl	%ebp
	ret

_execaddr:
	.long	0



Eric

Re: [Etherboot-developers] Q: does xstart work?

[<ke...@us...>]

>> > _execaddr is filled in with a linear address.
>
>My best guess is that the ADDR32 prefix is just ignored somewhere.
>As a linear address doesn't make any sense in 16bit real mode.

But it's in 32-bit PM at that point.

Re: [Etherboot-developers] Q: does xstart work?

[Eric W B. <ebi...@ln...>]

Eric W Biederman <ebi...@ln...> writes:

> ke...@us... (Ken Yap) writes:
> 
> > >Staring at the assembly in xstart there is the following code
> > >snippet:
> > >	pushl	%eax
> > >	ADDR32	LJMPI(_execaddr-_start16)
> > >1:
> > >...
> > >_execaddr:
> > >	.long	0
> > >
> > >What I don't get is how this snippet of code successfully jumps
> > >to an application.  ADDR32 should ljmpi should take a m16:32.
> > >Where the offset which comes first is 32 bits, and the segment
> > >that comes after is 16bits.  I just don't see how that works when
> > >we pass it a m16:16 in execaddr.
> > 
> > _execaddr is filled in with a linear address.

My best guess is that the ADDR32 prefix is just ignored somewhere.
As a linear address doesn't make any sense in 16bit real mode.

Eric