etherboot-developers — Discussion of developer issues

Re: [Etherboot-developers] [RFC] Relocation techniques...

[<ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >Given that it is safer for the loaded image to bring along it's
> >network driver, and stack, because the loaded image can be recompiled
> >and fixed.  
> 
> That's probably the solution, download a LinuxBIOS to provide PXE
> services. That's not silly at all. You can't buy new memory sticks
> smaller than 256M now. 100 Mb Ethernet is pretty standard and
> downloading a 500kB Linux kernel takes just a fraction of a second.
> Won't work on old machines? Well you can't run Windoze on old machines
> anyway, which is what those people want PXE for.
> 
> And the biggest laugh will be when people use PXE to download Etherboot
> using Peter and Vasil's clever hack, to download a LinuxBIOS, to
> download Windoze, because it's easier to get LinuxBIOS to work than
> broken PXE firmware.  I love it.

Call it freebios and the target is just about on track.

The core of LinuxBIOS which holds the name is really about taking
unitialized hardware, initializing the pieces it must (like memory),
and allocating non-conflicting resources to the hardware.  Then
LinuxBIOS reads in a bootloader (Etherboot) or a kernel (Linux) from
the rest of the rom chip, using a small ELF loader, and jumps to it. 
A table of information like memory sizes is provided at a fixed
location in RAM for the loaded OS to find.  Having a seperate
bootloader means we can reuse motherboard indpendent code trivially.

So I currently have a working BIOS with no callbacks!  

The freebios project originally set out to have the same interface as 
a traditional BIOS.  Some designs have been thrown around but not a
lot of progress has been made along that front.

But with the LinuxBIOS core maturing loading an x86 BIOS compatibility
interface is definitely worth thinking about.  It is a little more
oomph to get going but it should get the job done in the long term.

But I do like that idea PXE loading Etherboot to load a working PXE
stack!  

The loaded PXE stack might even be able to use the etherboot drivers,
so the etherboot project isn't off the look, just the rom image :)

Sort of like the LinuxBIOS project isn't off the hook of needing a
bootloader, despite the fact it is deliberately not included in the
core. :)  

Eric

[Etherboot-developers] Re: [RFC] Relocation techniques...

[<pjc...@el...>]

I'm not sure if this precisely fits in with the relocation being discussed
here, but it foiled my efforts to etherboot from my existing network 
cards.

I've got one of the 3com 3C905B-NM cards, for which 
disklessworkstations.com sells proms.  These are the cards that overload 
the MII signal to choose which ROM bank to read.  The current (hack) 
solution is to use a boot disk which locks the MII setting on.  That way 
the card can see more than 4KB of the boot rom, and load the etherboot
code into memory.

However, this is rather sloppy.  It leaves the one signal stuck on.  
Always.  Even when you don't boot from the bootrom.  Linux handles this 
situation just fine.  However, DOS TCP stacks pitch a fit.  This makes our 
Ghost boot disks fail, rendering the machines rather useless.

Apparently the right solution is to have a small piece of code which lives 
in the low 4KB of the ROM.  It switches the banks, copies the ROM into RAM 
somewhere, and switches back.  Then it boots.

All that to say, if you're toying with relocation and other such memory 
constraints, please consider building the little low-memory shim into your 
plans.

	-P

Re: [Etherboot-developers] [RFC] Relocation techniques...

[<ke...@us...>]

>Given that it is safer for the loaded image to bring along it's
>network driver, and stack, because the loaded image can be recompiled
>and fixed.  

That's probably the solution, download a LinuxBIOS to provide PXE
services. That's not silly at all. You can't buy new memory sticks
smaller than 256M now. 100 Mb Ethernet is pretty standard and
downloading a 500kB Linux kernel takes just a fraction of a second.
Won't work on old machines? Well you can't run Windoze on old machines
anyway, which is what those people want PXE for.

And the biggest laugh will be when people use PXE to download Etherboot
using Peter and Vasil's clever hack, to download a LinuxBIOS, to
download Windoze, because it's easier to get LinuxBIOS to work than
broken PXE firmware.  I love it.

Re: [Etherboot-developers] [RFC] Relocation techniques...

[<ebi...@ln...>]

Markus Gutschke <ma...@gu...> writes:

> Ken Yap wrote:
> > I'm not a big fan of callbacks either, it means designing another
> > interface which is slightly architecture dependent.
> 
> I guess, it depends on what kind of callbacks you decide to expose. I was mostly
> 
> thinking of making the networking driver available to the second stage
> loader. It should be possible to design a architecture-independent API to do so.
> 
> 
> If we have this ability, then it becomes trivial to do such things as:
> 
>   - write an externally-configurable graphical second stage loader. I think,
> this has been discussed a few weeks ago, and it seemed that the current API
> makes it difficult to fully implement this code. Mostly, because there is
> currently no good way to load configuration files and graphics files from the
> second stage code (or have I overlooked some feature in the API that provides
> this ability?)

Trivial.  You have a piece of preparation code that smashes all together into
one big image to load.  

Alternatively the type of NIC is supplied in the DHCP query.

>   - expose etherboot NIC drivers to the loaded Linux kernel. Admittedly, our
> drivers are not optimized for production use in a general-purpose OS, but this
> should be a great option for a rescue kernel that is guaranteed to run on any
> machine which can load etherboot.
> 
>   - maybe, some enterprising soul will even get around and implement full PXE on
> 
> top of etherboot. I don't particularly care for PXE, but requests for it keep
> being made.

For the people wanting PXE it is certainly a benefit.  

From a design standpoint I assume all code has bugs,
but that it actually works on the heavily exercised paths.

Further firmware is hard to update to fix bugs in your installed
base.  Depending on how hard it is to upgrade the ROMS on your
NIC this can range from very hard to almost trivial.  But even in the
best case it is still dangerous to upgrade your boot device, and can
leave a machine dead if the upgrade does not work properly.

So the design requirements are:
- Cope with small bugs in the firmware.
- You can't upgrade deployed firmware, on production machines.
- The firmware must be small.

Given that it is safer for the loaded image to bring along it's
network driver, and stack, because the loaded image can be recompiled
and fixed.  

Additionally when information is provided it is better to provide that
information in a table, instead of using callbacks.  Because if the
callback goes wrong your system is dead.  If the table is wrong the
loaded software can detect the bug and work around it.

So once you are into pure software land I don't have problems with
callbacks, but until that point I feel they should be avoided.

Eric

Re: [Etherboot-developers] [RFC] Relocation techniques...

[Markus G. <ma...@gu...>]

Ken Yap wrote:
> I'm not a big fan of callbacks either, it means designing another
> interface which is slightly architecture dependent.

I guess, it depends on what kind of callbacks you decide to expose. I was mostly 
thinking of making the networking driver available to the second stage loader. 
It should be possible to design a architecture-independent API to do so.

If we have this ability, then it becomes trivial to do such things as:

  - write an externally-configurable graphical second stage loader. I think, 
this has been discussed a few weeks ago, and it seemed that the current API 
makes it difficult to fully implement this code. Mostly, because there is 
currently no good way to load configuration files and graphics files from the 
second stage code (or have I overlooked some feature in the API that provides 
this ability?)

  - expose etherboot NIC drivers to the loaded Linux kernel. Admittedly, our 
drivers are not optimized for production use in a general-purpose OS, but this 
should be a great option for a rescue kernel that is guaranteed to run on any 
machine which can load etherboot.

  - maybe, some enterprising soul will even get around and implement full PXE on 
top of etherboot. I don't particularly care for PXE, but requests for it keep 
being made.


Markus

-- 
Markus Gutschke
3637 Fillmore Street #106
San Francisco, CA 94123-1600
+1-415-567-8449
ma...@gu...

Re: [Etherboot-developers] [RFC] Relocation techniques...

[<ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >For maximum flexibility the Virtual Address technique wins hands down
> >because it encompases most of the obvious effort needed to port
> >drivers to work on different cpu architectures.
> >
> >I'm still thinking about it.
> 
> With VM you could also do funky things like dynamically mapping in one
> out of a large set of drivers to reduce runtime footprint even when you
> have a lot of storage for the combined binary.
> 
> Question: what virtual address would you have Etherboot live at? Would
> it not preclude loading target code at that address range? I guess maybe
> not, you could have a different VM mapping for the target code and then
> switch it back on exiting Etherboot. Or maybe have Etherboot run in the
> highest available pages in VM and hopefully nobody ever wants to load
> target code there.

There are two possibilities, I have considered.
1) live at the end of memory.
2) If the image will overwrite etherboot, at some point stop the active
   driver relocate etherboot, and startup the everything again.

I would aim for the end of memory first, as it should be the least
contended location to work out of.

As for the virtual address it really doesn't matter, so long as the physical
one isn't a problem.  But except for a debugging build I doubt I would
use anything more than segment registers.
 
> >For me the low fruit is definitely solving the problem of compiling
> >in multiple drivers, and protocols and while still producing a working
> >binary.  For that the x86 bios calls definitely need to be clearly
> >seperated out.  (The ideal is to build a .com or .floppy image with
> >most of the drivers)
> 
> Yes, it's annoying that the BIOS calls are inline, limiting the total
> memory space of the 32-bit code to 64kB.

Definitely.  This is the first one to get.  Ideally we can leave
the BIOS code in the 16bit rom segment, and put the rest of etherboot elsewhere.

Eric

Re: [Etherboot-developers] [RFC] Relocation techniques...

[<ke...@us...>]

>I agree but I am unlikely to be one of those working on callbacks.  I do
>not believe callbacks are generally viable.  Or at the very least callbacks
>can easily lead to code being executed in ways it was not tested in,
>and breaking at inoportune moments.
>
>Plus there is some amount of bloat in just forwarding the callbacks to
>the appropriate location.

I'm not a big fan of callbacks either, it means designing another
interface which is slightly architecture dependent.

It's the age old problem of how to communicate load locations to the
loader. Etherboot does it by prepending a directory in front, but this
annoys people who want to boot kernel images fresh from the build.
Personally I don't see an extra tagging step is a big hassle. Maybe one
day all kernel images will be netloader friendly out of the build.
Another way might be a smart TFTP daemon that can assemble virtual
tagged images on the fly. Yet another way might be a special tagged flag
that says: this is just the header, keep the header around then append
or subtract something from the filename to get the body of the image to
fetch. I.e. the TFTP directory would contain:

	vmlinuz.nb.dir	512 bytes, just the header
	vmlinux.nb	the real image

But then you'd need a utility to generate the headers, you can't just
dump the image in the directory hot from the build, so why not just tag
the thing in the first place?

Re: [Etherboot-developers] [RFC] Relocation techniques...

[<ke...@us...>]

>For maximum flexibility the Virtual Address technique wins hands down
>because it encompases most of the obvious effort needed to port
>drivers to work on different cpu architectures.
>
>I'm still thinking about it.

With VM you could also do funky things like dynamically mapping in one
out of a large set of drivers to reduce runtime footprint even when you
have a lot of storage for the combined binary.

Question: what virtual address would you have Etherboot live at? Would
it not preclude loading target code at that address range? I guess maybe
not, you could have a different VM mapping for the target code and then
switch it back on exiting Etherboot. Or maybe have Etherboot run in the
highest available pages in VM and hopefully nobody ever wants to load
target code there.

>For me the low fruit is definitely solving the problem of compiling
>in multiple drivers, and protocols and while still producing a working
>binary.  For that the x86 bios calls definitely need to be clearly
>seperated out.  (The ideal is to build a .com or .floppy image with
>most of the drivers)

Yes, it's annoying that the BIOS calls are inline, limiting the total
memory space of the 32-bit code to 64kB.

Re: [Etherboot-developers] [RFC] Relocation techniques...

[<ebi...@ln...>]

Markus Gutschke <ma...@gu...> writes:

> Eric W. Biederman wrote:
> > Technique 1. (Shared library)
> > Technique 2. (Virtual Adresses)
> > Technique 3. (Fake Relocation)
> > So I'm probably going to aim at implementing Virtual Addresses in the
> > long term.  If all of the drivers need to be audited for multicast
> > support anyway it should be much worse to get a handle on their
> > address space operations.
> 
> I agree with you, that virtual addresses are the most desirable solution,
> although not neccessarily the easiest one. Whenever this discussion came up in
> the past, I always saw two crucial arguments that ultimately pointed towards
> implementing virtual addresses:
> 
>   - code size is crucial. While these days it is relatively common to have at
> least a couple of tens of kilobytes of available ROM space (my original design
> goal for etherboot was to fit into 8kB), there are still enough constraints to
> make it important to carefully manage code size -- especially if you plan on
> making these changes in order to be able to compile multiple drivers into the
> same image.
> 
> - the most interesting reason that I could see for this kind of change was the
> prospect that it would allow proper call backs into the driver. Once that is
> possible, there are all kinds of cool things that we can implement.

I agree but I am unlikely to be one of those working on callbacks.  I do
not believe callbacks are generally viable.  Or at the very least callbacks
can easily lead to code being executed in ways it was not tested in,
and breaking at inoportune moments.

Plus there is some amount of bloat in just forwarding the callbacks to
the appropriate location.

Ken's approach of just changing the DHCP configuration options is
interesting, as it really keeps exactly virtually the same interface
to the outside world, while providing most of the benefits of
callbacks.

Now that etherboot is reporting it's primary NIC a loaded image can be
reasonably expected to actually support the hardware. 

There are some interesting things I would like to see and after I
solve the easy problems I will see how the very interesting features
are implemented.

Eric

Re: [Etherboot-developers] [RFC] Relocation techniques...

[Markus G. <ma...@gu...>]

Eric W. Biederman wrote:
> Technique 1. (Shared library)
> Technique 2. (Virtual Adresses)
> Technique 3. (Fake Relocation)
> 
> So I'm probably going to aim at implementing Virtual Addresses in the
> long term.  If all of the drivers need to be audited for multicast
> support anyway it should be much worse to get a handle on their
> address space operations.

I agree with you, that virtual addresses are the most desirable solution, 
although not neccessarily the easiest one. Whenever this discussion came up in 
the past, I always saw two crucial arguments that ultimately pointed towards 
implementing virtual addresses:

  - code size is crucial. While these days it is relatively common to have at 
least a couple of tens of kilobytes of available ROM space (my original design 
goal for etherboot was to fit into 8kB), there are still enough constraints to 
make it important to carefully manage code size -- especially if you plan on 
making these changes in order to be able to compile multiple drivers into the 
same image.

- the most interesting reason that I could see for this kind of change was the 
prospect that it would allow proper call backs into the driver. Once that is 
possible, there are all kinds of cool things that we can implement.



Markus


-- 
Markus Gutschke
3637 Fillmore Street #106
San Francisco, CA 94123-1600
+1-415-567-8449
ma...@gu...

[Etherboot-developers] [RFC] Relocation techniques...

[<ebi...@ln...>]

I'd really like to relocate etherboot, and there are currently
two techniques I can use.  I'd like to get a little more feedback
before I go forward with anything.

Technique 1. (Shared library)
 -  Build etherboot as a shared library, and keep the relocation
    information through runtime, so the code can be relocation.
 Impacts.
  - Larger codesize.
  - Must redesign the x86 bios calls to be a sperate piece that partly
    lives below 1MB.

Technique 2. (Virtual Adresses)
 - Use virtual addresses.  Either a true page table or just setup
   some segments.
  Impacts.
  - osloader needs to change how it copies data to memory.
  - pci devices need to call ioremap to find an appropriate
    address to access their devices at.
  - pci devices need to call virt_to_bus to compute addresses to
    pass to dma functions.
  - Must redisgn the x86 bios calls to be a seperate piece that partly
    lives below 1MB.


Technique 3. ( Fake Relocation)
 - Relocate a stub allowing for loading over etherboot.
 Impacts.
  - osloader needs to change how it copies data to memory.
  - No callbacks of any kind, are allowed.

For code size it is a toss up between using Virtual Addresses,
and Fake Relocation.  There are enough relocations and symbol name strings
left the Shared Library technique is clearly larger.

For amount of work for the Shared Library and Fake Relocation is pretty
clearly less than using Virtual Addresses.

For maximum flexibility the Virtual Address technique wins hands down
because it encompases most of the obvious effort needed to port
drivers to work on different cpu architectures.

I'm still thinking about it.

For me the low fruit is definitely solving the problem of compiling
in multiple drivers, and protocols and while still producing a working
binary.  For that the x86 bios calls definitely need to be clearly
seperated out.  (The ideal is to build a .com or .floppy image with
most of the drivers)

So I'm probably going to aim at implementing Virtual Addresses in the
long term.  If all of the drivers need to be audited for multicast
support anyway it should be much worse to get a handle on their
address space operations.  

Plus people have been wondering where the ioremap calls are anyway.

Eric

Re: [Etherboot-developers] URI-schemes (was: Patch for TFTP-multicast released)

[<ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >O.k. I have been talking about this with the users on my end and for
> >me at least there is one very useful thing to be gained with the:
> >://<server>:<port>/filename
> >syntax.
> >
> >And that is the ability to specify a port.  Unless there is an option
> >to specify a different one.  This may be a wasteful bit of the slam
> >protocol as I have it currently defined.  But it currently uses one
> >port per file.  So I can't do the :/// trick.
> 
> Ok, this I agree is a useful facility not currently possible and should
> be allowed. I'm not against adopting parts of standard syntax that are
> useful. So I suggest that the parser ignore the host portion and
> permit a null host portion in the URI. So it would look like:
> 
> 	protocol://[host][:port]/tail

Sounds good.  Optional hosts are not normally allowed but that is because
there is not a good default and we have one.

The full standard syntax is:
protocol://<user>:<password>@<host>:<port>/tail

I don't see us needing the user and password code any time soon.  

When I get back to it if I implment the port I don't see a reason
not to implement the host.  However I will store the host in the arp
table (like next-server) so it doesn't matter to the rest of
etherboot.  Unless the code is larger than I expect.

Eric

Re: [Etherboot-developers] URI-schemes (was: Patch for TFTP-multicast released)

[<ke...@us...>]

>O.k. I have been talking about this with the users on my end and for
>me at least there is one very useful thing to be gained with the:
>://<server>:<port>/filename
>syntax.
>
>And that is the ability to specify a port.  Unless there is an option
>to specify a different one.  This may be a wasteful bit of the slam
>protocol as I have it currently defined.  But it currently uses one
>port per file.  So I can't do the :/// trick.

Ok, this I agree is a useful facility not currently possible and should
be allowed. I'm not against adopting parts of standard syntax that are
useful. So I suggest that the parser ignore the host portion and
permit a null host portion in the URI. So it would look like:

	protocol://[host][:port]/tail

Re: [Etherboot-developers] URI-schemes (was: Patch for TFTP-multicast released)

[<ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >1: Boot LTSP from server.hoffmeister.augustinnetz.de  => /vmlinuz.ltsp (TFTM)
> >2: Boot Doze from server.hoffmeister.augustinnetz.de   => /doze.img (TFTM)
> >3: Boot testfile from anselm.hoffmeister.augustinnetz.de => /test/image (NFS)
> >
> >How could I do this with option "next-server" ? Not at all. And having the 
> >server name in the URI is the trivial config, and most intuitive, too.
> 
> Next-server is scoped just like other options in dhcpd.conf so you can
> have as many next-servers as there are images, or share them in groups.
> Next-server is actually more flexible in usage in DHCPD even if less
> easy to read than embedding the server IP in the filename.
> 
> Don't fool yourself into thinking that generality of expression and
> adhering to a standard will automatically give you usefulness.
> Etherboot is not a web browser, the protocol is not TCP, you cannot
> fetch from just any server on the Internet without a lot of extra
> mechanism.

O.k. I have been talking about this with the users on my end and for
me at least there is one very useful thing to be gained with the:
://<server>:<port>/filename
syntax.

And that is the ability to specify a port.  Unless there is an option
to specify a different one.  This may be a wasteful bit of the slam
protocol as I have it currently defined.  But it currently uses one
port per file.  So I can't do the :/// trick.

Not that I think specifying a different server that is on the other
side of a gateway would be a good idea.  But I do think being able to
specify a server port in a standard way is good idea.

But this code can be put into a generic part of the parser and it
doesn't need to be replicated across protocol implementations.

As for specifying a different server, there are plenty of useful cases
where you don't need a gateway.  This can be useful for distributing
the load between machines, etc.

Eric

Re: [Etherboot-developers] (no subject)

[<ebi...@ln...>]

ke...@us... (Ken Yap) writes:

> >   (1).I wonder that if the Etherboot 5.1.2 rc support multicast?
> >why Mark all network drivers as not yet supporting multicast packets ?
> 
> Because the code to implement multicast in the drivers has to be
> written. It has only been implemented in the EEPRO100 driver. The
> details differ according to hardware.

And as a clarification it should take about a day or two to go through
all of the drivers (compare them to the Linux driver) and write the
needed 5 lines of code.  The eepro100 actually is one of the harder
NICs to support.  It is on my todo, I just haven't gotten that far yet.

Eric

Re: [Etherboot-developers] been testing 5.0.7rc2 and 5.1.2rc1

[<ebi...@ln...>]

ke...@us... writes:

> I've just tested 5.0.7rc2 and just as well I did because I had
> introduced a embarrassing bug, but I've fixed it in the CVS now. I
> should be able to hand it to Marty for wider testing on rom-o-matic
> soonish.
> 
> I also gave 5.1.2rc1 a whirl, no special options, and found that it
> wasn't able to complete loading FreeDOS. It ended with Unable to load
> file. Linux was ok. But I don't think it's anything to do with the OSes.
> I think it's a termination condition in the TFTP loader code that's not
> right and the FreeDOS image just happened to be the right length to
> tickle the bug.  Haven't looked into it closely.

This may be a tagging image versus elf image problem.  Given that the
tftp code was basically untouched and a heavily touched the binary
loaders.

Eric

Re: [Etherboot-developers] (no subject)

[<ke...@us...>]

>   (1).I wonder that if the Etherboot 5.1.2 rc support multicast?
>why Mark all network drivers as not yet supporting multicast packets ?

Because the code to implement multicast in the drivers has to be
written. It has only been implemented in the EEPRO100 driver. The
details differ according to hardware.

>   (2).I use VIA VT86C100A Rhine-II PCI Fast Ethernet with etherboot to 
>remote boot windows 98. I can boot it, but after I close the windows 98,I 
>boot it secondly, the bootrom report the NIC's MAC address is 
>00:00:00:00:00:00, could you tell me why?

Maybe it's telling you that you should use a better operating system?  :-)

[Etherboot-developers] (no subject)

[zhiguo d. <dz...@ho...>]

   (1).I wonder that if the Etherboot 5.1.2 rc support multicast?
why Mark all network drivers as not yet supporting multicast packets ?
   (2).I use VIA VT86C100A Rhine-II PCI Fast Ethernet with etherboot to 
remote boot windows 98. I can boot it, but after I close the windows 98,I 
boot it secondly, the bootrom report the NIC's MAC address is 
00:00:00:00:00:00, could you tell me why?


_________________________________________________________________
Join the world’s largest e-mail service with MSN Hotmail. 
http://www.hotmail.com

RE: [Etherboot-developers] been testing 5.0.7rc2 and 5.1.2rc1

[Timothy L. <tl...@ro...>]

> I've just tested 5.0.7rc2 and just as well I did because I had
> introduced a embarrassing bug, but I've fixed it in the CVS now. I
> should be able to hand it to Marty for wider testing on rom-o-matic
> soonish.

I wondered why the cvs from earlier this morning did not work.  Thanks for
the update.

Tim

[Etherboot-developers] been testing 5.0.7rc2 and 5.1.2rc1

[<ke...@us...>]

I've just tested 5.0.7rc2 and just as well I did because I had
introduced a embarrassing bug, but I've fixed it in the CVS now. I
should be able to hand it to Marty for wider testing on rom-o-matic
soonish.

I also gave 5.1.2rc1 a whirl, no special options, and found that it
wasn't able to complete loading FreeDOS. It ended with Unable to load
file. Linux was ok. But I don't think it's anything to do with the OSes.
I think it's a termination condition in the TFTP loader code that's not
right and the FreeDOS image just happened to be the right length to
tickle the bug.  Haven't looked into it closely.

Re: [Etherboot-developers] Patch for TFTP-multicast released

[<ke...@us...>]

>The question is what is the natural way to extend DHCP to direct
>a client to boot from other protocols.  URLs seem a natural extension,
>as does extending the next-server option.

I like the unification of the URI scheme, but the problem with putting
the IP address in the URL is that now you have two places where the
server IP could be taken from, the next-server option (siaddr) and the
IP in the filename. I see this as a complication which could cause
unnecessary confusion and verbosity, for the sake of mimicking the
HTTP/FTP/etc practice. Only once in a while does someone ask in the LTSP
list about having the TFTP server separate from the DHCP server and
putting one next-server option in global scope satisfies 99% of them.

A minor point is that when you do use the next-server option, it has the
advantage of separate scoping.  That is to say, you can have
next-servers scoped independently of filename.

>So I suggest we just test for protocol:/// in all cases.  We should require
>at least the double slash.  And the tripple slash probably works, in
>most cases. 

My feeling is for now just make the ip:port empty, i.e. the triple
slash. Or ignore the host portion. Let the future decide if moving the
host into the filename is a good idea. You could try to push things
along by hacking hosted tftp clients to accept URIs with IPs, like ncftp
did with FTP. That may give you some existing practice to judge.

>Given that more people are used to urls than parsing the weird dhcp
>history it might be worth the little bit of extra code to parse an ip
>address if that is present.

My feeling is not for now, at least not for the DHCP triggered
downloads. A bootloader is not a web browser and we don't need to mimic
every aspect of the URI syntax.

My ideal outcomes for this URI enhancement are:

1. Zero size (ok, ok, as small as possible)
2. Backward compatible with existing practice (TFTP with no surprises)
3. Provides some enhanced functionality which encourages people to use
it, e.g. if both DOWNLOAD_PROTO_NFS and DOWNLOAD_PROTO_TFTP are compiled
in, then you can switch just by editing the filename.
4. Minimum number of config options. If 1 and 2 are satisfied, we could
even make it a standard inclusion.
5. Makes download code modular (but of course).
6. Other stuff I haven't thought of yet.

Re: [Etherboot-developers] URI-schemes (was: Patch for TFTP-multicast released)

[<ke...@us...>]

>1: Boot LTSP from server.hoffmeister.augustinnetz.de  => /vmlinuz.ltsp (TFTM)
>2: Boot Doze from server.hoffmeister.augustinnetz.de   => /doze.img (TFTM)
>3: Boot testfile from anselm.hoffmeister.augustinnetz.de => /test/image (NFS)
>
>How could I do this with option "next-server" ? Not at all. And having the 
>server name in the URI is the trivial config, and most intuitive, too.

Next-server is scoped just like other options in dhcpd.conf so you can
have as many next-servers as there are images, or share them in groups.
Next-server is actually more flexible in usage in DHCPD even if less
easy to read than embedding the server IP in the filename.

Don't fool yourself into thinking that generality of expression and
adhering to a standard will automatically give you usefulness.
Etherboot is not a web browser, the protocol is not TCP, you cannot
fetch from just any server on the Internet without a lot of extra
mechanism.

[Etherboot-developers] URI-schemes (was: Patch for TFTP-multicast released)

[Anselm M. H. <an...@ho...>]

Hi list,

for me it seems as if the usage of either "next-server" or the server name 
inside URIs is more or less dshihad. Why not give them both a try? It won't 
do the great deal in codesize.

Even if you harshly dislike any server names in URI, you should keep to the 
standard

proto: / / <servername> : <port> / <path> / <filename>

so don't please use just two slashes followed by the filename. That could be 
annoying to users, esp if the first directory in path is called "boot.net"....
You wouldn't know what tftp://boot.net/vmlinuz.ltsp really means!

Eric had the idea to generally implement a URI parser which wouldn't break 
anything - nice idea. That could even - if supplied - override the 
"next-server" option from dhcp. Imagine the following situation:

I have a dumb terminal in the kitchen (P133-based) without disk. As my 
desktop is not powered 24/7 (but my server is, mostly), on the server there 
is dhcpd and tftpd and tftpd-mcast and so on. But when I test my things, I 
want to have the client get the image from my desktop pc, with menu enabled:

1: Boot LTSP from server.hoffmeister.augustinnetz.de  => /vmlinuz.ltsp (TFTM)
2: Boot Doze from server.hoffmeister.augustinnetz.de   => /doze.img (TFTM)
3: Boot testfile from anselm.hoffmeister.augustinnetz.de => /test/image (NFS)

How could I do this with option "next-server" ? Not at all. And having the 
server name in the URI is the trivial config, and most intuitive, too.

OK, I know that menuing is taken out. (saves code size, so good idea :-) and 
external menu programs could handle that better. But why should they have to 
manipulate two dhcp fields, filename as well as next-server?

Does anyone know a case when there are two succedent slashes inside a 
filename? I don't, so one could easily specify strings containing "//" as 
"reserved" aka "used for URI filename scheme". This would not break the 
default behaviour (no surprises, Ken? said) and offer us URIs. So any string 
of the form

<proto> <colon> <slash> <slash> <rest>

<proto> := 1-6 * [A-Za-z0-9 "-" "_"]

has to be treated as URI. Rest is of the form

<rest> := [ <servername> [ <colon> <portnumber> ]] <path>

with <servername>, <portnumber> optional; <portnumber> support not required.
<servername> must be in IP-dotted-quad (no DNS available) and would simply 
override the next-server; <path> MUST start with a <slash>.

The only exception may happen with <proto> = "FILE", where servername doesn't 
make sense. For that special purpose, we MUST supply the
FILE : / / / disk / 0    format, but as few filenames begin with FILE 
<colon>, we could also allow (anti-standard! but KDE does that too)
FILE : / disk / 0
what my code yet does.

Is this specification enough?

BTW1 {
<portnumber> is something not really needed, except possibly by TFTM, as 
there is a portnumber for that, but if you want to use your TFTP-server with 
multicast capability, why should that proggie also listen on another port, it 
runs on port 69 or so anyway!
}

Of course, my code is really buggy still, but I said that. Released early.
The URI_SUPPORT stuff was a quick hack. Monge that yourselves, I won't resist 
to download some more usable main.c then my own. Eric's idea to be 
implemented... or ask me and I just do it - no priority for me for now. 
Mainly tftm can be tested here :-)

I believe you cannot compile in nfs, tftp, slam and tftm at the same time. 
That would blow up too much... ok, time to search the old 32k EEPROMs.
But it COULD make sense to want to have NFS and TFTP at once. You cannot 
specify DOWNLOAD_PROTO_{NFS|TFTP} at the same time, can you? Which one would 
be default? Should we start some new option naming scheme like this:

DOWNLOAD_PROTO_DEFAULT = {NFS|TFTP|SLAM|TFTM}  sets the fallback case if no 
<proto> is given, cause no URI but a simple filename
DOWNLOAD_PROTO_{NFS|TFTP|SLAM|TFTM} enables support for that specific proto
SUPPORT_URI enables the URI parsing mechanism. 

So something like (I'm not firm in preprocessing directives...)

#if DOWNLOAD_PROTO_DEFAULT = "NFS"
	#define DOWNLOAD_PROTO_NFS
#elif DOWNLOAD_PROTO_DEFAULT = "TFTP"
	[...]
#else
	#define DOWNLOAD_PROTO_DEFAULT="TFTP"
	#define DOWNLOAD_PROTO_TFTP
#endif
#define PROTO_COUNT = 0
#ifdef DOWNLOAD_PROTO_NFS
	#redefine PROTO_COUNT = PROTO_COUNT + 1
#endif
	[similar 3 lines for TFTP... each]

#if (PROTO_COUNT > 1) || defined (CAN_BOOT_DISK)
	// URI support needed anyway
	#define SUPPORT_URI
#endif

could care for consistency.

Again, comments welcome. I have to stress-test my mailbox anyway.

BTW2 {
TFTM tested with LANCE code only. That's what VMware uses, I think, so many 
people could use it. RTL8139 (compushack fastline), NS8390 (ne2k-isa) and 
Via-Rhine (d-link, will be thrown away afterwards, replaced by RTL) to come 
this weekend.
}

Anselm

Re: [Etherboot-developers] Patch for TFTP-multicast released

[<ke...@us...>]

>>Given that more people are used to urls than parsing the weird dhcp
>>history it might be worth the little bit of extra code to parse an ip
>>address if that is present.
>
>I think not for now, not for DHCP triggered downloads at least. A
>bootloader is not a web browser.  The three /// allows the scheme to be
>expanded in that direction if it is deemed worthwhile later. Other
>solutions are possible, such as a proxy; I've been tempted to write a
>TFTP to HTTPS proxy.

Just to expand a bit on that point, if the syntax is made general to
include an IP then user expectations will increase correspondingly, then
we are expected to automatically cater for hosts on other subnets
(without a relay), routing, all that. Let's not go there for now.

Re: [Etherboot-developers] Patch for TFTP-multicast released

[<ke...@us...>]

>> Hmm, also a redundant specification for TFTM. The server should be taken
>> from siaddr, not the URI.
>
>The URL spec allows for it, and I started this trend. 

SF seems to have lost my posting on this. I'm not going to type it all
in again. Essentially I like the unification of the URIs but we don't
need to mimic all of the baggage. If you put an IP in the URI there are
two places to get the server IP from and it's a complication.
Practically all who post on LTSP asking about separating DHCP and TFTP
are catered for with next-server. Also next-server is scoped in ISC
DHCPD, but filenames cannot be separately scoped for server and file
portion.

So my feeling is, use triple slashes to skip the host portion, i.e. null
string for host.

My desired outcomes are:

1. Minimum code footprint.
2. Backward compatible with current TFTP.
3. Has payoff, e.g. can switch between TFTP and NFS by editing the
filename if both are compiled in.
4. Makes code more modular (of course!)
5. Minimum extra config variables.
6. Other stuff I haven't thought of.

If 1+2 are satisfied, it can be a standard feature, no need for a config
option.

>Given that more people are used to urls than parsing the weird dhcp
>history it might be worth the little bit of extra code to parse an ip
>address if that is present.

I think not for now, not for DHCP triggered downloads at least. A
bootloader is not a web browser.  The three /// allows the scheme to be
expanded in that direction if it is deemed worthwhile later. Other
solutions are possible, such as a proxy; I've been tempted to write a
TFTP to HTTPS proxy.