Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo
I'd like to explore options for adding standard mechanisms for TCP/IP into NitrOS9. There are a couple ways to get networking in OS9 now, and more projects in development. It would be very nice to have a standard interface so that software could be written that works with any of these, and so that a common set of userland tools could be used with all devices.
Existing tools such as Drivewire use the IP stack on the server to provide TCP/IP services to NitrOS9. There are very inexpensive "Tcp/ip on a chip" controllers that can be included in a hardware networking pak for the Coco. Given that NitrOS9 doesn't have any native networking facilities, and an IP stack can be so easily created "outside" the Coco, I think an interface between OS9 and an external IP stack makes more sense than creating one in NitrOS9 itself. If this doesn't feel right to anyone, please let me know. I am very interested in all opinions and thoughts on this subject.
Assuming the IP stack is external to NitrOS9, the goal would be to provide a standard way to do a number of networking tasks, and then a common set of userland tools that any networking device could use.
Services that could be provided include:
Configure/query interface parameters (ip address, gateway, etc)
Send and listen for ICMP packets
Send and listen for UDP packets
Establish outgoing TCP connections
Listen for incoming TCP connections
I'm not sure, but we may wish to provide some higher layer services as well. Both DriveWire and the TCP/IP controllers do have the capability to provide application layer services. One that stands out is DNS resolution. Although this could be implemented using lower level UDP/TCP calls, it is so common that I think implementing it might be justified.
I personally think providing services for things like SMTP or HTTP is "cheating", and I've already implemented simple lients for these using just TCP in OS9 without too much work.
NitrOS9 does not currently provide anything like the sockets interface found in *nix systems. DriveWire gets around this by using SCF devices. An application opens a psuedo device that returns a free SCF channel, and then issues commands that do things such as open an outbound TCP connection or instruct the external IP stack to listen on a specific port for incoming connections. This hybrid model is similar to using a serial port to communicate with a modem in command mode. Once a connection is established, the channel enters a raw pass through mode, again similar to a modem that has been dialed and connected.
I think an SCF based model like this could work, and it has the advantage that all OS9 software "just works" with such a model, but I'd love to hear any alternative ideas. One possibility is a new file manager that works similar to SCF but understands more of the networking concepts. This would be a very neat project but quite a bit more work than shoehorning onto SCF.
The SCF model works well for connection oriented things such as TCP, but I'm not sure how to implement things like ICMP or UDP that do not normally have a connection.
There are a lot of issues here. I don't expect anything to happen over night, but I'll spend as much time as necessary to make this happen eventually. I'd like to do something that everyone (or at least a majority) thinks is done the Right Way. DriveWire is sort of a hack to be sure, I would like to take some time and provide something well designed, open and usable by any networking project.
Comment, questions, concerns? :)
It's a reasonable idea, Aaron, and it makes sense to have OS-9 interface via Drivewire or CoCoNet rather than to attempt everything on the Coco. There is the issue that Drivewire and CoCoNet are different enough that it will be difficult to create a generic driver for NitrOS-9 that would function with both systems. Also there is sufficient competition and something close to animosity between the two camps that joint co-operation seems unlikely.
Why do I bring up CoCoNet? There are enough OS-9 users that have purchased Roger's system that if efforts were limited to Drivewire a significant number of OS-9 users would be out in the cold.
Unfortunately my lack of knowledge in the area of TCP/IP prevents me from contributing to this effort by generating any code.
I should clarify that this standard would not be part of DriveWire (or any specific project). I think it's very important to make it open and "neutral". DriveWire may well implement the standard, but DriveWire would certainly not define it and hopefully not be the only implementation. There are some projects underway that would add networking hardware directly to the CoCo. My conversations with the designers so far indicate that an open standard is something they would like to support and implement as well. There is no intention to limit this to DriveWire, quite the opposite.
At this time, Coconet does not provide any networking services under OS9 to my knowledge. It provides very rudimentary downloading of URLs using a DECB command, but that really isn't networking in the style that DW and the new networking hardware can provide. I don't know if Roger plans to add networking to his product as some point or not, but I did reach out to him and ask if he'd like to be involved in defining a standard. No response to date. If coconet does add networking, it would behoove his users if they could run the same software that DriveWire and other networking options use, and I will do anything I can to make this easy for Roger to do.
My goal is to standardize the way networking services are presented, not how they are implemented. For instance, there should be a standard tool to configure the interface IP address. How that request is actually performed by the device is of no concern, only that a user, by doing X, can configure an interface. Another example would be establishing an outbound TCP connection… if an application can open a path and make the connection in some standard way, then the details of how any particular device or software makes that happen don't matter to the application programmer.
One last detail.. If a standard way to do this can be created and agreed upon, I will port all of the DriveWire tools (telnet, inetd, the web server, smtp client, etc) so that they use the standard and can be used by any networking solution that also implements the standard. They will no longer be part of the DriveWire project, they'll just be.. there.. and available to anyone that wants them. I hope this makes it easier for any new coco networking projects.
I've put a lot of work into DriveWire, but I'd love to see it become unnecessary :)
I think the basic concept is sound. Done well, the API for different implementations (DriveWire, a dedicated controller,or even software on the CoCo) could be identical. That seems reasonably important, since our development community is already terribly small.
FWIW, I have a back-burner project to do a software TCP/IP implementation for the CoCo. I don't think it is at all unreasonable, depending on your purposes. Of course, I started it about 10 years ago - so I must not be in too much of a hurry… :-) Anyway, I have occasionally considered what an OS-9 API for such a stack would be. I think there are two reasonably good options.
One option would be a kernel extension module (e.g. kernelp3) that adds something like a socket system call. I think this isn't too difficult to implement - I believe there are community members that have written such things and I think there is some documentation available. This also has the advantage of offering a roughly-familiar API to application writers. The downsides are needing extra code to call from another languages such as C, Pascal, BASIC-09, etc and (perhaps more importantly) this is a low-level API that would be difficult to take advantage of DNS acceleration or whatever. It also isn't clear how one would configure the stack, although perhaps an extra syscall or two would suffice.
Another option I envisioned would be to implement a file manager that creates a namespace similar to the /dev/tcp and /dev/udp available in Bash:
This would provide a namespace that could be used with the normal APIs for opening and reading/writing files. That makes it immediately usable by any available programming language. It would also make it easy to offload DNS in particular, since you would open something like "/tcp/www.google.com/80" rather than doing a bunch of socket manipulation. Configuration would probably be done with a module like a device descriptor (which might work for the socket option as well). The downside would be having to implement a file manager, but I don't think this is insurmountable by any means.
Either of those options provide a reasonably clean and implementation-agnostic interface to application programmers. I think the file manager option would be easier on application writers. Of course, both of them are a bit more in-depth than simply using an SCF driver.
The main downsides I see for the SCF option are as you suggest: the API maps best to TCP and configuration or anything else special needs to somehow be done "in-band" (like with AT commands at the beginning). It probably is a bit easier for the implementors, but not if it means that you have to parse a bunch of in-band control information in the datastream. FWIW, I have always leaned towards the file manager option for my (mostly ignored) software stack project mentioned above, but since I never got that far… :-)
Well, I hope that is helpful. If nothing else, maybe it sheds a little light? Let's hope!
I like the idea of a file manager in the style of bash's /dev/tcp type tricks. I think it would be easy for users and developers to understand, and it fits well into the unified I/O style of OS9. Existing applications that use standard I/O should work nicely with such a thing. Also, the split between the networking API and various hardware or software solutions fits into the same model as the split between RBF and individual RBF device modules. The only downside is the complexity required in the file manager itself. It would be quite an undertaking, but a really nice way to do things.
Anybody want to help me work on something like that? I am willing to put in some time, but most of you have more experience and knowledge than I do.