[ Sorry this got so long; I ended up rambling ... ]
Date: Sun, 08 Jun 2003 11:32:56 +0200
From: Rudi Schlatte <rudi@...>
[Cc to Duane Rettig: I'd be glad for corrections if I misrepresented
your words or acl in any way]
I think you got it right, but since I am not the sockets expert here,
I myself may misrepresent what we believe here. I always start from
a purist view, to make the goal clear, and then I see how much I can
fit into that goal model. If it doesn't fit, it doesn't fit.
Daniel Barlow <dan@...> writes:
> Allegro's simple-streams interface includes a class called
> simple-socket-stream. However, it looks from the Allegro sockets
> documentation that the user is not expected to create socket-streams
> directly, but instead to call make-socket and let it select the
> appropriate subtype. That being so, I think the fact that their
> sockets are simple-streams is largely an implementation detail for
> So it seems that the directly instantiable simple-socket-stream class
> we have in sbcl contrib is a bonus. And what a lovely bonus. Here's
> the test case -
> (with-open-stream (s (make-instance 'socket-simple-stream
> :remote-host #(127 0 0 1)
> :remote-port 7))
> (string= (prog1 (write-line "Got it!" s) (finish-output s))
> (read-line s)))
> (sb-bsd-sockets::connection-refused-error () t))
IIUC, Allegro's make-socket opens the socket itself, then passes the
fd to make-instance of socket-simple-stream, but that's an
implementation artifact and not part of some protocol.
I asked Duane Rettig about the proper protocol for obtaining a socket
stream. He replied that (paraphrased) Allegro's socket implementation
carries some baggage from its Gray days, and that make-socket does
some work that could be handled by make-instance of socket stream.
Yes. I'm not sure how much of that work could be stuffed downward, but
I suspect more than might be expected. However, it is obvious from what
our sockets expert says and from this and further conversation that it
won't all fit. An object that is not a stream has no business in the
simple-streams protocol for its initialization.
Excerpt from the email exchange:
Rudi> (More generally, does the simple-stream protocol allow
Rudi> arbitrary options for make-instance / device-open?)
Duane> Yes, in fact that is the intended way for device-open options to
Duane> be specified. It is as if the make-instance/reinitialize-instance
Duane> were defined with &allow-other-keys, so that device-open can receive
Duane> options itself.
Yes. Note that you lose the automatic keyword checking ability that CLOS provides,
and device-open methods need to do what checking they desire.
That's how the :remote-host and :remote-port options to device-open
for socket-simple-streams came to be, even though they're not in the
Franz simple-streams documentation. I think any hairy socket options
can be passed down to device-open by that route.
With a cursory look, I don't see a :remote-host or :remote-port option to
device-open in our implementattion - ours are keywords to make-socket
instead. I assume then that you are talking about your own implementation.
> I think this is a thoroughly good thing and should be adopted as the
> basis of a high(er)-level socket interface in SBCL. I propose
> 1) a new class for each address family (internet, local-domain, ip
> version 6, decnet, chaosnet, etc). No, I'm not necessarily
> suggesting we need to implement all of these.
One should be careful here; having too many classes is one of the features
we balked at in Gray streams. They defined a class for each direction
(meaning three classes), a class for valence (character or binary), etc.,
and the combinatory explosion becomes unmanageable. It is best to try to
keep the class structure down to a manageable small set of major categories.
> 2) keyword arguments to the constructor for each of these that more or
> less follows the shape of Perl's IO::Socket::Foo modules
> (modulo using complete words and hyphens instead of StudlyCaps)
> (make-instance 'inet-socket :peer-host "www.google.com" :peer-port 80
> :protocol :tcp :type :stream :multihomed t)
> I can't put this stuff in sb-bsd-sockets as it currently stands
> because sb-simple-streams depends on it for the existing
> simple-socket-stream, and that would be circular. I don't think
> they're logically part of the simple-streams interface either, though.
> Can we lose simple-socket-stream as it currently stands (it could
>become the base class for these protocol family streams) if we get
>this stuff instead? Or should I add an SB-SOCKET-STREAMS contrib?
>That's a possibility, but I'm not sure it wins us anything useful
I'd say go the simple-streams route, but then I would. ;)
I would too, but then I would :-)
Actually, I should be careful here, because "go the simple streams
route" may mean something different to you than to me. It seems you are
already going the simple-streams route (that is a Good Thing), but be
careful not to put anything into your simple-streams that doesn't
As I see it, there's two parts of this:
- Create something that's a source / sink of a stream of bytes.
For files, there's sb-unix:unix-open, sb-bsd-sockets provides opened
socket fds, etc.
- Implement Lisp-stream semantics for that object, provide buffering.
Simple-streams does this.
Would it work to extend device-open for socket-simple-streams with
additional initargs instead of constructing a class hierarchy?
:protocol :chaosnet :funky-chaosnet-option 42
That is, as long as (device-open socket-simple-stream) can glean from
the passed options what type of byte source / sink to obtain from
sb-bsd-sockets, the socket-simple-stream class machinery should work
with all of them.
Server sockets and, perhaps, datagram sockets are another topic:
server sockets aren't streams, and I'm not sure if seeing a udp socket
as a stream is the right thing.
Here's what our sockets expert says:
There are lot of different sockets (active/passive, internet/file,
stream/datagram) and *if* all these are represented by different
classes then one should not have to remember all these class
names. I believe that everyone agrees on that.
What acl does is have a master socket creator function (make-socket).
You only have to know that function name.
What I read from the email you forwarded me is that people
are considering a master socket class (simple-socket-stream)
and doing a make-instance of that will do the work of acl's make-socket.
Then people point out (and rightly so) that you get into trouble
right away. You don't simple-socket-stream to include datagrams
since they are special streams and you don't want simple-socket-stream
to include passive streams since they aren't streams either.
So (make-instance 'simple-socket-stream) can only do a tiny fraction
of what (make-socket) does, and you're left telling people that
they have to remember what class names to use for each
kind of socket they want to build. This is precisely the situation
we wanted to avoid.
I largely agree with this. There is a dissonance between the
pure-stream nature of simple-streams and the hybrid stream/non-stream
nature of sockets. If the stream-like nature can be separated cleanly,
then that part of it can be put squarely into the simple-streams
I also consider udp to not be of stream-nature (udp packets are
building blocks for streams at a higher level). I don't know how
many ever really define "stream" (the ANSI dictionary definition
is a little weak in identifying the _nature_ of a stream), but
fundamental to my own understanding of stream-like behavior is the
in-order movement of data
That said, there are actually several places where operations that do
not look like stream operations might very well be. Our
internationalization expert and I have had a running disagreement as to
whether external-format processing was a stream-like operation or not.
I contended that it was, because every external-format operation could
be thought of in a stream-like way. He pointed out the one exception
to that; buffer-conversion - we have several related and oft-renamed
functions, one pair of which is native-to-string and string-to-native,
which convert between buffers of native raw octet data and lisp string
data as efficiently as possible. My answer to that was to implement
these two functions using streams, and in fact by using resource
management efficiently, my version of these two functions was actually
more efficient than his version (which in fact was fairly well-tuned, but
whose external-formats had to make exceptions for those non-stream-like
parts of the operation). We went with my version. We still have a
(healthy) disagreement as to whether external-formats really belong
in streams, but our compromise was that I let him keep the (huge) task
of external-formats maintenance, and he let me remove the non-stream
aspects of our _implementation_ of external-formats. The moral of
this paragraph is that sometimes operations do indeed fit into streams
that might not otherwise be thought of as stream-like.