CLISP - an ANSI Common Lisp

If character input is possible on stream
(either bivalent or character), then LISTEN
listens for character.

And only in case if stream is binary, LISTEN
listens for byte.

Strictly speaking, the proposed behaviour is
a violation of ANSI standard, because LISTEN
should only return T if a character available;
on binary-only stream characters are never
available, so LISTEN should return nil.

But it may be convenient and not harmful. Without
that it's impossible to perform nonblocking input
from, for example, binary socket stream.

Actually, I started investigating it because
I'm using a library (http://weitz.de/chunga),
that assumes LISTEN to be working with binary
streams. I was trying to use nonblocking
IO to optimize code that reads from
multiple sockets.

In fact having LISTEN working didn't really help
me; asynchronous IO would be much more efficient.
So, lisp IO looks slightly inconsistent and
poor by contemporary measures for me.

There are no bivalent streams in CLISP as AFAIK Lispworks provides them:
where you can write either #\a or 65 to e.g. a socket. With clisp, you
need to switch the stream-element-type beforehand.

What Sam may have asked is how your change proposal would work should
clisp implement these one day.

Also, there are streams where you can declare the element type to both
character and binary i/o. How would listen behave? The existence of such
beasts has lead to the introduction of the non-standard functions
READ-BYTE-SEQUENCE and READ-CHAR-SEQUENCE into clisp to supplement
READ-SEQUENCE -- and WRITE-SEQUENCE.
http://clisp.cons.org/impnotes/stream-dict.html#bulk-io

Actually, I wonder which streams support both character and binary or
INTEGER i/o *simultaneously* right now in clisp?

And the code:

*************** file clisp/src/stream.d **************

# UP: Determines whether an element of element-type corresponding
# to the stream is immediately available.
# I.e. in case when stream supports character input, tests for
# character immediately availably, otherwise tests for byte
# listen_element(stream)
# > stream: a stream
# < result: ls_avail if an element is available,
# ls_eof if EOF is reached,
# ls_wait if no element is available, but not because of EOF
# can trigger GC
global maygc signean listen_element (object stream) {
if (TheStream(stream)->strmflags & strmflags_rd_ch_B) {
return listen_char(stream);
} else {
return listen_byte(stream);
}
}

>LISTEN may care only about characters when
>(SUBTYPEP 'CHARACTER (STREAM-ELEMENT-TYPE s)).

I'm not very precise here, because according
to CLHS stream element type may be a subtype
of character.

What I mean is if stream element type allows
character reading, LISTEN considers only
characters.

>what about bivalent streams?

The simplest approach is to leave bivalent
streams working exactly as they were
working before. The only extension is
that LISTEN checks for byte availability
if stream is binary only.

I'm not sure how LISTEN treats bivalent streams
at present.

Taking into account that CLHS requires
LISTEN only to check for character availability,
LISTEN may care only about characters when
(SUBTYPEP 'CHARACTER (STREAM-ELEMENT-TYPE s)).

By the way, could you point me to any
information about bivalent streams in clisp?
I saw some mentions about element type
of (or character ([un]signed-byte 8)) in
the sources, but I can't find any documentation.

>As it turned out it isn't difficult to extend LISTEN semantics to be
>"check if an element is available in stream". I.e. for character stream
>"whether a char available?" and for byte streams "whether a byte
>available?".

indeed.

>Is such semantics acceptable?

what about bivalent streams?

As it turned out it isn't difficult to extend LISTEN semantics to be
"check if an element is available in stream". I.e. for character stream
"whether a char available?" and for byte streams "whether a byte
available?".

Is such semantics acceptable?

Note, with this semantics

(setf (stream-element-type *stream-var*) '(unsigned-byte 8))
(when (listen *stream-var*)
(setf (stream-element-type *stream-var*) 'character)
(read-char *stream-var*))

still may block when multibyte encoding is used as an external
format for the *stream-var*.

By the way, improving LISTEN to work with binary streams is one of the
issues in the clisp/src/TODO file.

Here is how it may be implemented:

******* file clisp/src/lispbibl.d **********

# UP: Determines whether an element of corresponding element-type
(CHARACTER or
# ([UN]SIGNED-BYTE 8) is immediately available.
# listen_element(stream)
# > stream: a stream
# < result: ls_avail if an element is available,
# ls_eof if EOF is reached,
# ls_wait if no element is available, but not because of EOF
# can trigger GC
extern maygc signean listen_element (object stream);
# is used by IO

************* file clisp/src/stream.d ***************

# UP: Determines whether an element of corresponding element-type
(CHARACTER or
# ([UN]SIGNED-BYTE 8) is immediately available.
# listen_element(stream)
# > stream: a stream
# < result: ls_avail if an element is available,
# ls_eof if EOF is reached,
# ls_wait if no element is available, but not because of EOF
# can trigger GC
global maygc signean listen_element (object stream) {
var decoded_el_t eltype;
test_eltype_arg(&TheStream(stream)->strm_eltype,&eltype);
if (eltype_ch == eltype.kind) {
return listen_char(stream);
} else {
return listen_byte(stream);
}
}

********* file io.d ***************

# (LISTEN [input-stream]), CLTL p. 380
LISPFUN(listen,seclass_default,0,1,norest,nokey,0,NIL) {
check_istream(&STACK_0); /* check input-stream */
if (builtin_stream_p(STACK_0))
VALUES_IF(ls_avail_p(listen_element(popSTACK())));
else funcall(S(stream_listen),1);
}

I've tried this with sockets.

P.S. Sorry for this hand-made 'patch', but I'm using
a version older than CVS head.

True indeed, trivial-sockets does not (as far as I know) provide a common
LISTEN/STATUS functionality.

Obviously this means that trivial-sockets must export its own
LISTEN/STATUS-like functionality in order to provide a portable API. It
cannot rely on its users to call CL:LISTEN instead. (I've never looked at
trivial-sockets).
Does it?

Logged In: YES
user_id=5735

I am sorry, I missed the "non-char" part.

http://www.lisp.org/HyperSpec/Body/fun_listen.html:
Returns true if there is a character immediately available
from input-stream...

since a character is never available from a non-char stream,
listen always returns nil there.
you can use socket-status for non-char streams.

Logged In: YES
user_id=162311

[3]> (trivial-sockets:open-stream "localhost" 12345
:element-type '(unsigned-byte 8))
#<IO UNBUFFERED SOCKET-STREAM (UNSIGNED-BYTE 8) localhost:12345>
Same behaviour when using socket:socket-connect.
trivial-sockets is just a wrapper.

this is the standard request for more information respons:
1. uname -a: Linux zion 2.6.18-gentoo #1 SMP Mon Oct 23
19:39:52 CEST 2006 x86_64 AMD Athlon(tm) 64 Processor 3200+
AuthenticAMD GNU/Linux (ops smp, i think i have to do some
recompiling)
gcc (GCC) 4.1.1 (Gentoo 4.1.1)
glibc-2.4-r4 (gentoo)
2. some default gentoo mirror. version 2.39.
3. gentoo compilation process.
4.
[1]> (lisp-implementation-version)
"2.39 (2006-07-16) (built 3367467198) (memory 3367467469)"
4.
[2]> *features*
(:ASDF :CLC-OS-DEBIAN :COMMON-LISP-CONTROLLER :ZLIB :PCRE
:CLX-MIT-R5 :CLX-MIT-R4 :XLIB
:CLX :CLX-LITTLE-ENDIAN :RAWSOCK :WILDCARD :READLINE
:REGEXP :SYSCALLS :I18N :LOOP
:COMPILER :CLOS :MOP :CLISP :ANSI-CL :COMMON-LISP :LISP=CL
:INTERPRETER :SOCKETS
:GENERIC-STREAMS :LOGICAL-PATHNAMES :SCREEN :FFI :GETTEXT
:UNICODE
:BASE-CHAR=CHARACTER :PC386 :UNIX)

Logged In: YES
user_id=5735

this is the standard request for more information.
1. what is your platform?
("uname -a" on a Unix system)
compiler version? libc (on Linux)?
2. where did you get the sources? when?
(absolute dates are prefered over the relative ones)
3. how did you build CLISP? (what command, options &c)
please do a clean build (remove your build directory and
build CLISP with "./configure --build build" or at least
do a "make distclean" before "make")
4. if you are using pre-built binaries, the problem is likely
to be in the incompatibilities between the platform on which
the binary was built and yours;
please try compiling the sources.
5. what is the output of (lisp-implementation-version)?
6. what is the value of *features*?
7. please supply the full output (copy and paste)
of all the error messages.
If you cannot build CLISP, you can obviously skip 5 and 6,
but then you should provide more information in 1.
please see <http://clisp.cons.org/clisp.html#bugs>
for more information.
Thanks.

PS. This bug report is now marked "pending"
and will auto-close unless you respond
(in which case it will auto-re-open).

Logged In: YES
user_id=5735

what does sock evaluate to?
what is "trivial-sockets:open-stream"?
does socket-status work?