[the same topic arose independently on both clisp and uffi mailing =
lists. I write to both.]
When a C programmers calls
x =3D foo(...,pointer or &array, ...) s/he defines an implicit =
1) the pointer points to data that will be valid at the time it will =
E.g. it may be a reference to an array to be filled up by the call, or =
installing an i/o buffer or callback address for later use.
When a Java/Lisp/Smalltalk programmers invokes a foreign function, the =
contract is slightly different:
2) the pointer into a Lisp/Java object is valid for the duration of =
Even the weaker contract 2) can be endangered in implementations that =
i) a moving (compacting) GC *and*
ii) callbacks into Lisp/Java or programmable signal handlers
E.g. suppose a signal handler or a callback is invoked while in foo. =
The handler calls Lisp code, which causes a GC. The address of the =
double float array passed to foo may not be valid anymore, leading to =
One might be tempted to ask: "what the heck? in my code I don't have a =
My answer is that only the system architect, s/he who knows
the complete system can hold such a claim about the system as a whole.
The individual component writer cannot know whether some
other component installed e.g. some signal handler in Lisp.
Ignoring this issue yields to unreliable "it works for me" solutions.
I'm opposed to simple looking solutions that only work in limited,
unspecified contexts. Such solutions are obviously too simple than
possible (w.r.t. Albert Einstein's famous quote).
There are several solutions:
a) mark Lisp objects passed as reference to be non-movable (during the =
foreign call), or
b) don't pass pointers to Lisp objects, use space on the
execution stack instead. That's what CLISP (foreign-call-out,
with-foreign-object), CMUCL (with-alien) and
other implementations do, or
c) use a non-moving GC, e.g. Boehm's conservative GC. However doing
so precludes nice GC design, probably generational GC and other
modern features of advanced GC schemes, or
d) add explicit declarations to the effect that the given foreign
call (e.g. memcpy) is free of callbacks (while XmainLoop() is not).
e) Give guarantees and a model about the context in which Lisp-level
signal handlers are called by the implementation (it's typically
delayed outside the signal handler). E.g. in CLISP, they could be
invoked by the bytecode interpreter loop.
f) forbid passing addresses of Lisp objects to functions not declared
free of callbacks in implementations where Lisp-level signal =
can be called anytime.
I hope this made very clear why I claim that the limited declarative =
power of the popular Allegro construct
(vec (:array :double)))
where vec is used for i/o and writes directly to a Lisp double float =
array is dangerous -- outside of Allegro. Such a declaration cannot be =
the base for a UniversalFFI. One has to consider i) and ii) above.
That's also why there must *not* be a (WITH-LISP-ARRAY-ADDRESS =
(addr-var array-obj) &body) in CLISP. One can only do so in limited =
contexts, e.g. for calling recv() and be sure (not hope) never to GC =
before doing so.
Oh, BTW, the AFFI I designed in 1994/5 as an FFI for Amiga-CLISP passes
addresses to Lisp arrays. It has a i) moving GC but no ii) callbacks
nor signal handlers, thus no reliability problem (not considering
buffer overflows and heap corruption here :-) and thus fulfils contract =
I'm specifically not saying that the ability to pass address of Lisp
arrays to foreign code wouldn't be useful or speedy. But it also
has to be reliably defined.
who's working on SW-reliability and dependability at
Deutsche Telekom / T-Systems.