CLISP support for a new portable FFI library

[James B. <ja...@ja...>]

Hi,

(Beware, this is a long one!)

I'm working on a new portable foreign function interface, CFFI, and
would like to support CLISP in my initial release.

The design approach I am taking with this library is to isolate a
small set of core operators needed to do basic foreign memory access
and call functions.  On top of these primtives, a portable foreign
type system for structure and aggregate types can be developed in
straight ANSI CL.

I've written a first draft of a specification for the implementation
specific interface at:

   http://www.jamesjb.com/cffi-sys-specification.html

A partial implementation of this specification for CLISP (and working
implementations for SBCL and OpenMCL) are at:

   http://www.jamesjb.com/cffi-clisp.lisp
   http://www.jamesjb.com/cffi-sbcl.lisp
   http://www.jamesjb.com/cffi-openmcl.lisp

The whole library (including a simple foreign type system implemented
in terms of the CFFI-SYS interface) will be available soon at
common-lisp.net.  There is also a (quite prelimiary) UFFI
compatibility layer.

There are a few operators that I didn't figure out how to implement
for CLISP (quoting from the spec):

  Accessor: %MEM-REF ptr type &optional (offset 0)

  Dereference a pointer OFFSET bytes from PTR to an OBJECT of built-in
  foreign type TYPE.  Returns the object as a Lisp value or foreign
  pointer.  When used with SETF this writes to the memory location.

  PTR is an FFI:FOREIGN-ADDRESS object.  TYPE is a keyword denoting a
  foreign type that can be converted to a CLISP C type.

As CFFI requires the user to specify type information when
dereferencing foreign pointers, I have tried to use untyped pointers
in the form of FFI:FOREIGN-ADDRESS objects.

However, I have not found a way to get something back from the
FOREIGN-ADDRESS that I can dereference.  I imagine %MEM-REF would be a
fairly simple operator to write in the C portion of the FFI.

The second operator is:

  Macro: %FOREIGN-FUNCALL name {arg-type arg}* &optional result-type

  Calls a foreign function passing arguments of type ARG-TYPE and
  value ARG, returning an object of type RESULT-TYPE, which defaults
  to :void.

  This is perhaps best illustrated by some examples:

  (%foreign-funcall "sqrtf" :float 16.0 :float) => 4.0
  (%foreign-funcall "time" :pointer (null-ptr) :unsigned-long)
    => 1118131064

It looks to me like it is necessary when using DEF-CALL-OUT to specify
the library name each time a foreign function that is to be found in a
shared library is defined.  In order to implement %FOREIGN-FUNCALL
compatibly with other Lisps, it would need to do something like loop
over all the loaded libraries and use the first symbol matching the
requested name.

The implementation is also missing a %LOAD-FOREIGN-LIBRARY function,
which I couldn't find a public interface for in CLISP.

This interface is still pretty fluid---please feel free to send me any
feedback you might have about improving it.  Particularly, foreign
variable access and C->Lisp callbacks are missing; we will need
another operator similar to %FOREIGN-FUNCALL for variables.

Ideally, someone familar with CLISP FFI internals could implement the
necessary primitives or point me at something I overlooked.  Otherwise
CLISP will not be supported until I figure out how to write the patch
myself. :-)

James