[clisp-list] Re: FFI howto: variable length arrays (long)

[Hoehle, Joerg-C. <Joe...@t-...>]

Sam Steingold wrote:
>yuk.  CLISP has %ELEMENT.  Why not use ELEMENT for the exported
>interface if it is to be different from %ELEMENT (or just rename
>%ELEMENT to ELEMENT and export it if they are the same)?
>ELEMENT* looks too much like MEMBER* in Emacs-Lisp.

They are not the same.

Element has been a macro providing compile-time error checking since day 1 of the FFI. I don't want to change that.

ELEMENT place indices* -> value

What I want to introduce is a function operating on a foreign-variable object.

ELEMENT* fv indices* -> value

I choose a trailing star to indicate proximity to the macro.
My Emacs-20.7 does not have member*.
A trailing or preceding % has a long Lisp history of "system-internal - don't dare to touch" which is not suitable for an exported interface.


BTW, one does not see much of a difference with ELEMENT. Let's talk about CAST instead. CAST additionaly takes a foreign type description.

I realized yesterday that I forgot talking about another advantage of ELEMENT/CAST etc.: so far they successfully managed (for better or worse) to hide the internal foreign type representation from the programmer. I want to define a lower-level interface where this becomes visible.

CAST place foreign-type -> place ; well-known, no change
 (CAST dest `(c-array uint8 ,actual))

CAST* foreign-variable internal-type -> foreign-variable
 (CAST* dest-fv (PARSE-C-TYPE `(c-array uint8 ,actual)))

I actually don't want
 (CAST* dest-fv `(c-array uint8 ,actual))
because
1. that would mean there's still another level below (the current %CAST) which would operate on the internal type instead of high-level type descriptions;
  (defun CAST* (fv type) (%CAST fv (PARSE-C-TYPE type))) ; no!
2. programmers would be able to handle type descriptions like objects, store them, pass them around etc.;
3. and still have guarantees that the amount of run-time parsing is minimised.


The same thought applies to FOREIGN-ADDRESS-VARIABLE/FUNCTION and all functions which accept type descriptions.

Another approach, maybe to be called "status quo", would be
- do not define nor export CAST* etc.
+ instead, use WITH-FOREIGN-PLACE and its converse C-VAR-OBJECT
 and have the programmer always operate on places.
- concerns about danger of automatic dereferencing still apply.

(defun foo (n)
  (simple-calloc 'uint8 :count n :read-only t))
(setq x (foo 300))
(some-foreign-function x)

(defun checksum (fv-object)      ; extremely inefficient in CLISP
  (with-foreign-place (buffer fv-object)
   (values
    (loop for i from 0 below (SIZEOF buffer)
          ;; only works on byte buffers where sizeof = length
          ;; maybe should use (third (TYPEOF buffer))
          sum (ELEMENT buffer i))
    (C-VAR-OBJECT (offset memory 0 `(c-array-max character 32))))))
(checksum x) -> 5234 ; #<FOREIGN-VARIABLE of type (c-array-max character 32)>

Let's not forget: my article did not mention that operating on foreign places is in CLISP an extremely expensive operation. The above checksum functions runs *several* orders of a magnitude slower than its native code counterpart which one could code using CMUCL, CormanLisp, xyz-Lisp or C.
1. byte-code overhead, not to be avoided in CLISP
2. ELEMENT overhead: for every index position, a new foreign-variable object is created, then thrown away...

But with-foreign-place does not help for FOREIGN-ADDRESS-FUNCTION etc., only for ELEMENT, SLOT, CAST, OFFSET.

Regards,
	Jorg Hohle.