#144 symbol-macrolet not always expanded

lisp error
closed-fixed
Sam Steingold
clisp (525)
5
2003-02-14
2003-01-31
Jörg Höhle
No

Some obsure macroexpand or walker problem.
SYMBOL-MACROLET doesn't always expand.

(in-package "FFI")

(setq fx (ffi::lookup-foreign-variable
"ffi_user_pointer" (ffi::parse-c-type '(c-array
character 4))))
;-> #<FOREIGN-VARIABLE "ffi_user_pointer" #x00668C0C>

(defmacro with-foreign-var ((var fvar) &body body)
(let ((fv (gensym (symbol-name var))))
`(LET ((,fv ,fvar))
(SYMBOL-MACROLET ((,var (FOREIGN-VALUE ,fv)))
,@body))))

(with-foreign-var (x fx) (typeof x))
*** - TYPEOF is only allowed after FOREIGN-VALUE:
(TYPEOF X)

(with-foreign-var (x fx) (identity (typeof x)))
;-> (C-ARRAY CHARACTER 4)

(with-foreign-var (x fx) (element x 0))
*** - ELEMENT is only allowed after FOREIGN-VALUE:
(ELEMENT X 0)

(with-foreign-var (x fx) (values (element x 0)))
;-> #\Null

(with-foreign-object (fv '(c-array-max character 4)
"ab")
(with-foreign-var (x fv) (typeof x)))
;-> (C-ARRAY-MAX CHARACTER 4)
; works even without identity!?!

BTW, if somebody wants to suggest a better name?
; WITH-C-VAR is like a composition of
WITH-FOREIGN-OBJECT and WITH-FOREIGN-VAR

Using CLISP-2.28 built with MS-VC6

Regards,
Jörg Höhle.

Discussion

  • Sam Steingold
    Sam Steingold
    2003-01-31

    Logged In: YES
    user_id=5735

    typeof signals the above error at macroexpansion time.
    typeof must be macroexpanded before x is replaced by its
    symbol-macro expansion.
    you will need to modify typeof -- and all the others like it...

     
  • Sam Steingold
    Sam Steingold
    2003-01-31

    • status: open --> closed-invalid
     
  • Jörg Höhle
    Jörg Höhle
    2003-02-03

    • status: closed-invalid --> open-invalid
     
  • Jörg Höhle
    Jörg Höhle
    2003-02-03

    Logged In: YES
    user_id=377168

    I don't understand what you mean.
    typeof has been macroexpanding its argument for half a
    decade (and looks ok to me):
    (defmacro typeof (place &environment env)
    (setq place (macroexpand place env))
    This doesn't look like a FFI bug to me, but a macrolet or
    symbol-macrolet one, or perhaps an interpreter bug.
    The error message mentioning place obviously shows that the
    above macroexpand didn't find that X was a (symbol-)macro.

     
  • Jörg Höhle
    Jörg Höhle
    2003-02-03

    Logged In: YES
    user_id=377168

    It really looks like a bug in the interpreter to me:
    (let () (declare (compile)) (with-foreign-var (x fx) (typeof
    x)))
    WARNING in function #:COMPILED-FORM-36 :
    FX is neither declared nor bound,
    it will be treated as if it were declared SPECIAL.
    (C-ARRAY CHARACTER 4)
    No bogus error, it works in compiled mode.

    (locally (declare (compile)) (with-foreign-var (x fx)
    (typeof x))) works
    (locally (with-foreign-var (x fx) (typeof x))) breaks

     
  • Sam Steingold
    Sam Steingold
    2003-02-03

    • status: open-invalid --> open-accepted
     
  • Sam Steingold
    Sam Steingold
    2003-02-03

    Logged In: YES
    user_id=5735

    ok, this IS a bug in typeof (foreign1.lisp).
    look at
    <http://www.lisp.org/HyperSpec/Body/fun_macroexpa_acroexpand-1.html>:
    (let ((x (list 1 2 3)))
    (symbol-macrolet ((a (first x)))
    (macroexpand 'a)))
    A, false
    i.e., macroexpand DOES NOT pick up the symbol-macrolet
    expansions.
    macroexpand should be replaced with expand-form (which does).
    please try that the report your experience here.
    thank you.

     
  • Jörg Höhle
    Jörg Höhle
    2003-02-04

    Logged In: YES
    user_id=377168

    Your counterexample is broken.
    There is no way MACROEXPAND can pick up symbol-macrolet or
    macrolet expansions without being given a second argument:
    the environment.
    The TYPEOF etc. macros do obtain and pass &environment.

    I'm wondering what expand-form is that you mention. It's not
    in CLHS and my clisp 2.28 says
    FFI[13]> (apropos "expand-form")
    SYSTEM::%EXPAND-FORM function
    SYSTEM::MACROEXPAND-FORM function

    I have doubts that the perfectly (to me) legal CL code using
    symbol-macrolet, macroexpand etc. needs CLISP specific
    extensions to work.

    Now I remember that one year ago, I hit this same kind of
    error when introducing C-VAR-ADDRESS into foreign1.lisp. I
    vaguely remember that it seemed to disappear after I got
    that macro with the other ones like typeof into the file,
    instead of using interactive definitions. Back then, I
    apparently did not file a bug report.

    BTW, something maybe strange I just noticed: init.lisp
    defines %expand-form and dispatches using #+/-COMPILER.
    However, in final image generation (using only .fas),
    "compiler" is loaded far after that DEFUN place, so it is
    the #-compiler section that gets defined. Is this intended?

     
  • Sam Steingold
    Sam Steingold
    2003-02-04

    Logged In: YES
    user_id=5735

    can you give me a self-contained test case which does not
    reference FFI?

     
  • Jörg Höhle
    Jörg Höhle
    2003-02-13

    Logged In: YES
    user_id=377168

    Here's code which I arbitrarily put into package FFI, but
    does not need it.
    (in-package "FFI")
    (defmacro with-foreign-var ((var fvar) &body body)
    (let ((fv (gensym (symbol-name var))))
    `(LET ((,fv ,fvar))
    (SYMBOL-MACROLET ((,var (FOREIGN-VALUE ,fv)))
    ,@body))))
    ;(with-foreign-var (x "fake foreign variable") (ffi:typeof
    x)) breaks

    (defmacro my-typeof (place &environment env)
    (let ((exp-place (macroexpand place env)))
    (format t "MY-TYPEOF expansion: ~S~%" exp-place)
    (unless (and (consp exp-place)
    (eq (car exp-place) 'FOREIGN-VALUE))
    (error "MY-TYPEOF not upon a place: ~S" exp-place))
    `(format t "Foreign type from ~S~%" ,(second
    exp-place))))
    ;(macroexpand-1 '(my-typeof x)) -> error
    ;(macroexpand-1 '(my-typeof (foreign-value x)))

    ;(with-foreign-var (x "fake foreign variable") (my-typeof
    x))

    ; sometimes works non-reproducibly:
    1. Break FFI[69]>
    MY-TYPEOF expansion: (FOREIGN-VALUE #:X1834)
    MY-TYPEOF expansion: (FOREIGN-VALUE #:X1835)
    Foreign type from "fake foreign variable"
    NIL

    ; but usually
    FFI[72]>
    MY-TYPEOF expansion: X
    *** - MY-TYPEOF not upon a place: X

    While embedding inside progn, identity or even with multiple
    forms makes the code work:

    ;(with-foreign-var (x "fake foreign variable") (identity
    (my-typeof x)))
    FFI[86]> MY-TYPEOF expansion: (FOREIGN-VALUE #:X1843)
    Foreign type from "fake foreign variable"
    NIL

    ;(with-foreign-var (x "fake foreign variable") 1 (my-typeof
    x)) works
    ;(with-foreign-var (x "fake foreign variable") "ab"
    (my-typeof x)) breaks
    ;(with-foreign-var (x "fake foreign variable") (progn
    (my-typeof x))) works
    Exactly as with typeof in the original bug report.

    I find the "ab" (my-typeof x) case revealing: obviously "ab"
    is
    ignored, but it doesn't seem to be treated as a
    documentation string
    either, since adding one of the above progn, identity or
    list wrappers
    leads to:
    ;(with-foreign-var (x "fake foreign variable") "ab" (progn
    (my-typeof x)))
    *** - SYMBOL-MACROLET: doc-strings are not allowed here:
    ("ab" (PROGN (MY-TYPEOF X)))

    Maybe that's a hint, may be a red herring.

     
  • Sam Steingold
    Sam Steingold
    2003-02-14

    • status: open-accepted --> closed-fixed
     
  • Sam Steingold
    Sam Steingold
    2003-02-14

    Logged In: YES
    user_id=5735

    thank you for your bug report.
    the bug has been fixed in the CVS tree.
    you can either wait for the next release (recommended)
    or check out the current CVS tree (see http://clisp.cons.org\)
    and build CLISP from the sources (be advised that between
    releases the CVS tree is very unstable and may not even build
    on your platform).