Steel Bank Common Lisp

On Tue, 12 Oct 2004, John Morrison wrote:

> Based upon a backtrace and my painfully limited knowledge of
> macrology, I think that the macro named sb-alien:define-alien-routine
> actually tries to reference the foreign dynamic symbol during the
> macroexpansion.  If it's not there (say in the case of some function
> declared by an included system file like "typeinfo" yet not defined in
> the already-loaded foreign shared-library), then I land in the
> debugger as a result of a simple-error while loading the alien
> bindings.

Correct. You can't use DEFINE-ALIEN-ROUTINE before the corresponding 
shared object is loaded (unless the symbol is available in the sbcl 
runtime directly.)

Why can't you load the shared object earlier? (If it's a library you
use that does this, then it's a bug in the library, IMO.)

OTOH, I do have a vague plan how lazyness could be achieved there,
but the prospect isn't really attractive.

In short: if the foreign symbol isn't there yet turn the DEFUN
generated by DEFINE-ALIEN-ROUTINE into something like

    (defun foo (&rest args)
      (let ((*alien-definition-or-lose* t))
        (eval <original define-alien-routine form>)
      (apply 'foo args))

*eugh* -- I've the feeling there are going to be cases of this behaving 
badly that i can't think of right now...

Cheers,

  -- Nikodemus              Schemer: "Buddha is small, clean, and serious."
                   Lispnik: "Buddha is big, has hairy armpits, and laughs."

Hi;

Thanks for replying...

On Tuesday 12 October 2004 10:53 am, Nikodemus Siivola wrote:
> Why can't you load the shared object earlier? (If it's a library you
> use that does this, then it's a bug in the library, IMO.)

Sorry for not being more clear.  I *do* load the shared object first,
and then load the "bindings.lisp" file.  The problem is that if some
of the c++ symbols referenced in the lisp bindings file are *not* in
the shared object, then I get a simple-error that seems to be hard to
catch/handle.

FYI, I am automatically generating the lisp bindings file via the
output of GCCXML, using XMLS and some Common Lisp.

I think I have handled the "obvious" cases of "artificial" functions
(e.g., copy constructors) defined yet not always generated by GCC, and
I prevent them from appearing in the bindings.

However, if system header files are included in any of the source
(say, <stdio.h>, <typeinfo>), they may declare system functions that
will not be in my shared library.  I am not sure if it's computable to
identify them and prevent their inclusion in the auto-generated lisp
bindings.  I have to think about this.

Anyway, that's why I wanted to ignore any errors like that.

> OTOH, I do have a vague plan how lazyness could be achieved there,
> but the prospect isn't really attractive.
>
> In short: if the foreign symbol isn't there yet turn the DEFUN
> generated by DEFINE-ALIEN-ROUTINE into something like
>
>     (defun foo (&rest args)
>       (let ((*alien-definition-or-lose* t))
>         (eval <original define-alien-routine form>)
>       (apply 'foo args))
>
> *eugh* -- I've the feeling there are going to be cases of this behaving
> badly that i can't think of right now...

OK.  Thanks.  I am sure I will have more stupid questions later.

Thanks again,

-jm

>
> Cheers,
>
>   -- Nikodemus              Schemer: "Buddha is small, clean, and serious."
>                    Lispnik: "Buddha is big, has hairy armpits, and laughs."

-- 
==== John Morrison
==== MAK Technologies Inc.
==== 10 Fawcett Street, Cambridge, MA 02138
==== http://www.mak.com/
==== vox:617-876-8085 x115
==== fax:617-876-9208
==== jm@...

On Tue, 12 Oct 2004, John Morrison wrote:

> I think I have handled the "obvious" cases of "artificial" functions
> (e.g., copy constructors) defined yet not always generated by GCC, and
> I prevent them from appearing in the bindings.
>
> However, if system header files are included in any of the source
> (say, <stdio.h>, <typeinfo>), they may declare system functions that
> will not be in my shared library.  I am not sure if it's computable to
> identify them and prevent their inclusion in the auto-generated lisp
> bindings.  I have to think about this.
>
> Anyway, that's why I wanted to ignore any errors like that.

Aha, gotcha. So presumably signalling UNKNOWN-FOREIGN-FUNCTION-ERROR or
some-such would solve this for you -- as then you'd be able to handle 
those?

Cheers,

  -- Nikodemus              Schemer: "Buddha is small, clean, and serious."
                   Lispnik: "Buddha is big, has hairy armpits, and laughs."

Hi;

On Wednesday 13 October 2004 03:35 am, Nikodemus Siivola wrote:
> Aha, gotcha. So presumably signalling UNKNOWN-FOREIGN-FUNCTION-ERROR or
> some-such would solve this for you -- as then you'd be able to handle
> those?

I think my problem is actually more fundamental than that (hey, be
nice! :-).  It's the fact that I don't know how to "use" a macro
(e.g., define-alien-routine) that might signal an error (whether
SIMPLE-ERROR or UNKNOWN-FOREIGN-FUNCTION-ERROR) during macroexpansion.

Basically, my ugly eval+quote kludge (if I can so dignify it by
calling it a kludge) delays macroexpansion until after the
error-catching form becomes active.

I have to think, or at least hope, that this (an error-signalling
macro) is a Well-Known Problem with a Well-Known Solution.

If I only knew what that Well-Known Solution was!

-jm

>
> Cheers,
>
>   -- Nikodemus              Schemer: "Buddha is small, clean, and serious."
>                    Lispnik: "Buddha is big, has hairy armpits, and laughs."
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Sbcl-help mailing list
> Sbcl-help@...
> https://lists.sourceforge.net/lists/listinfo/sbcl-help

-- 
==== John Morrison
==== MAK Technologies Inc.
==== 10 Fawcett Street, Cambridge, MA 02138
==== http://www.mak.com/
==== vox:617-876-8085 x115
==== fax:617-876-9208
==== jm@...

John Morrison <jm@...> writes:

> Hi;
>
> On Wednesday 13 October 2004 03:35 am, Nikodemus Siivola wrote:
>> Aha, gotcha. So presumably signalling UNKNOWN-FOREIGN-FUNCTION-ERROR or
>> some-such would solve this for you -- as then you'd be able to handle
>> those?
>
> I think my problem is actually more fundamental than that (hey, be
> nice! :-).  It's the fact that I don't know how to "use" a macro
> (e.g., define-alien-routine) that might signal an error (whether
> SIMPLE-ERROR or UNKNOWN-FOREIGN-FUNCTION-ERROR) during macroexpansion.
>
> Basically, my ugly eval+quote kludge (if I can so dignify it by
> calling it a kludge) delays macroexpansion until after the
> error-catching form becomes active.
>
> I have to think, or at least hope, that this (an error-signalling
> macro) is a Well-Known Problem with a Well-Known Solution.
>
> If I only knew what that Well-Known Solution was!

To be fair to you, I'm not sure that there is one, but in defence of
the alien interface (which does a lot of heavy lifting at compile-time
so that access to alien thingies doesn't completely kill performance)
it's not clear what could be done to mitigate this either.  At least,
I'm confused.

I think, in fact, that error-signalling macros are relatively rare
(except for trivial cases such as syntax or malformed-source errors):
and it may well be that the alien macros are bad and wrong in that
respect.  (I also think that being utterly sure that the libraries are
loaded before you use DEFINE-ALIEN-ROUTINE might be your best bet in
the short term, because the alternative to error-signalling macros is
probably quite a lot of restructuring of alien code).  It may be that
DEFINE-ALIEN-ROUTINE could be persuaded to do something like
  (defmacro define-alien-routine (name ...)
    (if (find-foreign-symbol name)
        `(%define-alien-routine ...)
        `(%define-alien-routine-to-be-looked-up-later ...)))
but that may not be a net win.  I forget what macroexpand-time
computation define-alien-routine does (and I've just returned from a
conference, so my clarity of thought and of expression isn't at its
usual crystalline self...)

I hope this hasn't muddied the waters much more.

Cheers,

Christophe

Hi;

On Thursday 14 October 2004 05:43 pm, Christophe Rhodes wrote:
> To be fair to you, I'm not sure that there is one, but in defence of

Thank you for making me feel less inadequate (even if that was not
your intent!).

> respect.  (I also think that being utterly sure that the libraries are
> loaded before you use DEFINE-ALIEN-ROUTINE might be your best bet in
> the short term, because the alternative to error-signalling macros is
> probably quite a lot of restructuring of alien code).  It may be that

[ snip snip snip ]

> I hope this hasn't muddied the waters much more.

No, that helped, and I think I understand why define-alien would want
to actually wire stuff together (address-wise) at define-time
(requiring the foreign address).

Well, how about I take you up on your suggestion, which I take to be:

given that the whole point of the exercise is to load a shared
library, at binding-build-time I should require not only the output of
gccxml corresponding to the library, but the library itself.  Then, I
could check whether the symbol exists before emitting its binding.

It complicates the build process somewhat, but it would definitely be
more bullet-proof (and allow me to rip out my already-written code
which tries to infer "artificial" symbols from their mangled names).

Do I understand correctly?

Thanks for the help!

-jm

> Cheers,
>
> Christophe
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Sbcl-help mailing list
> Sbcl-help@...
> https://lists.sourceforge.net/lists/listinfo/sbcl-help

-- 
==== John Morrison
==== MAK Technologies Inc.
==== 10 Fawcett Street, Cambridge, MA 02138
==== http://www.mak.com/
==== vox:617-876-8085 x115
==== fax:617-876-9208
==== jm@...

On Thu, 14 Oct 2004, John Morrison wrote:

> given that the whole point of the exercise is to load a shared
> library, at binding-build-time I should require not only the output of
> gccxml corresponding to the library, but the library itself.  Then, I
> could check whether the symbol exists before emitting its binding.

Sounds sane to me.

Now that you mention it, I'd suggest that you use dlopen RTLD_LOCAL & 
dlsym to discover if the symbol exist (as opposed to hooking into SBCL 
internals): this way you can be sure that the symbol really is in the 
library and not in some other library currently loaded into SBCL, and also 
avoid future breakage due to internal interfaces changing.

Cheers,

  -- Nikodemus              Schemer: "Buddha is small, clean, and serious."
                   Lispnik: "Buddha is big, has hairy armpits, and laughs."