Steel Bank Common Lisp

On Sat, 10 Apr 2010 05:03:47 +0200, Waldek Hebisch wrote:

> How can one pass Lisp data without any convertion to foreign routines? 
> I would like to use GMP routines to perform some operations on sbcl
> bignums (on large numbers GMP is much faster than native sbcl routines).
> 
> I tried:
> 
> (defun ttt (x)
>   (sb-sys:with-pinned-objects (x)
>     (SB-ALIEN::alien-funcall
>        (SB-ALIEN::extern-alien "ppp"
>            (SB-ALIEN::function SB-ALIEN::void t))
>        x)))
> 
> but I get compilation error "unknown alien type: T".  When I replaced
> 't' by '(* t)' the function compiles fine but trying:
> 
> (ttt (expt 10 30))
> 
> I get:
> 
> debugger invoked on a TYPE-ERROR:
>   The value 1000000000000000000000000000000 is not of type
>     (OR NULL SYSTEM-AREA-POINTER (ALIEN (* T))).

AFAIK GMP uses a C structure to represent things.  You need to
allocate and initialize this structure, then pass the pointer to it.

BTW, can you provide a link to the benchmarks that show how SBCL is
much slower compared to GMP?  I am curious.

Tamas

Tamas K Papp wrote:
> On Sat, 10 Apr 2010 05:03:47 +0200, Waldek Hebisch wrote:
> 
> > How can one pass Lisp data without any convertion to foreign routines? 
> > I would like to use GMP routines to perform some operations on sbcl
> > bignums (on large numbers GMP is much faster than native sbcl routines).
> > 
> > I tried:
> > 
> > (defun ttt (x)
> >   (sb-sys:with-pinned-objects (x)
> >     (SB-ALIEN::alien-funcall
> >        (SB-ALIEN::extern-alien "ppp"
> >            (SB-ALIEN::function SB-ALIEN::void t))
> >        x)))
> > 
> > but I get compilation error "unknown alien type: T".  When I replaced
> > 't' by '(* t)' the function compiles fine but trying:
> > 
> > (ttt (expt 10 30))
> > 
> > I get:
> > 
> > debugger invoked on a TYPE-ERROR:
> >   The value 1000000000000000000000000000000 is not of type
> >     (OR NULL SYSTEM-AREA-POINTER (ALIEN (* T))).
> 
> AFAIK GMP uses a C structure to represent things.  You need to
> allocate and initialize this structure, then pass the pointer to it.

I do not want to use C structure, I want to operate on Lisp
integers.  Once I get bignum on C side the rest is easy.
I did similar thing for Closure CL (which shares much of its
bignum support with sbcl).  In case Closure CL it was necessary
to add an assembler copy routine.  As byproduct use of assembler
avoided typechecking problem.  But AFAICS for sbcl it should be
possible to avoid copy (at least some copies).  And using
assembler with sbcl looks at least as problematic as passing
Lisp data to C.

> BTW, can you provide a link to the benchmarks that show how SBCL is
> much slower compared to GMP?  I am curious.

Simple bechmark like:

(setf a (expt 10 1000))
(setf b (expt 10 1000))

(defun ts-mul(n) (dotimes (i n) (setf c (* a b))) nil)

(time (ts-mul 100000))

On my machine sbcl needs 0.989 s, ECl (wich uses GMP) 0.582 s.
For bigger numbers the ratio is more in GMP favour:
(setf a (expt 10 10000))
(setf b (expt 10 10000))
(time (ts-mul 10000))

Gives me 9.31 s using sbcl and 1.394 s using ECL.  Similar
results for division and gcd (but GMP seem to win for much
smaller numbers).

-- 
                              Waldek Hebisch
hebisch@...

On Sat, 10 Apr 2010 16:57:42 +0200, Waldek Hebisch wrote:

> Tamas K Papp wrote:
>> On Sat, 10 Apr 2010 05:03:47 +0200, Waldek Hebisch wrote:
>> 
>> > How can one pass Lisp data without any convertion to foreign
>> > routines? I would like to use GMP routines to perform some operations
>> > on sbcl bignums (on large numbers GMP is much faster than native sbcl
>> > routines).
>> > 
>> > I tried:
>> > 
>> > (defun ttt (x)
>> >   (sb-sys:with-pinned-objects (x)
>> >     (SB-ALIEN::alien-funcall
>> >        (SB-ALIEN::extern-alien "ppp"
>> >            (SB-ALIEN::function SB-ALIEN::void t))
>> >        x)))
>> > 
>> > but I get compilation error "unknown alien type: T".  When I replaced
>> > 't' by '(* t)' the function compiles fine but trying:
>> > 
>> > (ttt (expt 10 30))
>> > 
>> > I get:
>> > 
>> > debugger invoked on a TYPE-ERROR:
>> >   The value 1000000000000000000000000000000 is not of type
>> >     (OR NULL SYSTEM-AREA-POINTER (ALIEN (* T))).
>> 
>> AFAIK GMP uses a C structure to represent things.  You need to allocate
>> and initialize this structure, then pass the pointer to it.
> 
> I do not want to use C structure, I want to operate on Lisp integers. 

So you want to use a foreign library, but you don't want to deal with
its API?  Curious.

> byproduct use of assembler avoided typechecking problem.  But AFAICS for
> sbcl it should be possible to avoid copy (at least some copies).  And
> using assembler with sbcl looks at least as problematic as passing Lisp
> data to C.

There is no need to use assembler, you just initialize the structure(s)
using CFFI or SBCL's FFI.  Foreign function interfaces in Common Lisp
implementations are no more "problematic" than in any other language.
You can even use tools like SWIG to generate the CFFI definitions.

Tamas

Tamas K Papp <tkpapp@...> writes:

> On Sat, 10 Apr 2010 16:57:42 +0200, Waldek Hebisch wrote:
>
>> I do not want to use C structure, I want to operate on Lisp integers. 
>
> So you want to use a foreign library, but you don't want to deal with
> its API?  Curious.

I'm not sure why you think this is curious.  It sounds like Waldek is
saying that the in-memory representation of an sbcl bignum is
sufficiently close to an in-memory representation of a gmp bignum that
it's possible to pun them.  The disadvantage of doing a principled
translation between GMP structures and Lisp bignums is...

>> byproduct use of assembler avoided typechecking problem.  But AFAICS for
>> sbcl it should be possible to avoid copy (at least some copies).  And
>> using assembler with sbcl looks at least as problematic as passing Lisp
>> data to C.
>
> There is no need to use assembler, you just initialize the structure(s)
> using CFFI or SBCL's FFI.  Foreign function interfaces in Common Lisp
> implementations are no more "problematic" than in any other language.
> You can even use tools like SWIG to generate the CFFI definitions.

... that to convert between the worlds, you'd need to copy, and copying
takes time.  I don't know if it takes too much time, that's application
dependent, but if there's a way to remove the overhead, then why not?
Frankly, dismissing this out of hand strikes me as curious.

Cheers,

Christophe

Waldek Hebisch <hebisch@...> writes:

> I tried:
>
> (defun ttt (x)
>   (sb-sys:with-pinned-objects (x)                         
>     (SB-ALIEN::alien-funcall
>        (SB-ALIEN::extern-alien "ppp"
>            (SB-ALIEN::function SB-ALIEN::void t))    
>        x)))

Try

(defun ttt (x)
  (sb-sys:with-pinned-objects (x)
    (let* ((addr (sb-kernel:get-lisp-obj-address x))
           (sap (sb-sys:int-sap addr)))
      (sb-alien:alien-funcall
        (sb-alien:extern-alien "ppp" (sb-alien:function sb-alien:void (* t)))
        sap))))

The pointer that the C side will receive will still have its lisp tag
on.

Cheers,

Christophe

On Sat, 10 Apr 2010 17:48:31 +0100, Christophe Rhodes wrote:

> Tamas K Papp <tkpapp@...> writes:
> 
>> On Sat, 10 Apr 2010 16:57:42 +0200, Waldek Hebisch wrote:
>>
>>> I do not want to use C structure, I want to operate on Lisp integers.
>>
>> So you want to use a foreign library, but you don't want to deal with
>> its API?  Curious.
> 
> I'm not sure why you think this is curious.  It sounds like Waldek is
> saying that the in-memory representation of an sbcl bignum is
> sufficiently close to an in-memory representation of a gmp bignum that
> it's possible to pun them.  The disadvantage of doing a principled
> translation between GMP structures and Lisp bignums is...

I think you have misunderstood me (and it may have been my fault).  My
impression was that Waldek wanted automagical conversion between SBCL
bignums and GMP.  Conversion may be possible (I have seen you code snippet),
but not automatically.  Equivalence has to be established.

>>> byproduct use of assembler avoided typechecking problem.  But AFAICS
>>> for sbcl it should be possible to avoid copy (at least some copies). 
>>> And using assembler with sbcl looks at least as problematic as passing
>>> Lisp data to C.
>>
>> There is no need to use assembler, you just initialize the structure(s)
>> using CFFI or SBCL's FFI.  Foreign function interfaces in Common Lisp
>> implementations are no more "problematic" than in any other language.
>> You can even use tools like SWIG to generate the CFFI definitions.
> 
> ... that to convert between the worlds, you'd need to copy, and copying
> takes time.  I don't know if it takes too much time, that's application
> dependent, but if there's a way to remove the overhead, then why not?
> Frankly, dismissing this out of hand strikes me as curious.

I don't recall dismissing this.  I just claimed that dealing with the
API at some level is necessary.  If the representations are (near)
equivalent, so much the better.

Tamas

In article <871venuucg.fsf@...>,
 Christophe Rhodes <csr21@...> wrote:
> Tamas K Papp <tkpapp@...> writes:
> 
> > On Sat, 10 Apr 2010 16:57:42 +0200, Waldek Hebisch wrote:
> >
> >> I do not want to use C structure, I want to operate on Lisp integers. 
> >
> > So you want to use a foreign library, but you don't want to deal with
> > its API?  Curious.
> 
> I'm not sure why you think this is curious.  It sounds like Waldek is
> saying that the in-memory representation of an sbcl bignum is
> sufficiently close to an in-memory representation of a gmp bignum that
> it's possible to pun them.  The disadvantage of doing a principled
> translation between GMP structures and Lisp bignums is...

It's not, unless you restrict yourself to positive integers. GMP integer 
use a sign-magnitude representation, while we use 2's complement, 
probably to simplify bit operations.

Paul Khuong

Paul Khuong <pvk@...> writes:

> In article <871venuucg.fsf@...>,
>  Christophe Rhodes <csr21@...> wrote:
>> I'm not sure why you think this is curious.  It sounds like Waldek is
>> saying that the in-memory representation of an sbcl bignum is
>> sufficiently close to an in-memory representation of a gmp bignum that
>> it's possible to pun them.  The disadvantage of doing a principled
>> translation between GMP structures and Lisp bignums is...
>
> It's not, unless you restrict yourself to positive integers. GMP integer 
> use a sign-magnitude representation, while we use 2's complement, 
> probably to simplify bit operations.

Well, that's a shame :-)  Oh well.

Then sb-alien can be used to write support routines for automatic
conversion to and from the GMP representation: defining alien type class
methods for naturalize-gen and deport-gen (look at the c-string methods
for inspiration).  This would allow Waldek to have an integrated feel
for whatever GMP calls he needs to make, I think.

Best,

Christophe

Christophe Rhodes wrote:
> 
> Try
> 
> (defun ttt (x)
>   (sb-sys:with-pinned-objects (x)
>     (let* ((addr (sb-kernel:get-lisp-obj-address x))
>            (sap (sb-sys:int-sap addr)))
>       (sb-alien:alien-funcall
>         (sb-alien:extern-alien "ppp" (sb-alien:function sb-alien:void (* t)))
>         sap))))
> 
> The pointer that the C side will receive will still have its lisp tag
> on.
> 

Thanks, that is what I needed.  Now I can multiply sbcl bignums via
GMP (division and GCD will require more effort).

-- 
                              Waldek Hebisch
hebisch@...

Christophe Rhodes wrote:
> Paul Khuong <pvk@...> writes:
> 
> > In article <871venuucg.fsf@...>,
> >  Christophe Rhodes <csr21@...> wrote:
> >> I'm not sure why you think this is curious.  It sounds like Waldek is
> >> saying that the in-memory representation of an sbcl bignum is
> >> sufficiently close to an in-memory representation of a gmp bignum that
> >> it's possible to pun them.  The disadvantage of doing a principled
> >> translation between GMP structures and Lisp bignums is...
> >
> > It's not, unless you restrict yourself to positive integers. GMP integer 
> > use a sign-magnitude representation, while we use 2's complement, 
> > probably to simplify bit operations.
> 
> Well, that's a shame :-)  Oh well.
> 
> Then sb-alien can be used to write support routines for automatic
> conversion to and from the GMP representation: defining alien type class
> methods for naturalize-gen and deport-gen (look at the c-string methods
> for inspiration).  This would allow Waldek to have an integrated feel
> for whatever GMP calls he needs to make, I think.
>

I am using sb-bignum::negate-bignum and
sb-bignum::negate-bignum-in-place (if appropriate).  I am not sure
how much automation is possible (and worthwhile).  I am interfacing
to mpn routines, there is only handful of them but interfaces are
irregular.

-- 
                              Waldek Hebisch
hebisch@...