On Wed, May 28, 2014 at 7:09 PM, Stephan Frank <defclass@googlemail.com> wrote:
On 29.05.2014 00:12, Alastair Bridgewater wrote:
First thoughts:  "Ugh."

well, OK; to me that approach overall looked a lot cleaner than the current variant (though I admit that whole call into lisp for allocation isn't so great)

Exactly, call into lisp for allocation?  Ugh.
 

  - On both, the runtime code can be made to do a certain amount of
allocation straight-up, without having to do the whole call-into-c thing.

Do you mean call-into-lisp? How would this prevent the malloc calls from within GMP?

Sorry, yes. You'd need a bit in C that knows how to safely allocate whatever object you need and arrange for it to be pinned, but it saves the whole call-into-lisp and return dance.
 


The lead-up to this level of cleverness is cleaning up and exposing more
of the runtime interfaces for contribs and outside systems to use, which
may not be entirely a good thing, or to move some of the logic required
for sb-gmp over to the runtime (which has a certain amount of precedent
already).

I am afraid you lost me somewhere in there. Also, isn't it somewhat "ugh" as well to have an optional contrib trigger such changes to the runtime?

It is, but as I said, there's precedent.

And what I'm sortof suggesting here is building an interface so that alien code doing lisp heap allocation is semi-supported.
 

And, of course, others may disagree entirely with this whole idea.

-- Alastair Bridgewater


On Wed, May 28, 2014 at 4:52 PM, Stephan Frank <defclass@googlemail.com
<mailto:defclass@googlemail.com>> wrote:

    Hello,

    some users may have noticed, that sb-gmp can be quite fragile at times.
    This is in most part the result of internal (re)allocation from within
    GMP if the Lisp-side pre-allocated buffers are not big enough. Sometimes
    there is no way to prea-llocate in advance which was solved by copying
    from C to SBCL space with a factor 3 penalty.

    A have toyed with using the custom allocation method of GMP to pull all
    allocation to the Lisp side (apart from temporary/dynamic-extent
    with-alien C structure setup before calling into C).

    This new allocation scheme lives in the 'newalloc' branch of the sb-gmp
    development repository [1].

    The new method works now well for me (on amd64) and I would appreciate
    feedback from others, especially from people who have problems with the
    current version in SBCL, whether this new approach yields better, stable
    results when exercising the test suite (use the one from the repository
    branch).

    Note, that this version does not yet work in conjunction with sb-mpfr
    (or only by accident).


    An overview of the pros and cons of the old and new method:

    old - pros:
    - mostly working
    - page pinnig when calling into C

    old - cons:
    - two methods of memory handling
        * memory fault when pre-allocation assumption is violated
        * factor 3 copying penalty when handling allocation within GMP
        * several different implementations for each case with different
    corner cases
    - finalizers required for GMP random states


    new - pros:
    - unified and overall simpler memory handling, more data
    structures/logic moved to the Lisp side instead of alien structures
    - appears to be more robust once I got the pointer-fiddling right
    - all allocation requests are handled within SBCL
    - robust against allocations and re-allocations from within GMP (no more
    memory faults due to allocation)
    - avoids copying from C memory space in all cases (no more factor 3
    copying penalty)
    - no finalizers required

    new - cons:
    - requires without-gcing instead of page-pinning only
    - allocation calls from C into Lisp
    - GMP random state handling got a bit more complicated (but is well
    excercised within the test suite and appears to be robust).


    Regs,
    Stephan


    [1] https://github.com/sfrank/sb-gmp/tree/newalloc

    ------------------------------------------------------------------------------
    Time is money. Stop wasting it! Get your web API in 5 minutes.
    www.restlet.com/download <http://www.restlet.com/download>

    http://p.sf.net/sfu/restlet
    _______________________________________________
    Sbcl-devel mailing list
    Sbcl-devel@lists.sourceforge.net
    <mailto:Sbcl-devel@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/sbcl-devel