On Wed, 2013-05-15 at 13:15 -0400, Alastair Bridgewater wrote:
> On May 14, 2013 6:33 PM, "Craig Lanning" <clanning@...> wrote:
> > On Tue, 2013-05-14 at 18:09 +0300, Nikodemus Siivola wrote:
> > > Then the correct answer is ALLOCATION/WITH-FIXED-ALLOCATION;
> > > everything else is more or less negotiable. Ie. the compiler needs
> > > know how to emit allocation sequences.
> > Now, that was an eye opener.
> > Ultimately, what I'd like to do is define a set of functions and
> > variables that can be define either in Lisp for a GC implemented in
> > or in C for a GC implemented in C (and brought into Lisp via the
> > facility).
> > The company I work for has a very special use case that I think can
> > made easier by rewriting the GC. I read the opinions about why the
> > is in C instead of Lisp. I understand them, agree with a few, and
> > disagree with others. I think that the changes I need to do to the
> > will be easier to implement if I can do it in Lisp instead of C.
> Are you able to share any details about the sort of changes that you
> have in mind? Or about the use case, even in general terms?
I can give a generic description of what I want to do. I just can't
tell you the specific reason why the application needs to do this.
Basically, the application has a certain object organization that it
needs to be able to keep localized. If we need to drop one of our main
objects so that we can load another one, it would be advantageous to
know that all allocation related to that object is contained within a
known memory block. Then we just declare the block empty and zero it.
I started my Lisp career working on Symbolics Lisp Machines. One thing
that they had was a concept of areas. An area was an allocation block
that could be constrained somehow. For the LispM's some areas held only
cons cells, others held symbol names. Areas could also be excluded from
collection. The aspect I'm interested in is that the user could create
an area, designate whether it should be handled by the GC, what objects
it would hold, and when the contents of the area are no longer needed,
just flush the contents without running any GC.
> > After looking at ALLOCATION/WITH-FIXED-ALLOCATION, it appears that
> > switching the GC from a C implementation to a Lisp implementation
> > be very non-trivial. Has anyone given any serious thought to what
> > need to be done to implement the GC in Lisp?
> Some years ago (late 2008, maybe), I did some preliminary
> investigation into writing parts of the GC in Lisp, although I was
> mostly focusing on how to implement things like the dispatch involved
> in scavenging an object.
> If you're planning to implement a GC in Lisp, one of the things to
> make sure of is that the code and data that implements the GC is
> available while you're running the GC, which is something that seemed
> very difficult in my original context, but for what you're doing
> simply arranging for any data tables required to be in static space or
> otherwise pinned and for code objects to not move would cover the
> worst of it. Well, and there are the FDEFINITION objects to consider,
> but putting them into static space as well should work, if you can
> arrange that (I can think of a couple of approaches here).
> Another aspect to consider is if the GC code itself should be allowed
> to allocate memory. If it shouldn't, then you have to be careful
> about how you write the code in order to avoid allocation and you may
> also want to figure out how to tell the compiler that any allocation
> would be an error (so that you don't backslide during maintenance).
> The inline allocation logic itself is actually fairly straightforward,
> modulo the overflow handling. Each thread has an alloc_region (in a
> single-threaded system there is a global alloc_region), which contains
> two fields of interest to the allocation logic: the current
> allocation pointer and the end of the region. Everything else in the
> alloc_region is of interest to the GC only, and the general layout
> might well be completely different for a different GC.
> You would also have to deal with the "safepoint" or "pseudo-atomic"
> logic, and I haven't really thought overmuch about what would be
> involved here, plus there's the whole issue of actually triggering a
> collection cycle. And there's the matter of scavenging any interrupt
> contexts and the various thread stacks.
> And if you're changing the heap layout too drastically, or need to
> arrange for certain things to be in certain heap spaces even in the
> cold-core, you may well end up modifying genesis
> (SYS:SRC;COMPILER;GENERIC;GENESIS.LISP), the program that actually
> creates the cold-core from a set of cross-compiled FASLs.
> Anyway, I've given the matter a certain amount of thought, and I'm
> reasonably confident that I could write SOMETHING that would work, but
> it would take a while and I simply don't have a use-case that would
> make it worth the effort.
> I hope that this brain-dump helps, and would love to be kept "in the
> loop" if you decide to go forward with writing a new GC for SBCL in
> Lisp. Or even a new GC for SBCL in C.
The "brain-dump" does help. You mentioned a few things that I hadn't
run across yet. I certainly will keep posting info to the list. I'm
interested to run any GC performance tests on whatever I end up
building. I intend to provide the SBCL team with a GC chapter for the
Internals manual at the very least. As I work on the GC changes, any
code that is generally useful, SBCL is welcome to have. I will try to
make sure that any of our "proprietary" changes are really nothing more
than specific configuration of the general GC that I build.
Based on the info from Nikodemus and Alastair, it looks like this will
be a longer term project than I had originally thought. Fortunately,
the application will still work with the existing GC so we're ok for the
time being. Changing the GC would just make the application run more
quickly and more efficiently.
>>>CONFIDENTIALITY NOTICE>>> This electronic mail message, including any and/or all attachments, is for the sole use of the intended recipient(s), and may contain confidential and/or privileged information, pertaining to business conducted under the direction and supervision of the sending organization. All electronic mail messages, which may have been established as expressed views and/or opinions (stated either within the electronic mail message or any of its attachments), are left to the sole responsibility of that of the sender, and are not necessarily attributed to the sending organization. Unauthorized interception, review, use, disclosure or distribution of any such information contained within this electronic mail message and/or its attachment(s), is(are) strictly prohibited. If you are not the intended recipient, please contact the sender by replying to this electronic mail message, along with the destruction of all copies of the original electronic mail message (along with any attachments).