Re: [Sbcl-help] Garbage collection for server with user sessions

[Nikodemus S. <nik...@ra...>]

On 27 February 2010 02:49, Brett van de Sande <bv...@as...> wrote:

> However, I am having lots of trouble with garbage collection, since

Can you be more specific? Are you running out of memory, are the GC
pauses too long, or what?

> the behaviour of the server badly violates the basic assumption of
> the generational garbage collector:  in my case, it is more likely that
> old data (after a session has closed) can be garbage collected
> than newer data.

I don't see how this violates the generational hypothesis, actually.
Surely "session closing" is a rare event: most of the time the older
the object is, the less likely it is to be up for collection.

Anyways.

> 1.  Modify the garbage collector so that it gcs the older data first.
> (that is, do generation 0, 1, 2 in the usual way, but do generations
> 3, 4, 5 backwards).

This is a non-starter, pretty much.

> 2.  Is there any way to garbage collect for a specific object
> and its descendents?

Nope.

What you can and want to do depends on your problem.

If this is "just" a performance issue, then a bigger nursery and
making higher GENERATION-BYTES-CONSED-BETWEEN-GCS for the top-few
generations will probably help. See section 7.13 in the fine manual
(assuming you have SBCL 1.32.33 or newer.)

If you are running out of heap space, running a full collection after
a session has closed is likely to help -- but this has obvious
performance implications.

[ Actually, one GC toggle that we don't have but could is SCHEDULE-GC
where you could specify the depth of the next GC, so that when a
session ends you could do (SCHEDULE-GC :FULL) -- so that next time GC
runs, it will be a full GC. Need to think about this. ]

Other alternatives are:

* Use stack allocation for short-lived data, so it will not cause
promotion of older objects so fast. This may require restructuring
control flow, obviously.

* Structure your large data to minimize the number of pointers: if you
can pack things into eg. (SIMPLE-ARRAY (UNSIGNED-BYTE 32) 1), it will
mean less work for the GC.

These don't radically change the equation, just ameliorate it. The
basic problem remains as you stated it: you have big chunks of old
data that become garbage rarely. The way around that is to:

* Keep long-lived data outside the heap: either in foreign memory, or
store it in a DB. Then you can release/delete it when you want.

Cheers,

 -- Nikodemus