Vassilis Radis wrote on Fri, Nov 30, 2012 at 07:11:29PM +0200:
>You are looking at physical memory versus objects you hold in software
> 1. Why top - reported memory keeps growing, while (room) says it does not?
> The only thing I can think of is some leakage through ffi. I am not
> directly calling out with ffi but maybe some drakma dep does and forgets to
> free its C garbage. Anyway I dont know if this could even be an
> explanation. Could it be something else? Any insights as to where I should
> start searching? 2. Why isnt --dynamic-space-size honoured?
view. They are always different, and C has worse problems with this
(because it can never move objects).
RSS size in this case is essentially "number of pages currently dirty
(touched)" and not swapped out. As the allocator allocated and the GC
moves things around they don't always give back pages that are no
longer occupied to the OS immediately. Doing so is prohibitively
expensive because it requires full system calls. To simplify: after
you move a generation worth of objects from A to B you can't outright
free all of A without huge performance penalty. And some objects
might be nailed down due to the slightly conservative GC.
For my toy at work I improved this a lot by reducing the number of
generations to 4 total (newest, not GCed + 2 others) and setting
sb-ext:generation-bytes-consed-between-gcs to a low value (2 MB). But
our toy has different allocation patterns than regular software so
YMMV. Also made it faster.
You can also set
(sb-alien::define-alien-variable "small_generation_limit" sb-alien:int)
from 1 to 0, which will trigger the "giving back" much more often, but
there is a huge performance penalty, and system-wide affecting other
As I said you will find that C has even worse problems with this
because it can never move objects and hence never compact parts of the
heap. One reason why allocation in C is so much slower than in a
Common Lisp like SBCL is that strategies for avoiding this problem
have to be executed at allocation time, and you might spend a lot of
CPU time and memory bandwidth on doing this for an object that gets
destroyed a microsecond later anyway.
Did I ever report on this RSS lowering things? I forgot. I wanted
to. I think it's likely that a lower number of generations benefits a
lot of applications, even non crazy stuff.
> Thank you,
> Keep yourself connected to Go Parallel:
> TUNE You got it built. Now make it sing. Tune shows you how.
> Sbcl-help mailing list
Martin Cracauer <email@example.com> http://www.cons.org/cracauer/