Re: [Sbcl-devel] STRING-CONCAT benchmark exhausts heap on x86

[<wil...@ai...>]

On Thu, Apr 26, 2007 at 02:51:36PM +0200, Andreas Fuchs wrote:
> Nikodemus Siivola wrote:
> > Andreas Fuchs wrote:
> > 
> >> x86 builds on my AMD64 machine have started exhausting the heap in the
> >> STRING-CONCAT benchmark, between 1.0.4.100 and 1.0.4.101.
> > 
> > This coincides with the new STRING-OUTPUT-STREAM implementation that
> > uses chained buffers. Without looking at code right now I can hypothesize
> > on some causes:
> > 
> >   * ALLOCATE-VECTOR allocating more space then the strings really need:
> >     since we allocate more (but smaller strings) any error is going to
> >     be compounded.
> > 
> >   * More live strings in memory, with each string "wasting" just enough
> >     memory to take it over the top.
> > 
> >   * Conservativeness holding on to discarded strings from previous run:
> >     more strings means more garbage we could mistake for live data.
> > 
> >   * Some horrible bug in the STRING-OUTPUT-STREAM implementation.
> 
> As another data point, running the benchmark on older versions also
> exhausts the heap, if I don't run a full GC first. Could just be the
> second item, then /-:

OK, I am fairly comfortable with a guess that the generational
conservative GC simply requires a sizable amount of slop which varies
depending on input patterns (bit patterns looking like addresses,
number of cells allocated exceeding a GC/tenure/whatever cutoff at a
particular point...), and fundamentally-harmless changes pushed it
over the edge in this case.

I will probably release 1.0.5 sometime tomorrow if that still looks
like a reasonable guess by then.

-- 
William Harold Newman <wil...@ai...>
PGP key fingerprint 85 CE 1C BA 79 8D 51 8C  B9 25 FB EE E0 C3 E5 7C
Until then, if you find yourself gnawing on the corner of your desk
due to a temporary inability to reach LtU (what, you mean that's just
me?), look in the directory where you've saved the papers you saw
here, and read one of the ones that you never got around to reading.
By the time you're done, LtU will be accessible again. :)
  -- Anton van Straaten on Lambda the Ultimate at 11/01/2005