On Fri, Jun 15, 2012 at 1:13 AM, Nikodemus Siivola <nikodemus@random-state.net> wrote:
On 13 June 2012 20:05, Vsevolod Dyomkin <vseloved@gmail.com> wrote:

> I'm writing a sophisticated DSL for working with linguistic parse trees,
> which is a set of top-level macros, each of which is comprised of sereval
> tens (up to 100 and more) other macros, that  heavily rely on special
> variables manipulation. Most of the macros are compiled OK, but in rare
> cases they cause the compiler to run out of memory (and I use
> dynamic-space-size of 3,5G).

> The problem seems to occur inside SB-C::LIFETIME-POST-PASS, like in this
> stacktrace:

You realize you're not showing the actual error?

Yes, because, strictly speaking, there's no error: as I said, if given enough resources the program terminates. It just requires too many resources, and I localized this demand in compiler lifetime analysis. So I was wondering how I can control that (maybe turn off some optimizations...)
 

> The processing, actually, seems to terminate eventually (if given enough
> time and memory), but it's taking pretty long (order of minutes) and for
> some examples falls into LDB with GC not having enough memory.
>
> I've looked into the code of lifetime analysis in the compiler, but haven't
> found any explicit way to control its behavior.
>
> I've also tried to experiment with different compiler policies: like setting
> debug to 0, compilation-speed to 0 or 3, speed to 0, space to 0, etc, but it
> didn't really make any difference.

(SPACE 3) is more likely to help than (SPACE 0) -- but unlikely to
make much of a difference.

The way to fix an issue like this is to reduce the size of functions generated.

SBCL doesn't do much in the way of interprocedural optimizations
unless you inline things -- and from the way you describe the system
it sounds like you're inlining almost everything.

If you generate a bunch of out-of-line functions that call each other,
then each function can be compiled separately, and you should not get
a blowup like this.

I guess, that my problem is related to the way I use special variables (there are a couple of them, that a threaded through the whole code and manipulated everywhere).

Anyway, thanks for the pointers, I'll have to think of that.

Best,
Vsevolod