Re: [Reduce-algebra-developers] Stack overflow

[Arthur N. <ac...@ca...>]

On Sun, 27 Nov 2011, Raffaele Vitolo wrote:
> Dear All,
>
> I use rule lists to define a set of variables. In a particularly heavy
> computation with long rule lists I get the following message:
>
> +++ Error stack overflow:
> Cont? (Y or N)
>
> I found in internet very limited information about enlarging the stack
> memory size in reduce, so I'm asking your help to understand what I can do.
>
> Raffaele Vitolo.
>

>From the formt of the error message I am taking it that this is the 
CSL-based version, so my answer supposes that.

With CSL there are two stacks used. One is entirely under control of CSL 
itself and is a stack of Lisp data-structures. There is provision for 
extending that using the "-K" command-line option. I will use this moment 
as an opportunity to explain that I have recently been trying to put in 
some work on the CSL documentation, and when you fetch from subversion 
these days you will find that csl/cslbase/csl.pdf is the work in progress 
on that - but under its section on command line options it mentions this 
case.

I believe however that the issue you have run into is not this internal 
"lisp stack" so at this stage using "-K" will not help you. There is the 
regular stack that the C code that implements CSL uses. Here I need to 
check what operating system you are using since that matters!

On Windows the stack size gets fixed and bound in at the moment you like 
an application. If you look in csl/cslbase/csl.c around line 1430 you will 
find me reading it via use of GetModuleHandle and the best recipes I could 
find. And if you when you configure csl you go "--with-csl --enable-debug" 
then you see it prints a message on startup saying how big a stack it 
found. As best I understand at present to have a larger stack I would need 
to adjust the Makefile and pass extra options to the linker, and I feel 
that no fixed different size would really be a LOT better than where we 
are now.

On Linux/Unix/MacOSX I hope to be able to read the stack size as set by 
the "ulimit" command in bash and whatever applies if you are using a 
different shell. Go "ulimit -a" in bash to see where you are. If a user 
has set ulimit to "unlimited" then I default to allowing 4 Mbytes - 
because unlimited will really not be what it says and actually overflowing 
leads to corruption and abrupt disaster.

So on *some* systems
    ulimit -a       examine existing size
    ulimit -s 8192  make stack bigger
    ulimit -a       check it has grown
may help.

Now there is a separate issue as to whether it is good that Reduce 
recurses ridiculously deep in the particular cases you have in hand, so if 
you can provide a reasonably compact example that leads to stack overflow 
then please send it to me so I can at least wonder if there is an easy way 
to reduce Reduce's use of stack in that instance.

Note also that CSL and PSL may have different sets of optimisations in 
theor compilers as regards stack usage. I believe that PSL (by having FULL 
control of its compilation) can handle tail recursion better than CSL 
(that relies on an underlying C compiler) in some cases. But I also hope 
that the advantage there is not 100% one-sided. So try both the CSL and 
PSL version as a way to try to get the biggets possible calculation 
completed as well as sending us a test case to see if we can make a proper 
enhancement!

              Arthur