#564 FMSLogo disappears when running tree recursive procedures with no base case

[David Costanzo]

When FMSlogo runs a tree recursive procedure with no base case, it takes 100% CPU of one processor, grows quickly in memory, then disappears when it runs out of memory (which, for a 32-bit Windows program, is at most 2GiB of virtual memory, regardless of how much virtual memory the machine has).

A simple example is:

:::text
TO TREERECURSE
  TREERECURSE
  TREERECURSE
END

This is a regression that was introduced in FMSLogo 6.30.0. In FMSLogo 6.29.1, it prints something like:

:::text
Out of memory in TREERECURSE
[TREERECURSE]

I expect that this was introduced by the change which improved evaluation time by 40%.

While FMSLogo is full of bugs that make it crash when it runs out of memory for specific allocations, this particular one is both a regression and can be encountered in normal usage, by writing a buggy procedure. Indeed, that's how I found it. I wrote a procedure called GETENVIRONMENTVARIABLETEST that was supposed to run GETENVIRONMENTVARIABLE, but instead ran GETENVIRONMENTVARIABLETEST. Therefore, unlike the other memory bugs, this one should be fixed as a high priority (in the next release).

How Reproducible:
Every Time

Steps to Reproduce:

1. Run DEFINE "X [[] [X X]]
2. Run X
3. Wait for FMSLogo to run out of memory. You can watch in task manager. Since it's a 32-bit program, it'll run out of memory at 2GiB of virtual memory.

What Happens:
When FMSLogo reaches 2GiB of virtual memory, the FMSLogo window disappears.

Expected Result:
When FMSLogo reaches 2GiB of virtual memory, FMSLogo prints an error to the commander history:

:::text
Out of memory in X
[X X]

[David Costanzo]

I have committed a fix in [r5668]. It will be available in whatever version follows FMSLogo 8.0.2, which I expect will be FMSLogo 8.1.0.

This was, indeed, caused by the 40% improvement in evaluation times. The 40% improvement was accomplished by allocating "stack frames" from the heap, instead of constructing each frame as a list of NODE*. At the time, I must not have realized that throwing an OUT_OF_MEM_UNREC when malloc() failed would immediately exit.

My fix mimics the strategy for when creating a new NODE* fails. It allocates a "reserve tank" of 50 stack frames up-front, to be used only if it runs out of memory, and only for the purpose of halting the current evaluation gracefully. This is no better or not worse than it was before.

After running out of memory, I found that I could re-run the tree-recursive procedures and get the same "out of memory" error. However, FMSLogo is certainly not is a good state. Attempts to use "File -> Load..." after an "out of memory" error cause FMSLogo to disappear. However this is also reproducible in MSWLogo 6.5b, so is not part of the regression that I caused.