#2104 Condition traps of a native activation are turned off by a nested API call through its context; an untrapped SYNTAX error then crashes the interpreter

[Josep Maria Blasco]

The rexxapi book, Rexx Exit Context Interface, says: "API calls made using
the RexxExitContext APIs may cause Rexx syntax errors or other conditions to
be raised. These calls are invoked as if the current context is operating
with SIGNAL ON ANY enabled. Any conditions will be trapped and held in a
pending condition until the current context returns."

This does not hold when the same context is used again, by native code
reached from inside one of those calls, while the call is still running:

1. A command handler (exit) calls SendMessage0(probe, "RUN"). Its
   ApiContext turns on the condition traps of the handler's
   NativeActivation.
2. RUN, a Rexx method, calls a native method that uses the handler's exit
   context, saved by the handler and still valid (the handler has not
   returned): saved->SetContextVariable("X", ...).
3. That call's ApiContext is built on the same NativeActivation
   (contextToActivation(c)), and its destructor calls
   disableConditionTraps() unconditionally. The handler's activation,
   still inside SendMessage0, no longer traps.
4. RUN then raises a SYNTAX error that nobody traps. SendMessage0 does
   not catch it; the exception unwinds through the handler's C++ frames and
   the interpreter crashes (SIGSEGV in Activity::cleanupStackFrame, from
   CommandHandler::call).

With step 2 left out, the SYNTAX error is trapped by SendMessage0 and
reported as expected. A trapped SYNTAX error (SIGNAL ON SYNTAX in the Rexx
caller) is not affected.

The attached apicontext-traps.cpp shows both (public API only):

g++ -o apicontext-traps apicontext-traps.cpp -I<api> -I<api>/platform/unix -lrexx -lrexxapi
./apicontext-traps nopoke   ->  trapped as expected: SYNTAX 42.3
./apicontext-traps          ->  Segmentation fault

Why the nested use, and a question. It comes from an extension we would
like to propose: command environments written in Rexx. Its native handler
sends COMMAND to a Rexx object, which reads and sets the issuing program's
variables, and reads and writes its WITH redirection, through methods that
use the handler's exit and I/O redirector contexts while the command runs.
A method context has no equivalent of the exit context's caller variables
or of the redirector, so there is no other way through the public API.

The book says that call-outs must use the context passed to them, so before
we go further: is this use (a still-active exit or I/O redirector context
used from a native method that the exit's own API call reached) meant to be
supported? If it is, the fix below makes it safe. If it is not, we would
redesign the extension so that the native handler does all the variable and
I/O work itself (at the price of deferred output and variable updates), and
a sentence in the book saying so would help others. Either way the crash
contradicts the guarantee quoted above.

Proposed fix. ApiContext restores the trap state it found instead of
turning traps off (attached apicontext-traps.diff): the constructors
remember context->conditionTrapsEnabled() before enabling them, and the
destructor disables them only if they were off. A nested call then leaves
the outer call's traps as they were.

Effect beyond nesting. While a native method or routine runs, its
activation already traps conditions (NativeActivation::run and
callNativeRoutine turn traps on for the call); today the first API call it makes turns them off for the
rest of the call, and with the fix they stay on, as run intended. We ran
the whole test suite (test/trunk r13260, 402 groups) against main/trunk
r13260 with and without the fix, one after the other: identical results
(24319 tests, the same 6 groups failing with the same 10 failures, all
about file permissions of our environment, which runs as root).

The I/O redirector functions (ReadInput, WriteOutput, ...) have the
same pattern, and in addition clear the activation's pending condition on
return (clearConditions), which looks just as unsafe when nested.