[Oorexx-devel] Risk of GC crash in trunk

[Jean-Louis F. <jfa...@gm...>]

I fixed a GC crash in my sandbox (was lucky to have a reproducible
crash...). I leave you to decide what to do in trunk, if risk confirmed.
The crash occured only with my sandbox, not with trunk (same oorexx script
used in both).

RexxActivity::generateProgramInformation calls frame->createStackFrame.
In RexxActivation::createStackFrame, the last line is
    return new StackFrameClass(type, getMessageName(), (BaseExecutable
*)getExecutableObject(), target, arguments, getTraceBack(),
getContextLineNumber());
The operator 'new' of StackFrameClass is overloaded and returns a new
object before the constructor is called.
This object is not protected.
The call to getTraceBack triggered a GC and the new object was put on the
list of dead objects.
The object is returned to generateProgramInformation which puts it in the
stackFrames list.
Later, crash because the behaviour pointer of the stack frame is NULL.

Fixed by passing a ProtectedObject to the 'new' operator of StackFrameClass.

StackFrameClass.hpp
    void *operator new(size_t, ProtectedObject &);

StackFrameClass.cpp
void *StackFrameClass::operator new(size_t size, ProtectedObject &p)
{
    /* Get new object                    */
    p = new_object(size, T_StackFrame);
    return p;
}

RexxActivation.cpp
    ProtectedObject p;
    return new (p) StackFrameClass(type, getMessageName(), (BaseExecutable
*)getExecutableObject(), target, arguments, getTraceBack(),
getContextLineNumber());


Made the same change in RexxSource::createStackFrame and
RexxNativeActivation::createStackFrame, just in case. Got no crash there,
but GC can happen the same way.

Jean-Louis