#2107 Terminate waits forever if the creating thread called DetachThread

[Josep Maria Blasco]

Description:

If the thread that created an interpreter instance calls DetachThread() on
the RexxThreadContext that RexxCreateInterpreter() returned, a later
Terminate() on that instance never returns:

RexxInstance *in; RexxThreadContext *tc;
RexxCreateInterpreter(&in, &tc, NULL);
tc->DetachThread();      // the creating thread leaves the instance
in->Terminate();         // waits forever

The API documentation does not forbid this: DetachThread() detaches the
thread represented by the RexxThreadContext from its interpreter instance,
and Terminate() "will wait for all threads to complete processing before
returning". An embedder that creates an instance on one thread and
uses it from others (attaching and detaching per call) naturally detaches the
creating thread too.

Cause (interpreter/runtime/InterpreterInstance.cpp, trunk r13267):
InterpreterInstance::detachThread(Activity *) removes the root activity from
allActivities and gives it back with ActivityManager::returnActivity()
like any attached activity. terminate() then decides that only the root
activity is left with

terminated = allActivities.items() == 1;

With the root activity gone the count is 0, so terminated is false and
terminate() waits on terminationSem. Nothing will post it: the posts in
detachThread() and poolActivity() only happen when another activity
leaves.

Only changing that line to <= 1 would not be enough: detachThread() and
poolActivity() would then post when one non-root activity is still running,
and the end of terminate() calls ActivityManager::returnRootActivity() on
an activity that returnActivity() already cleaned up
(cleanupActivityResources() twice: its semaphores destroyed twice).

Fix (attached diff, against r13267): the instance records that its root
activity was detached (rootDetached, set in detachThread()); the three
"only the root activity is left" tests use a helper, onlyRootLeft(), that
expects 0 activities instead of 1 in that case; and terminate() skips
returnRootActivity() when detachThread() already returned it.

Tested (Linux, r13267 Release): the reproducer hangs without the fix and
passes with it; with the fix, 50 create/Terminate cycles each pass for: no
detach; plain detach; detach while a Rexx thread started with reply is still
running (Terminate waits for it); detach, attach again, run, detach; detach
and Terminate from another thread; another thread attaches, runs and detaches
before the creating thread detaches. Without the fix every case but the first
hangs. ooRexx test suite (test/trunk r13267, testOORexx.rex -X native_api):
23,383 tests, the same 9 failures and 0 errors with and without the fix
(file-system tests: CHAROUT, LINEOUT, File, Stream, SysFileXXX, which also
fail on the unpatched build in this environment).

Found while developing a native ooRexx/Python bridge (an extension package in
the style of BSF4ooRexx). Workaround for released versions: never detach
the thread that created the instance; leave it attached until Terminate().