#27 6.4.2 and 7.0.0 CLIPS never returned after standard input was exhausted

[Batch Job 1]

CLIPS fix: CLIPS never returned after standard input was exhausted

CLIPS never returned after standard input was exhausted

What happened. Once standard input reached end of file, CLIPS took a whole core and grew its memory without bound, for ever.

clips -f2 a-file-that-does-not-call-exit.clp < /dev/null

The file's own commands ran. Then CLIPS printed its banner and a prompt, and never came back.

The same thing happened to any program whose commands were piped in and did not finish with (exit), which is the ordinary shape of a shell pipeline.

Why it is a fix and not a preference. Nothing in the manual says a CLIPS session must end with an explicit (exit), and the counterpart loop in the same file ends when its input does. CommandLoopBatchDriver returns at the end of a batch source, and CommandLoop is the interactive version of the same loop reading standard input. The difference between them was not a decision about interactive sessions; it was that one of the two handled the end of its input and the other turned it into a newline.

The unbounded memory growth is the part no reading excuses. The command string grew by one character on each pass of a loop that could not end.

Where it came from. DefaultGetNextEvent in clips_core_source_642/core/commline.c:

   inchar = ReadRouter(theEnv,STDIN);

   if (inchar == EOF) inchar = '\n';

   ExpandCommandString(theEnv,(char) inchar);

A newline is a reasonable thing to append once: it is what finishes a command the user has typed. It is not reasonable to append it again every time, and once standard input is exhausted ReadRouter answers EOF on every call, so every pass of CommandLoop appended another one. CommandLoop's own while (true) had no other way out.

The change. End of input is recorded rather than turned into a newline, and the loop reads that and leaves. Three small pieces:

• commline.h gains EndOfInputReached beside the existing HaltCommandLoopBatch, and SetEndOfInputReached/GetEndOfInputReached beside its accessors. The struct is allocated with memset to zero, so the flag starts false the same way the neighbouring one does.
• DefaultGetNextEvent sets it on EOF and appends nothing.
• CommandLoop tests it after the input step, discards any partial command, writes a newline so a shell prompt lands on its own line, and returns — which is what CommandLoopBatchDriver does when its input runs out.

main.c then returns EXIT_SUCCESS where it returned -1. That line was unreachable before, because CommandLoop never returned and (exit) leaves through ExitRouter; it is now the status of a session that ended because its input did, and 255 would be the wrong answer for that.

The setter, the getter, the caller and the loop all sit inside the one #if ! RUN_TIME block that already holds CommandLoop and DefaultGetNextEvent, so a run-time build is unaffected.

What was checked. run-fix-tests.sh builds the core as it stands and runs seven shapes of input under a time limit, and builds it again with the newline substitution put back and runs the first of them. The build as it stands exits; the reverted build is still running when the limit expires, having produced nothing. That is the discriminating evidence, and the runner produces it.

The seven shapes: a file that does not call (exit) with standard input at end of file; commands piped in and then end of input; commands piped in ending with (exit); (exit 3), whose status has to survive as 3; an unfinished command followed by end of input, which has to be discarded rather than executed; a file that does call (exit), which must not print a banner or a prompt; and a file with no (exit) followed by further commands on standard input, which have to run before the session ends.

The whole of ../../v1/fact-functions/tests gives identical output, and every other suite under ../ and ../../clips-optimizations/ passes. CLIPS 7.0.0's own test suite drives CLIPS by piping (batch "<name>.tst") into it, which is the path this changes, and agrees with this tree on every comparison but the ones ../reports/README.md lists.

Still present upstream. CLIPS 7.0.0 has the same DefaultGetNextEvent and the same while (true) in CommandLoop.