#26 6.4.2 and 7.0.0 `read-number` read from the wrong position after the first call on a file

[Batch Job 1]

read-number read from the wrong position after the first call on a file

What happened. Reading numbers from a file with read-number gave back the first one and then FALSE for every number after it.

(open "numerals.txt" numerals "r")
(read-number numerals)   ; 12
(read-number numerals)   ; FALSE
(read-number numerals)   ; FALSE

The file held 12, -7, 3.5, 0 and 9999, one per line. Reading the same five from standard input gave all five.

It was worse than one good read per file. The position was never reset on the file path, so a second function that opened the same file and read from it got FALSE for its first read as well, having inherited where the previous one stopped.

Why it is a fix and not a preference. Section 12.4.11 documents read-number as taking an optional logical name and reading a number from it, and says it returns FALSE if errors are encountered while reading. Nothing distinguishes a file from standard input, and there is no error in a file of valid numerals.

ReadlineFunction, forty lines away in the same file, is the same shape of function with the same problem to solve, and it already makes the distinction the right way round.

Where it came from. ReadNumber in clips_core_source_642/core/iofun.c collects the characters of the number with:

      inputString = ExpandStringWithChar(theEnv,inchar,inputString,&RouterData(theEnv)->CommandBufferInputCount,
                                         &inputStringSize,inputStringSize + 80);

The fourth argument is the position in inputString to write at. CommandBufferInputCount belongs to the command prompt: the standard input router raises and lowers it as characters are typed, so that a backspace takes one back off. ReadNumberFunction set it to zero on each side of the read — but only on the stdin branch:

   if (strcmp(logicalName,STDIN) == 0)
     {
      RouterData(theEnv)->CommandBufferInputCount = 0;
      …
      ReadNumber(theEnv,logicalName,&theToken,true);
      …
      RouterData(theEnv)->CommandBufferInputCount = 0;
     }
   else
     { ReadNumber(theEnv,logicalName,&theToken,false); }

So reading a file wrote the first number's characters at whatever position the count happened to hold, which on a fresh environment is zero — hence the first read working — and then left the count where that number ended. The next read allocated a fresh string and began writing several characters into it, leaving the bytes before that unwritten, and strtoll was handed those.

The change. ReadNumber takes the position as an argument. Standard input passes the command buffer's count, as before, because the router's backspace handling is coupled to it. Anything else passes a position of its own, starting at zero.

   else
     {
      size_t currentPosition = 0;

      ReadNumber(theEnv,logicalName,&theToken,false,&currentPosition);
     }

This is the same split ReadlineFunction makes a few hundred lines earlier, where the file branch already declares size_t currentPos = 0; and passes that.

What was checked. The behavioural tests are the discriminating evidence here, because the fix changes answers. tests/read-from-a-file.clp reads five numerals one per line; a sixth read past the end, which must be EOF; five numerals on one line rather than five lines; a symbol between two numerals, where the symbol must give FALSE and the numeral after it must still be read; two files open at once and read from alternately, so neither file's position can be standing in for the other's; three reads written out separately rather than in a loop; the types of the five, so a numeral arriving as a symbol would show; and a last pair of reads after all of the above, which is where the stale count used to make even the first read fail.

Against the core as it stands every one of those gives the numbers in the file. Against unmodified 6.4.2 the same script gives (12 FALSE FALSE FALSE FALSE) for the first and FALSE for the rest, and (SYMBOL SYMBOL SYMBOL SYMBOL SYMBOL) for the types.

The answers now match CLIPS 7.0.0's stdin path exactly — 12 -7 3.5 0 9999, and 42 FALSE 17 for the file with a symbol in it — which is the same function reading the same numerals through the branch that was already resetting the position.

The whole of ../../v1/fact-functions/tests gives identical output, and every other suite under ../ and ../../clips-optimizations/ passes.

Still present upstream. CLIPS 7.0.0 has the same ReadNumberFunction and gives the same wrong answers, line for line with 6.4.2, while its own ReadlineFunction shows the intended split.