#1248 COMP-3 compared against a numeric literal gives the wrong result for some values (trunk r5698)

[Brazil]

Summary: comparing a signed COMP-3 item against a numeric literal*
gives the wrong result for certain values. Comparing the same field, holding
the same value, against a COMP-3 item of the same width gives the right
answer — in the same run.

Found on trunk r5698 (cobc (GnuCOBOL) 4.0-early-dev.0), built on
macOS 15 / arm64 with clang. Deterministic across runs; unaffected by
-std=default, -std=ibm or -std=mvs.

The contradiction

One field, one value, two comparisons, one run:

       IDENTIFICATION DIVISION.
       PROGRAM-ID. GCV2.
       DATA DIVISION.
       WORKING-STORAGE SECTION.
       01  V  PIC S9(18) COMP-3.
       01  ZI PIC S9(18) COMP-3 VALUE 0.
       01  T  PIC -9(18).
       01  L  PIC X(3).
       01  I  PIC X(3).
       PROCEDURE DIVISION.
           MOVE -1 TO V.                     PERFORM SHOW.
           MOVE -999999999999999 TO V.       PERFORM SHOW.
           MOVE -9999999999999999 TO V.      PERFORM SHOW.
           MOVE -99999999999999999 TO V.     PERFORM SHOW.
           MOVE -999999999999999999 TO V.    PERFORM SHOW.
           MOVE -123456789012345678 TO V.    PERFORM SHOW.
           MOVE -100000000000000000 TO V.    PERFORM SHOW.
           STOP RUN.
       SHOW.
           MOVE V TO T.
           IF V < 0      MOVE 'lt ' TO L ELSE MOVE 'NOT' TO L.
           IF V < ZI     MOVE 'lt ' TO I ELSE MOVE 'NOT' TO I.
           DISPLAY T '  vs literal 0: ' L '   vs S9(18) item: ' I.

-000000000000000001  vs literal 0: lt    vs S9(18) item: lt
-000999999999999999  vs literal 0: NOT   vs S9(18) item: lt
-009999999999999999  vs literal 0: lt    vs S9(18) item: lt
-099999999999999999  vs literal 0: lt    vs S9(18) item: lt
-999999999999999999  vs literal 0: NOT   vs S9(18) item: lt
-123456789012345678  vs literal 0: NOT   vs S9(18) item: lt
-100000000000000000  vs literal 0: lt    vs S9(18) item: lt

The two columns must agree. The right-hand one is correct throughout.

Which values

The program below moves 36 values into a single PIC S9(18) COMP-3 field and
evaluates IF V < 0 on each. X marks a negative value reported as not
negative:

significant digits   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
all nines            . . . . . . . . .  .  .  X  .  .  X  .  .  X
123456789123...      . . . . . . . . .  .  X  X  .  .  X  .  .  X

Right answers sit either side of every wrong one, and the declared picture
never changes, so this is not a width guard. Narrower fields fail too:
PIC S9(15) COMP-3 VALUE -999999999999999 and
PIC S9(15) COMP-3 VALUE -123456789012345 both report not-negative.

       IDENTIFICATION DIVISION.
       PROGRAM-ID. GSWEEP.
       DATA DIVISION.
       WORKING-STORAGE SECTION.
       01  V PIC S9(18) COMP-3.
       01  T PIC -9(18).
       01  L PIC X(3).
       PROCEDURE DIVISION.
           MOVE -9 TO V. PERFORM SHOW.
           MOVE -99 TO V. PERFORM SHOW.
           MOVE -999 TO V. PERFORM SHOW.
           MOVE -9999 TO V. PERFORM SHOW.
           MOVE -99999 TO V. PERFORM SHOW.
           MOVE -999999 TO V. PERFORM SHOW.
           MOVE -9999999 TO V. PERFORM SHOW.
           MOVE -99999999 TO V. PERFORM SHOW.
           MOVE -999999999 TO V. PERFORM SHOW.
           MOVE -9999999999 TO V. PERFORM SHOW.
           MOVE -99999999999 TO V. PERFORM SHOW.
           MOVE -999999999999 TO V. PERFORM SHOW.
           MOVE -9999999999999 TO V. PERFORM SHOW.
           MOVE -99999999999999 TO V. PERFORM SHOW.
           MOVE -999999999999999 TO V. PERFORM SHOW.
           MOVE -9999999999999999 TO V. PERFORM SHOW.
           MOVE -99999999999999999 TO V. PERFORM SHOW.
           MOVE -999999999999999999 TO V. PERFORM SHOW.
           MOVE -1 TO V. PERFORM SHOW.
           MOVE -12 TO V. PERFORM SHOW.
           MOVE -123 TO V. PERFORM SHOW.
           MOVE -1234 TO V. PERFORM SHOW.
           MOVE -12345 TO V. PERFORM SHOW.
           MOVE -123456 TO V. PERFORM SHOW.
           MOVE -1234567 TO V. PERFORM SHOW.
           MOVE -12345678 TO V. PERFORM SHOW.
           MOVE -123456789 TO V. PERFORM SHOW.
           MOVE -1234567891 TO V. PERFORM SHOW.
           MOVE -12345678912 TO V. PERFORM SHOW.
           MOVE -123456789123 TO V. PERFORM SHOW.
           MOVE -1234567891234 TO V. PERFORM SHOW.
           MOVE -12345678912345 TO V. PERFORM SHOW.
           MOVE -123456789123456 TO V. PERFORM SHOW.
           MOVE -1234567891234567 TO V. PERFORM SHOW.
           MOVE -12345678912345678 TO V. PERFORM SHOW.
           MOVE -123456789123456789 TO V. PERFORM SHOW.
           STOP RUN.
       SHOW.
           MOVE V TO T.
           IF V < 0 MOVE 'ok ' TO L ELSE MOVE 'BAD' TO L.
           DISPLAY T ' ' L.

What is and is not affected

construct	result
IF V < 0, IF V > 0 against a literal	wrong for the values above
against a COMP-3 item of a different width	wrong
against a COMP-3 item of the same width	correct
IF V = 0	correct
IF V IS NEGATIVE	correct
MOVE V TO an edited field	correct — the value is intact
the same field declared PIC S9(18) DISPLAY	correct

IF V < 0 and IF V > 0 are both false for an affected value, which is what
suggests the comparison is collapsing to "equal".

Context

I maintain a small COBOL-74 compiler targeting S/370 and use GnuCOBOL as its
differential oracle. This surfaced as a disagreement on a regression test. My
compiler answers all 36 sweep values correctly on the target, which is what
sent me looking at this side rather than mine.

Twelve significant digits is well inside ordinary use, which is why I thought
it worth writing up carefully rather than just noting it.