#470 Calculations with FLOAT-DECIMAL-16 are partly faulty

[Ingo Bauer]

Calculations with FLOAT-DECIMAL-16 are partly faulty.
Compiler 2.2.0, build 2017-10-25
System macOS 10.13.1

[Simon Sobisch]

In general: float items are only aproximate - and all of your samples raise a non-fatal EC-SIZE-OVERFLOW, which you could check with ON SIZE ERROR and/or FUNCTION EXCEPTION-STATUS.
You may want to do further checks and tell us about your findings.

[Ingo Bauer]

In fact, the simplest arithmetic operations with FLOAT-DECIMAL-16 produce an error message, see table and example in the appendix. The same arithmetic operations with FLOAT-SHORT / -LONG are error-free.

Example-Table
1.0E3 / 2.0E0 --> OK
1.0E3 / 2.1E0 --> ERR
: :
1.0E3 / 2.4E0 --> ERR
1.0E3 / 2.5E0 --> OK
1.0E3 / 2.6E0 --> ERR
: :
1.0E3 / 3.0E0 --> ERR
: :
: :

[Ingo Bauer]

New example...

[Edward Hart]

The behaviour seems to depend on whether ON SIZE ERROR is specified or not:

:::cobolfree
COMPUTE FD16 = 1.0E+3 / 2.1E+0
DISPLAY FD16 " " FUNCTION EXCEPTION-STATUS
*> Displays "9265504389275257E-9".

COMPUTE FD16 = 1.0E+3 / 2.1E+0
     ON SIZE ERROR CONTINUE
     NOT ON SIZE ERROR CONTINUE
END-COMPUTE
DISPLAY FD16 " " FUNCTION EXCEPTION-STATUS
*> Displays "0E0 EC-SIZE-OVERFLOW".

Ingo's examples above suggest EC-SIZE-OVERFLOW is raised whenever it tries to store a non-decimal fraction in a FLOAT-DECIMAL-16.

[Ingo Bauer]

I wrote a test program again, maybe this helps

[Simon Sobisch]

Ron, can you have a look at this please?

[Ron Norman]

I will have a look. Then I can learn what FLOAT-DECIMAL-16 actually means...

[Ron Norman]

The problem is definitely with that data type FLOAT-DECIMAL-16. If I change FD16 to be COMP-2 then I do get:
--- FLOAT-DECIMAL-34 ---
1>> 9216586861751152073732718894009216E-27
2>> 530503678779840848806366047745358E-26

--- FLOAT-DECIMAL-16 ---
3>> 9216586.86175115 OK
4>> 5305036.787798408 ERROR

Also notice that FLOAT-DECIMAL-34 and FLOAT-DECIMAL-16 are DISPLAYed in exponent format rather that inserting the decimal place as happend for COMP-1/2.

Should that also be fixed?
Say if there are fewer than 16 decimal places, then default to DISPLAY with the decimal inserted, otherwise use the nnnnnE+/-ee style.

[Simon Sobisch]

I assume COMP-1/2 are not exactly the same as the stadard decimal floating-point usages.
To have the specification in, here are the most important quotes from the COBOL2014 Standard about these:

• FLOAT-DECIMAL-16 and FLOAT-DECIMAL-34 are "standard decimal floating-point usages", category numeric
• the FLOAT-DECIMAL clause in the OPTIONS paragraph specifies the encoding and the endianness that is implied for these
• both the encoding and endianness may be specified in the USAGE, if they aren't the implied ones are used
• The FLOAT-DECIMAL-16 phrase specifies that the data item is a floating-point data item in 64-bit basic decimal interchange format (decimal-64) as specified in ISO/IEC/IEEE 60559:2011, clause 3.5, Decimal interchange format encodings.
• The value storedd can be no greater than +9.9999 999 999 999 999E+384, nor can it be less than -9.999 999 999 999 99E+384
• The nonzero values closest to zero that can be represented are -1.0E-398 and +1.0E-398. The largest positive integer value with a nonzero trailing digit that can be represented is +(1.0E+17 - 1), or 9 999 999 999 999 999. Every value is the product of an integer and the value 1.0E-398.
• The FLOAT-DECIMAL-34 phrase specifies that the data item is a floating-point data item in 128-bit basic decimal interchange format (decimal128) as specified in ISO/IEC/IEEE 60559:2011, clause 3.5, Decimal interchange format encodings.
• The value in a data item described with the FLOAT-DECIMAL-34 usage can be no greater than +9999 999 999 999 999 999 999 999 999 999 999E+6144, nor can it be less than -9.999 999 999 999 999 999 999 999 999 999 999E+6144. The nonzero values closest to zero that can be represented are 1.0E-6176 and +1.0E-6175. The largest positive integer value with a nonzero trailing digit that can be represented is +(1.0E+35 - 1), or 9 999 999 999 999 999 999 999 999 999 999 999. Every value is the product of an integer and the value 1.0E-6175.
• The format or formats for a standard-decimal intermediate data item (SDIDI) are defined by the implementor. In all respects the numeric values of operands, the results of numeric operations on those values, and the exception conditions raised are those that would be obtained ifthe form of an SDIDI had been that of a data item declared USAGE FLOAT-DECIMAL-34 in the particular implementation.

To the one direct question: yes, I think it would be best to display these standard defined items in exponent form.

[Ron Norman]

The problem is a simple coding error and I have a fix. So now I get results as follows:
--- FLOAT-DECIMAL-34 ---
A: 9216586861751152073732718894009216E-27
B: 530503678779840848806366047745358E-26

--- FLOAT-DECIMAL-16 ---
A: 9216586861751152E-9
B: 5305036787798408E-9

--- COMP-2 ---
A: 9216586.86175115
B: 5305036.787798408

Micro Focus 5.1 Server Express does not support FLOAT-DECIMAL but when you DISPLAY a COMP-2 field it is output as follows:
--- COMP-2 ---
A: .921658686175115220E 007
B: .530503678779840840E 007

[Simon Sobisch]

The calculations are assumed to be fine with [r2292] which merged Ron's changes in [r2289] and [r2290].

Note: There are some issues with the intermediate rounding and currently the result differs heavily if the ON SIZE ERROR is specified or not - but this is tracked in [bugs:#478].