#691 Test failures on 3.1.1

[Luigi Baldoni]

While trying to update the gnu-cobol package on openSUSE, I've incurred into three unexpected test failures:

577: Recursive CALL of RECURSIVE program             FAILED (run_misc.at:1305)
696: DELETE FILE, INDEXED                            FAILED (run_file.at:335)
943: CALL unusual PROGRAM-ID.                        FAILED (run_extensions.at:1605)

Not sure what to make of it, I'm also attaching the full build log if it can be of any use.

[Simon Sobisch]

Test 577 is noted in the news to be potentially not portable, especially on non-gcc builds.
Test 696 may be a false positive, test 943 is something I've not seen with a test failure for quite a while.
Thank you for the report and the bug output. Can you please attach tests/testsuite.log which includes the actual test details of all failed tests and the system configuration?
If possible you may want to also run make test to verify with the NIST COBOL85 suite that the package work as expected (will need perl as test-dependency, not needed for runtime).

[Luigi Baldoni]

Can you please attach tests/testsuite.log which includes the actual test
details of all failed tests and the system configuration?

Unfortunately it was a remote build on OBS and I couldn't obtain that log.

I tried building gnu-cobol 3.1.1 locally but alas the errors have multiplied (see attached archive).
But at least it seems like most of them are caused by a compiler switch.

If possible you may want to also run make test to verify with the NIST COBOL85 suite that the package work as expected

Alas building in a network-isolated chroot prevents me from doing that.

[Simon Sobisch]

To work around the compiler flags issue please use one of --enable-hardening or --disable-hardening (previous versions effectively used the later, but as you have those flags as default you may want to explicit enable them).

Concerning the use of NIST in the chroot isolated environment: this is normally no problem if you pre-download that via an additional source (one of https://www.itl.nist.gov/div897/ctg/suites/newcob.val.Z or https://sourceforge.net/projects/gnucobol/files/nist/newcob.val.tar.gz/download) and then before make test copy that file to tests/cobol85.

[Luigi Baldoni]

To work around the compiler flags issue please use one of --enable-hardening or --disable-hardening (previous versions effectively used the later, but as you have those flags as default you may want to explicit enable them).

I've tried both and now the unexpected failures are reduced to 2.
Please see new attachment.

Concerning the use of NIST in the chroot isolated environment: this is
normally no problem if you pre-download that via an additional source
(one of https://www.itl.nist.gov/div897/ctg/suites/newcob.val.Z or https://sourceforge.net/projects/gnucobol/files/nist/newcob.val.tar.gz/download) and then before make test copy that file to tests/cobol85.

That worked flawlessly. Do you recommend running make test in addition to make check?

[Luigi Baldoni]

Addendum: I've also tried disabling LTO and also disabling hardening I get the best result so far, but 602 still fails. Without LTO but with hardening, three tests still fail, see newest attachment.

[Simon Sobisch]

Thank you for the reports, the CANCEL of RECURSIVE called programs in the testcase (602) is now commented out - that obviously does not fix the underlying bug, but this one is actually mentioned as possible issue in the NEWS entry for 3.1 (but I think that's actually much older).

The compiler was correct in the other two test cases that (only theoretically) the buffer could be overwritten, I've made it bigger to silence that with [r4105].