#3 Endless make/configure loop w/ gcc-8.1.0 and clang-6.0.0 on i686 (32 bit)

[martin.vaeth]

Running "make" on an i686 system w/ gcc-8.1.0 (or clang-6.0.0, see below) takes endless: The same configure script is called in an endless loop (build log attached).

Observations:

1. The build logs show suspicious messages which never occured before: configure.in:151: warning: AC_TRY_RUN called without default to allow cross compiling
2. No such problems on x86_64 (64 bit)
3. No such problems with gcc-7.3.0
4. No such problems when manually stopping and running "smake -r all" (with an smake which was compiled previously with gcc-7.3.0).
5. Removing -O and/or replacing by -O0 or -O2 does not change anything.
6. No change when adding -fno-strict-aliasing
7. When compiling smake with clang(-6.0.0), there is the same problem (build log also attached; this log is the output of an ebuild run which patches the compiler to be clang)
8. Tested Schily Tools versions 2018.05.02, 2018.04.17, and 2017.09.25

[Jörg Schilling]

The problem is that smake believes that

$(SRCROOT)/autoconf/configure

is older than one of:

$(SRCROOT)/autoconf/autoconf.m4 \  
$(SRCROOT)/autoconf/acgeneral.m4 \  
$(SRCROOT)/autoconf/acspecific.m4 \  
$(SRCROOT)/autoconf/acoldnames.m4 \  
$(SRCROOT)/autoconf/aclocal.m4 \  
$(SRCROOT)/autoconf/rules.cnf.in \  
$(SRCROOT)/autoconf/xconfig.h.in 

And as a result believes that it needs to call $(SRCROOT)/autoconf/autoconf.

So you either used a defective tar implementation to unpack the archive (and the time stamps are wrong) or the compiler created defective code for smake.

What do you get when you use such a smake from psmake on a just unpacked source, chdir to drectory "incs" and then call "smake -d"?

[martin.vaeth]

When I stop the loop, the timestamps of the mentioned files appear to be correct:

% ls -l --sort=time autoconf
-rwxr-xr-x 1 vaeth vaeth 676053 Mai 9 12:32 configure
-r--r--r-- 1 vaeth vaeth 32587 Mär 12 21:32 configure.in
-r--r--r-- 1 vaeth vaeth 58234 Mär 12 21:31 xconfig.h.in
-r--r--r-- 1 vaeth vaeth 1110 Mär 12 20:46 rules.cnf.in
-r--r--r-- 1 vaeth vaeth 103193 Jun 28 2017 aclocal.m4
-r--r--r-- 1 vaeth vaeth 87354 Jun 27 2017 acspecific.m4
-r-xr-xr-x 1 vaeth vaeth 20626 Mai 18 2017 config.sub
-r-xr-xr-x 1 vaeth vaeth 28696 Mai 9 2017 config.guess
-r--r--r-- 1 vaeth vaeth 85700 Mär 22 2016 acgeneral.m4
-r--r--r-- 1 vaeth vaeth 4136 Aug 12 2009 acoldnames.m4
-r-xr-xr-x 1 vaeth vaeth 5597 Apr 19 2009 autoconf
-r--r--r-- 1 vaeth vaeth 2585 Jan 5 1999 autoheader.m4
-r--r--r-- 1 vaeth vaeth 1186 Jan 5 1999 autoconf.m4
-r-xr-xr-x 1 vaeth vaeth 5585 Nov 20 1996 install-sh

What do you get when you use such a smake from psmake
on a just unpacked source, chdir to drectory "incs"
and then call "smake -d"?

I do not understand: When I unpack the tarball, "incs" is empty,
and smake -d just prints a help text.
When I call "smake -d" in the main directory, it depends on
whether I compiled smake with gcc-7.3.0 or gcc-8.1.0.

In the former case everything is fine:

MAKEFLAGS value: '-d'
Make: RULES/ldummy.lnk is out of date to: (NULL POINTER)
...echo " ==> MAKING SYMLINKS in ./RULES/" && \
cd ./RULES && sh ./MKLINKS
==> MAKING SYMLINKS in ./RULES/
...echo " ==> MAKING SYMLINKS in ./TEMPLATES/" && \
cd ./TEMPLATES && sh ./MKLINKS
==> MAKING SYMLINKS in ./TEMPLATES/
Make: incs/Dnull is out of date to: incs
...if [ ! -f incs/Dnull ]; then \
echo > incs/Dnull; \
fi
Make: incs/Dcc.i686-linux is out of date to: (NULL POINTER)
...sh conf/cc-config.sh cc cc incs/Dcc.i686-linux
Trying to find cc
Found cc
cc is gcc
Making gcc the default compiler in 'incs/Dcc.i686-linux'
Make: incs/i686-linux-gcc/Inull is out of date to: (NULL POINTER)
...echo " ==> MAKING DIRECTORY \"incs/i686-linux-gcc/Inull\""; umask 002; mkdir -p incs/i686-linux-gcc
==> MAKING DIRECTORY "incs/i686-linux-gcc/Inull"
...echo > incs/i686-linux-gcc/Inull
Make: incs/i686-linux-gcc/rules.cnf is out of date to: incs/i686-linux-gcc/Inull
...echo " ==> CONFIGURING RULES \"incs/i686-linux-gcc/rules.cnf\""; /bin/rm -f incs/i686-linux-gcc/rules.cnf incs/i686-linux-gcc/xconfig.h; cd incs/i686-linux-gcc; CONFIG_NOFAIL=TRUE CC="gcc" CFLAGS="
-O " CXXFLAGS=" -O " CPPFLAGS=" -D_GNU_SOURCE " LDFLAGS="-Llibs/i686-linux-gcc -L/opt/schily/lib -Wl,-R/opt/schily/lib -Wl,-R/opt/schily/lib" sh ../../autoconf/configure
==> CONFIGURING RULES "incs/i686-linux-gcc/rules.cnf"
creating cache ./config.cache
checking host system type... i686-pc-linux-gnu
[...]

In the latter case, I get a bad result:

MAKEFLAGS value: '-d'
Make: RULES/ldummy.lnk is out of date to: (NULL POINTER)
...echo " ==> MAKING SYMLINKS in ./RULES/" && \
cd ./RULES && sh ./MKLINKS
==> MAKING SYMLINKS in ./RULES/
...echo " ==> MAKING SYMLINKS in ./TEMPLATES/" && \
cd ./TEMPLATES && sh ./MKLINKS
==> MAKING SYMLINKS in ./TEMPLATES/
Make: incs/Dnull is out of date to: incs
...if [ ! -f incs/Dnull ]; then \
echo > incs/Dnull; \
fi
Make: incs/Dcc.i686-linux is out of date to: (NULL POINTER)
...sh conf/cc-config.sh cc cc incs/Dcc.i686-linux
Trying to find cc
Found cc
cc is gcc
Making gcc the default compiler in 'incs/Dcc.i686-linux'
Make: incs/i686-linux-gcc/Inull is out of date to: (NULL POINTER)
...echo " ==> MAKING DIRECTORY \"incs/i686-linux-gcc/Inull\""; umask 002; mkdir -p incs/i686-linux-gcc
==> MAKING DIRECTORY "incs/i686-linux-gcc/Inull"
...echo > incs/i686-linux-gcc/Inull
Make: ./autoconf/configure is out of date to: ./autoconf/xconfig.h.in
...echo " ==> AUTOCONFIGURING GLOBAL \"./autoconf/configure\""; \
cd ./autoconf && sh ./autoconf
==> AUTOCONFIGURING GLOBAL "./autoconf/configure"
configure.in:151: warning: AC_TRY_RUN called without default to allow cross compiling
configure.in:152: warning: AC_TRY_RUN called without default to allow cross compiling
configure.in:158: warning: AC_TRY_RUN called without default to allow cross compiling
configure.in:162: warning: AC_TRY_RUN called without default to allow cross compiling
configure.in:163: warning: AC_TRY_RUN called without default to allow cross compiling
[...]

[Jörg Schilling]

Oh sorry, I ment chdir to directory "inc" and then run the problematic "smake -d"

[Jörg Schilling]

Sorry again, use "smake -dddd" to get the time stamps printed.

[martin.vaeth]

The output with the two different smake binaries is attached

[Jörg Schilling]

Sorry, I cannot send a patch as there was a major rewrite meanwhile...

Please add:

error("gnewtime() = %s (%llx)\n", prtime((date_t)t), (long long)t);

just before the return() in gnewtime()

Rerun the test with the failing smake, without any -d debug. You should see:

gnewtime() = Tue Jan 19 04:14:07 2038 (7fffffff)

or

gnewtime() = (NULL POINTER) (7fffffffffffffff)

depending on whether you have a 32 bit or 64 bit binary.

[martin.vaeth]

For the working (gcc-7) version, this is the case. For the failing (gcc-8) version, I get on 32 bit

gnewtime() = Thu Feb 12 15:07:07 2009 (49942d0b)

On 64 bit, the patch causes a segfault with gcc-8 (currently, I no longer have gcc-7 on 64 bit; I would need to recompile it).

[martin.vaeth]

Aha: It seeems that the code of gnewtime() is relying on underflow behaviour. In
https://gcc.gnu.org/gcc-8/changes.html
it says:

-fno-strict-overflow is now mapped to -fwrapv -fwrapv-pointer and signed integer overflow is now undefined by default at all optimization levels. Using -fsanitize=signed-integer-overflow is now the preferred way to audit code, -Wstrict-overflow is deprecated.

Compiling with -fno-strict-overflow works with gcc-8 on 32 bit.

[Jörg Schilling]

This seems to be an indication for a really bad compiler.

A compiler that does crazy things should at least print a warning....

BTW: did you see that the code tries to avoid such overflows?

The code was an attempt to get the maximum portability while making no assumtion on the type of the opaque time_t.

[martin.vaeth]

This seems to be an indication for a really bad compiler.

As mentioned, I had the same problem with clang-6.0.0.
I guess the main motivation was that relying on overflow/underflow was the cause of some buffer overflows and other security vulnerabilities when signed types had suddenly become unsigned.
I also would have preferred a segfault, but probabably an explicit test was too costly.

BTW: did you see that the code tries to avoid such overflows?

I never looked at assembler output (actually, I do not even know how to get it without RTFM).

while making no assumtion on the type of the opaque time_t.

That's one of many reasons why C++ is so much nicer than C: For templates, it plays no role whether types are opaque: std::numeric_limits<time_t>::max() always does the right thing.
I guess, for C there is no other way than to do a configure test. OTOH, then this test happens at compile time instead of run time which is also an advantage.</time_t>

[Jörg Schilling]

You cannot look into the assembly code as this includes the bugs from the compiler ;-)

I would be interested to know that you see in case that you add:

long long ll;
unsigned long long ull;

ll = TYPE_MINVAL(long long);
printf("min ll %llx\n", ll);
ll = TYPE_MAXVAL(long long);
printf("max ll %llx\n", ll);

ull = TYPE_MINVAL(unsigned long long);
printf("min ull %llx\n", ull);
ull = TYPE_MAXVAL(unsigned long long);
printf("max ull %llx\n", ull);

to the beginning of main().

If this does not print what we expect, then these compilers are plain junk.

[martin.vaeth]

On 32 and 64 bit the output is:

min ll 8000000000000000
max ll 7fffffffffffffff
min ull 0
max ull ffffffffffffffff

[Jörg Schilling]

OK, then at least compiler math for constants is something we still may rely on.

[martin.vaeth]

schily-tools-2018.05.17 compiles on 32/64 bit without -fno-strict-overflow, so I guess the ticket can be closed.

BTW, considering how hesistant clang and gcc implemented spectre fixes, I suppose that the reason for the change in overflow handling are not any security considerations but pure optimization attempts: My guess is that

while (a > 0) {
   while ((t + a) > t) { ...

was optimized to

while (a > 0) {
  while (1) { ...

which is valid under the assumption that overflow does not happen.

What I fail to understand is why not even -Wstrict-overflow=5 warned about this assumption.
I did not check whether clang is better concerning the warning.

[Jörg Schilling]

Since the change in the behavior of the compiler did affect known behavior since ~ 44 years, I would call such a change without a warning a compiler bug.