#171 atlas-3.9.37 fails to build on linux PPCG564Altivec

[acrux @ cruxppc . org]

atlas-3.9.37 fails to build on a 2x PPC970MP based machine with GNU/Linux CRUX PPC 2.7 (64bit).
Toolchain: binutils-2.21, glibc-2.12.2, gcc-4.5.2, libgmp-5.0.1, libmpfr-3.0.0-p8, libmpc-0.8.2, ppl-0.11.1, cloog-ppl-0.15.10

Anyway it seems to be caused from a couple of macosx flags "-faltivec-force_cpusubtype_ALL" that are wrongly inserted when it compiles dmvtime.
I was unable to locate them to fix this build issue.

My configure is:

../configure -Fa alg -fPIC -b 64 -s PPC -O Linux \
-Ss flapack '$SRC/lapack-3.3.0/lapack.a' \
-Ss pmake 'make -j4' \
--shared --cc=gcc -v 2 \
--prefix=/usr --mandir=/usr/man

Here attached the ATLAS errata file and more logs here:
http://cruxppc.org/~acrux/atlas/3937/

thanks a lot for any help.
--nico

=============================================================

Useful information:

root@radio:~# uname -a
Linux radio 2.6.38-rc6 #1 SMP PREEMPT Thu Feb 24 20:53:14 CET 2011 ppc64 PPC970MP, altivec supported GNU/Linux
root@radio:~# cat /proc/cpuinfo
processor : 0
cpu : PPC970MP, altivec supported
clock : 2500.000000MHz
revision : 1.1 (pvr 0044 0101)

processor : 1
cpu : PPC970MP, altivec supported
clock : 2500.000000MHz
revision : 1.1 (pvr 0044 0101)

processor : 2
cpu : PPC970MP, altivec supported
clock : 2500.000000MHz
revision : 1.1 (pvr 0044 0101)

processor : 3
cpu : PPC970MP, altivec supported
clock : 2500.000000MHz
revision : 1.1 (pvr 0044 0101)

timebase : 14318378
platform : Maple
model : TSS,7047-191

root@radio:~# lscpu
Architecture: ppc64
Byte Order: Big Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 1
CPU socket(s): 4
NUMA node(s): 1
Model: TSS,7047-191
L1d cache: 32K
L1i cache: 64K
NUMA node0 CPU(s): 0-3

root@radio:~# gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/powerpc64-unknown-linux-gnu/4.5.2/lto-wrapper
Target: powerpc64-unknown-linux-gnu
Configured with: ../gcc-4.5.2/configure --with-cpu=default64 --prefix=/usr --mandir=/usr/man --libexecdir=/usr/lib --enable-languages=c,c++,objc --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-shared --disable-nls --with-x=no --enable-long-long --with-system-zlib --with-pkgversion='CRUX PPC' --host=powerpc64-unknown-linux-gnu --build=powerpc64-unknown-linux-gnu --target=powerpc64-unknown-linux-gnu
Thread model: posix
gcc version 4.5.2 (CRUX PPC)

root@radio:~# gfortran -v
Using built-in specs.
COLLECT_GCC=gfortran
COLLECT_LTO_WRAPPER=/usr/lib/gcc/powerpc64-unknown-linux-gnu/4.5.2/lto-wrapper
Target: powerpc64-unknown-linux-gnu
Configured with: ../gcc-4.5.2/configure --with-cpu=default64 --prefix=/usr --mandir=/usr/man --libexecdir=/usr/lib --enable-languages=fortran --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-shared --disable-nls --with-x=no --enable-long-long --with-system-zlib --with-pkgversion='CRUX PPC' --host=powerpc64-unknown-linux-gnu --build=powerpc64-unknown-linux-gnu --target=powerpc64-unknown-linux-gnu
Thread model: posix
gcc version 4.5.2 (CRUX PPC)

[acrux @ cruxppc . org]

error_PPCG564AltiVec.tgz

[acrux @ cruxppc . org]

i tried a new build with more personalized flags:
../configure -b 64 -s PPC -O Linux -Ss flapack 'lapack-3.3.0/lapack.a' --shared --cc=gcc -v 2 -Ss pmake 'make -j4' --prefix=/usr -F alg "-fPIC -O3 -mcpu=970 -maltivec -mabi=altivec -mpowerpc-gfxopt -ftree-vectorize -ftracer -ftree-loop-linear -ftree-loop-distribution -fpeel-loops -floop-interchange -floop-strip-mine -floop-block" -Si latune 1 -D f '-DAdd_ -DF77_INTEGER=int -DStringSunStyle'

but i got the same failure... thus I saw the same macosx flags "-faltivec -force_cpusubtype_ALL" compiling ATL_dmvnk.c . I can see these flags only in ARCHS/PPCG5{32,64}AltiVec/kern/gcc/*.sum

Please advise.
my best regards,
--nico

[R. Clint Whaley]

Moving to ATLAS bug tracker

[R. Clint Whaley]

The problem is coming from the architectural defaults, which contain these flags because I tuned on an OS X box. I will need to figure out how to get these flags out of the arch defs, but add them back in when people are using apple's maddening gcc.

For now, your best bet is to manually remove the flags yourself. You can install w/o arch defs by passing
-Si archdef 0
to configure. This will cause the full install to run, and will take an eternity.

The other thing you can do as a general rule (since I know you are interested in other PPC machines, which will have this problem as well), is manually strip the problematic flags from the archdefs yourself. They are stored in ATLAS/CONFIG/ARCHS as gzipped tarfiles. If you untar those guys, and then write a script to remove the problematic flags from all routines, and then retar/gzip the files (using the same name), then you can install using your modified arch defs.

Please let me know if you do either of these. In the meantime, I'll see if I can figure something out for fixing this arch def bug long-term.

Thanks for the bug report,
Clint

[R. Clint Whaley]

shell file to remove apple flags

[R. Clint Whaley]

I've given it a look, and I think it will be a while before I can get a clean fix for this bug. In the meantime, you can save the killflags.sh file attached below to your ATLAS/CONFIG/ARCHS directory, and if you then issue:
./killflags.sh PPCG432AltiVec PPCG532AltiVec PPCG564AltiVec

Then you should be able to install under linux using the arch defs. This should work for any developer release. Let me know if this workaround fixes the problem for you.

Thanks,
Clint

[acrux @ cruxppc . org]

thanks for your script, it worked fine and macosx flags were removed albeit ARCHS/PPCG4AltiVec
was already clean i.e. no hardcoded macosx flags.

I guess, the best practice is to avoid every hardcoded flags. Example give, we on CRUX PPC have a modern toolchain and some of the hardcoded cflags are quite deprecated (or simply, now, already bydefault in -O3) and again with this practice we can use new ones (like new gcc's graphite loops).

Maybe using CFLAGS= '${DKCFLAGS} -m64 -DPFYDIST=0'
instead of CFLAGS='-mpowerpc64 -maltivec -mabi=altivec -mcpu=970 -mtune=970 -O3 -fno-schedule-insns -fno-rerun-loop-opt -m64 -DPFYDIST=0'

Anyway with this new build on my dual PPC970MP (1MB L2 for each core) I got a CacheEdge DETECTION error (as I remember i got the same kind of error with 3.8.3). Now attached new error_PPCG564AltiVec.tgz, make.inc and full build log.

my updated 64bit toolchain: binutils-2.21, glibc-2.12.2, gcc-4.5.2, libgmp-5.0.1, libmpfr-3.0.0-p8, libmpc-0.8.3, ppl-0.11.2, cloog-ppl-0.15.10

and my new build script (even it still needs some work in shared libs section):

export SMP="$(cat /proc/cpuinfo | grep ^processor | wc -l)"

if \[ "$HOSTTYPE" == "powerpc64" \]; then
    CONF="-b 64"
    else CONF="-b 32"
fi
if \[ "$SMP" \!= "1" \]; then
    PT="pt"
fi

../configure $CONF -s PPC -O Linux -v 2 \
    -C ac gcc -F ac '$\{CFLAGS\} -fPIC' \
    -C if gfortran -F if '$\{FFLAGS\} -fPIC' \
    \--shared \
    -D f '-DAdd\_ -DF77\_INTEGER=int -DStringSunStyle' \
    -Ss flapack 'lapack-3.3.0/lapack.a' -Si latune 1 \
    -Ss pmake 'make -j$\{SMP\}' \
    \--prefix=/usr

make -j1
make -j1 $\{PT\}check

make install

install -D -m 755 lib/\*.so usr/lib

where:
CFLAGS="-O3 -mcpu=970 -mabi=altivec -fsigned-char -fomit-frame-pointer -ftree-loop-linear -ftree-loop-distribution -fpeel-loops -floop-interchange -floop-strip-mine -floop-block -fgraphite-identity -floop-parallelize-all"
FFLAGS="-O3 -mcpu=970 -mabi=altivec -fomit-frame-pointer -ftree-loop-linear -ftree-loop-distribution -fpeel-loops -floop-interchange -floop-strip-mine -floop-block -fgraphite-identity -floop-parallelize-all"

also i noticed a kind of limit in the number of compiler's flags (i.e. the lenght of CFLAGS+FFLAGS) of before the configure script turns out an error.

Talking about PPC970 (aka G5 PowerPC64):
PPC970 - 512KB L2
PPC970FX - 512KB L2
PPC970MP - dual core with 1MB L2 for each core
PPC970GX - 1MB L2

[acrux @ cruxppc . org]

new: cache edge detection error

[acrux @ cruxppc . org]

new: Make.inc

[acrux @ cruxppc . org]

new: full build log

[R. Clint Whaley]

For the new problems, would be best to open up a new support request, so I can keep it separate from the bug here.

I don't have time right now to investigate the G5 properly, and I think there are several problems there. In the meantime, what about trying an install w/o using the arch defaults just to see if something will work? Should just need to add -Si archdef 0 to your configure line . . .

Open up a new support request to reply and/or to follow up about new problems.

Thanks,
Clint

[R. Clint Whaley]

Hi,

I think the G5 now completely works in 3.9.70. Please open up a new help request if you experience problems with that release as well.

Thanks,
Clint