#167 ERROR 914 DURING MVNTUNE, Core7, 32bit, Intel compiler

[Anonymous]

Dear Clint,

Thank you for supporting ATLAS. I tried to install atlas-3.9.33
with lapack-3.3.0.tgz using Intel compilerm, but it aborted:

../configure \
--prefix=/opt \
--shared \
--with-netlib-lapack-tarfile=/incoming/lapack-3.3.0.tgz \
-Si latune 1 \
-Si omp 4 \
-b 32 \
-D c -DL2SIZE=8388604 \
-Fa alg -fPIC \
-F ic "-DNoChange" \
-F gc "-DNoChange" \
-C ic /usr/bin/gcc \
-C if /opt/intel/fortran/bin/ifort \
-F if "-fpp -nus -Zp16 -align all -ftrapuv -pad -nbs -nofor_main" \
-v 2 \
>& configure.log

...................

DONE BASIC KERNEL TESTS:

900000:dmvn_C.c (M=435, N=8, lda=441) gets -26527070000000001790098628230119598965187185619547247129463806911326681199764952305231803292872136644635901107949032688853649722104791156730725783325977818211062081325021137322753833410743038420178139357818466195089671329553684330077783018037052836517492313321499331893103904307423987236380747996594176.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -26527070000000001790098628230119598965187185619547247129463806911326681199764952305231803292872136644635901107949032688853649722104791156730725783325977818211062081325021137322753833410743038420178139357818466195089671329553684330077783018037052836517492313321499331893103904307423987236380747996594176.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -26527070000000001790098628230119598965187185619547247129463806911326681199764952305231803292872136644635901107949032688853649722104791156730725783325977818211062081325021137322753833410743038420178139357818466195089671329553684330077783018037052836517492313321499331893103904307423987236380747996594176.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -26527070000000001790098628230119598965187185619547247129463806911326681199764952305231803292872136644635901107949032688853649722104791156730725783325977818211062081325021137322753833410743038420178139357818466195089671329553684330077783018037052836517492313321499331893103904307423987236380747996594176.00 MFLOPS
900000:dmvn_C.c (M=290, N=12, lda=294) gets -26006930000000000248470263313542411985991117715836914811355735589823430945230770943294152459968394649174603928228015281831266479088234746710178470465872683855133090634896597071851453314861780114740526871627385164029449419381346108179268415023265206624698580761623022110723833232822723944506867325075456.00 MFLOPS
900000:dmvn_C.c (M=217, N=16, lda=224) gets -25687670000000000265552098391420191286673631534524060668144782194672950780602599630738204891731014405988041685422447314308314762584905642272944474301559733693108218563019280163570529963004642800043715427446204877605626917323284009961997741957145769138334778483039788732433483024323345981905100095881216.00 MFLOPS
900000:dmvn_C.c (M=174, N=20, lda=182) gets -25590820000000001618262151491767357826497470813117936909844871858778713276371658613361708877219706514448596876477596234252092572034764676941775063885516542905626755867206074099409195517437228470588426364022661491846355479809457306803223856089199820833981819701702261825502997311913770561217320355627008.00 MFLOPS
900000:dmvn_C.c (M=145, N=24, lda=154) gets -25486789999999998706841898396965225006795049812126582493247664268320180690696589581356501627064652653713306748506997874808883236071678336689631157605767549499204099944772056820949073218980521809302914385436304132969227509209007886280753812009477576731904848201746712328343762158221460652632986653556736.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -26527070000000001790098628230119598965187185619547247129463806911326681199764952305231803292872136644635901107949032688853649722104791156730725783325977818211062081325021137322753833410743038420178139357818466195089671329553684330077783018037052836517492313321499331893103904307423987236380747996594176.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -26527070000000001790098628230119598965187185619547247129463806911326681199764952305231803292872136644635901107949032688853649722104791156730725783325977818211062081325021137322753833410743038420178139357818466195089671329553684330077783018037052836517492313321499331893103904307423987236380747996594176.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -265270700000xmvnksearch: /dist/atlas-3.9.33/build_32/..//tune/blas/gemv/mvnksearch.c:1543: TuneExtractMUNU: Assertion `kp' failed.
00001790098628230119598965187185619547247129463806911326681199764952305231803292872136644635901107949032688853649722104791156730725783325977818211062081325021137322753833410743038420178139357818466195089671329553684330077783018037052836517492313321499331893103904307423987236380747996594176.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -26527070000000001790098628230119598965187185619547247129463806911326681199764952305231803292872136644635901107949032688853649722104791156730725783325977818211062081325021137322753833410743038420178139357818466195089671329553684330077783018037052836517492313321499331893103904307423987236380747996594176.00 MFLOPS
900000:dmvn_C.c (M=290, N=12, lda=294) gets -26006930000000000248470263313542411985991117715836914811355735589823430945230770943294152459968394649174603928228015281831266479088234746710178470465872683855133090634896597071851453314861780114740526871627385164029449419381346108179268415023265206624698580761623022110723833232822723944506867325075456.00 MFLOPS
900000:dmvn_C.c (M=217, N=16, lda=224) gets -25687670000000000265552098391420191286673631534524060668144782194672950780602599630738204891731014405988041685422447314308314762584905642272944474301559733693108218563019280163570529963004642800043715427446204877605626917323284009961997741957145769138334778483039788732433483024323345981905100095881216.00 MFLOPS
900000:dmvn_C.c (M=174, N=20, lda=182) gets -25590820000000001618262151491767357826497470813117936909844871858778713276371658613361708877219706514448596876477596234252092572034764676941775063885516542905626755867206074099409195517437228470588426364022661491846355479809457306803223856089199820833981819701702261825502997311913770561217320355627008.00 MFLOPS
900000:dmvn_C.c (M=145, N=24, lda=154) gets -25486789999999998706841898396965225006795049812126582493247664268320180690696589581356501627064652653713306748506997874808883236071678336689631157605767549499204099944772056820949073218980521809302914385436304132969227509209007886280753812009477576731904848201746712328343762158221460652632986653556736.00 MFLOPS
make[3]: *** [res/dMVNK.sum] Aborted
make[3]: Leaving directory `/d2/dist/atlas-3.9.33/build_32/tune/blas/gemv'
make[2]: *** [/dist/atlas-3.9.33/build_32/tune/blas/gemv/res/dMVNK.sum] Error 2
make[2]: Leaving directory `/d2/dist/atlas-3.9.33/build_32/bin'
ERROR 914 DURING MVNTUNE!!. CHECK INSTALL_LOG/dMVNTUNE.LOG FOR DETAILS.
make[2]: Entering directory `/d2/dist/atlas-3.9.33/build_32/bin'
cd /dist/atlas-3.9.33/build_32 ; make error_report
make[3]: Entering directory `/d2/dist/atlas-3.9.33/build_32'
make -f Make.top error_report
make[4]: Entering directory `/d2/dist/atlas-3.9.33/build_32'
uname -a 2>&1 >> bin/INSTALL_LOG/ERROR.LOG
/usr/bin/gcc -v 2>&1 >> bin/INSTALL_LOG/ERROR.LOG
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-unknown-linux-gnu/4.5.0/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: ../configure --prefix=/usr --libexecdir=/usr/lib64 --enable-shared --enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu --enable-languages=c,c++,fortran,java,objc,obj-c++
Thread model: posix
gcc version 4.5.0 (GCC)
/usr/bin/gcc -V 2>&1 >> bin/INSTALL_LOG/ERROR.LOG
gcc: '-V' option must have argument

......................

Useful info:

uname -a
Linux pethome 2.6.37 #2 SMP Sun Jan 9 22:01:16 EST 2011 x86_64 x86_64 x86_64 GNU/Linux

/opt/intel/fortran/bin/ifort -V
Intel(R) Fortran Compiler for applications running on IA-32, Version 10.1 Build 20071116 Package ID: l_fc_p_10.1.011
Copyright (C) 1985-2007 Intel Corporation. All rights reserved.
FOR NON-COMMERCIAL USE ONLY

My previous install of atlas-3.9.25 with the same compiler, the same flags
and lapack-3.2.2 was successful.

I saw 3156174 issue, but unfortunately all correspondence is gone. :-(
Is there a workaround?

Sincerely,
Leonid
2011.02.01_12:08:03

[Nobody/Anonymous]

Tar ball with error report

[R. Clint Whaley]

Leonid,

The first thing is to see if this problem is due to lack of precision on timing, which is what I suspect. Can you do a now configure and add the flag
-D c -DPentiumCPS=3500
and see if this error repeats itself or not?

BTW, I see that you are using OpenMP. Is there some reason for that? I'm not sure I'm keeping OpenMP going forward . . .

Thanks,
Clint

[bllac]

Dear Clint,

Thank you for the prompt reply. I did this:
added option -D c -DPentiumCPS=2920:

../configure \
--prefix=/opt \
--shared \
--with-netlib-lapack-tarfile=/incoming/lapack-3.3.0.tgz \
-Si latune 1 \
-Si omp 4 \
-b 32 \
-D c -DL2SIZE=8388604 \
-D c -DPentiumCPS=2920 \
-Fa alg -fPIC \
-F ic "-DNoChange" \
-F gc "-DNoChange" \
-C ic /usr/bin/gcc \
-C if /opt/intel/fortran/bin/ifort \
-F if "-fpp -nus -Zp16 -align all -ftrapuv -pad -nbs -nofor_main" \
-v 2 \
>& configure.log

Installation aborted at the same place with a little bit different message.

DONE BASIC KERNEL TESTS:

900000:dmvn_C.c (M=435, N=8, lda=441) gets -16152270000000000211598086760669994511715016270455448125790711440103805123049030959145412692774385656154264006343734099515433653100418896918498530664434242549221213810463354829813264022094766578474606224992504467629154827857483703816885609739755176877916235908861906850053264548062995966847309311901696.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -16152270000000000211598086760669994511715016270455448125790711440103805123049030959145412692774385656154264006343734099515433653100418896918498530664434242549221213810463354829813264022094766578474606224992504467629154827857483703816885609739755176877916235908861906850053264548062995966847309311901696.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -16152270000000000211598086760669994511715016270455448125790711440103805123049030959145412692774385656154264006343734099515433653100418896918498530664434242549221213810463354829813264022094766578474606224992504467629154827857483703816885609739755176877916235908861906850053264548062995966847309311901696.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -16152270000000000211598086760669994511715016270455448125790711440103805123049030959145412692774385656154264006343734099515433653100418896918498530664434242549221213810463354829813264022094766578474606224992504467629154827857483703816885609739755176877916235908861906850053264548062995966847309311901696.00 MFLOPS
900000:dmvn_C.c (M=290, N=12, lda=294) gets -15835560000000000152529841572288273962491120328934521053691145127820944305127634979580964659214716192153878676217119200625158609594440444090912205623738835680332996315395556387165468022379114440147633347142599455301921539018342244412583618375294580232454435100304004958194237624723189377172886609985536.00 MFLOPS
900000:dmvn_C.c (M=217, N=16, lda=224) gets -15641160000000000639581370373647190865815071018093842221868914522666957015544700981570649818338299640888066899850690211936254218518016564459020164133232768528121016164176192912744282870559989481874427471222884341200584839095487536734552273462061222011242112844348648867174391202443518847833373946150912.00 MFLOPS
900000:dmvn_C.c (M=174, N=20, lda=182) gets -15582189999999999881407718874549102576148793195309387653052221406867907605257260341816780605439672831579842913939593347152163951738639504308145094755303676115314494298763609657520929659261936037586321298390913285157764571143346830454772832811369525843925074023330328723295963873909547373248348984180736.00 MFLOPS
900000:dmvn_C.c (M=145, N=24, lda=154) gets -15518839999999998974123963648737578678451172485371635266xmvnksearch: /dist/atlas-3.9.33/build_32/..//tune/blas/gemv/mvnksearch.c:1543: TuneExtractMUNU: Assertion `kp' failed.
795225459904343259539949729453388491018507781284855855209334629931010450833371103659835626403649258461905138717439218066886164205940178842321991736933883641562836566160789552836435664649051098226193032141109329459279949640674399126573466586710016.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -16152270000000000211598086760669994511715016270455448125790711440103805123049030959145412692774385656154264006343734099515433653100418896918498530664434242549221213810463354829813264022094766578474606224992504467629154827857483703816885609739755176877916235908861906850053264548062995966847309311901696.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -16152270000000000211598086760669994511715016270455448125790711440103805123049030959145412692774385656154264006343734099515433653100418896918498530664434242549221213810463354829813264022094766578474606224992504467629154827857483703816885609739755176877916235908861906850053264548062995966847309311901696.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -16152270000000000211598086760669994511715016270455448125790711440103805123049030959145412692774385656154264006343734099515433653100418896918498530664434242549221213810463354829813264022094766578474606224992504467629154827857483703816885609739755176877916235908861906850053264548062995966847309311901696.00 MFLOPS
900000:dmvn_C.c (M=435, N=8, lda=441) gets -16152270000000000211598086760669994511715016270455448125790711440103805123049030959145412692774385656154264006343734099515433653100418896918498530664434242549221213810463354829813264022094766578474606224992504467629154827857483703816885609739755176877916235908861906850053264548062995966847309311901696.00 MFLOPS
900000:dmvn_C.c (M=290, N=12, lda=294) gets -15835560000000000152529841572288273962491120328934521053691145127820944305127634979580964659214716192153878676217119200625158609594440444090912205623738835680332996315395556387165468022379114440147633347142599455301921539018342244412583618375294580232454435100304004958194237624723189377172886609985536.00 MFLOPS
900000:dmvn_C.c (M=217, N=16, lda=224) gets -15641160000000000639581370373647190865815071018093842221868914522666957015544700981570649818338299640888066899850690211936254218518016564459020164133232768528121016164176192912744282870559989481874427471222884341200584839095487536734552273462061222011242112844348648867174391202443518847833373946150912.00 MFLOPS
900000:dmvn_C.c (M=174, N=20, lda=182) gets -15582189999999999881407718874549102576148793195309387653052221406867907605257260341816780605439672831579842913939593347152163951738639504308145094755303676115314494298763609657520929659261936037586321298390913285157764571143346830454772832811369525843925074023330328723295963873909547373248348984180736.00 MFLOPS
900000:dmvn_C.c (M=145, N=24, lda=154) gets -15518839999999998974123963648737578678451172485371635266795225459904343259539949729453388491018507781284855855209334629931010450833371103659835626403649258461905138717439218066886164205940178842321991736933883641562836566160789552836435664649051098226193032141109329459279949640674399126573466586710016.00 MFLOPS
make[3]: *** [res/dMVNK.sum] Aborted
make[3]: Leaving directory `/d2/dist/atlas-3.9.33/build_32/tune/blas/gemv'
make[2]: *** [/dist/atlas-3.9.33/build_32/tune/blas/gemv/res/dMVNK.sum] Error 2
make[2]: Leaving directory `/d2/dist/atlas-3.9.33/build_32/bin'
ERROR 914 DURING MVNTUNE!!. CHECK INSTALL_LOG/dMVNTUNE.LOG FOR DETAILS.

I put error report in http://lpetrov.net/misc/20110202_error_Corei732SSE3.tgz

> BTW, I see that you are using OpenMP. Is there some reason for that? I'm
> not sure I'm keeping OpenMP going forward . . .

I use OpenMP in my applications. What is your suggestion?
To drop "-Si omp 4" ?

Sincerely,
Leonid
2011.02.02_11:22:07

[R. Clint Whaley]

Leonid,

To say I'm confused is putting it mildly. Before we try all the funkiness of using the intel compiler and so on, let's try an install with the minimum of changes, and then add back in flags until we get failure (assuming we don't with a straight vanilla install).

So, get 3.9.34, and pass ONLY '-b 32 -D c -DPentiumCPS=2920' to configure, and let's see if that has the error. If it does, post here and tell me where I can find the error file. If it succeeds, add one of the flags you used before, and keep adding them until you get the failure again.

Thanks,
Clint

[R. Clint Whaley]

BTW, I meant get the newest dev release, not 3.9.34. The newest release today is 3.9.36.

[R. Clint Whaley]

Hi,

Just managed to find an event so similar, it must be the same problem. On my solaris box, during the tuning of the rank-2 update at the same place (dr2_C.c), the search goes into an infinite loop, where MFLOPS is always 0.
I will attempt to track down, and will update this page when I have a fix.

In the meantime, don't feel the need to do the test I mentioned, since I can probably find the problem by looking at my Solaris box.

I am escalating this support request to the bug tracker.

[R. Clint Whaley]

This bug should be fixed in 3.9.37, please reopen if it reoccurs.

Cheers,
Clint