Menu
▾
▴
Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-07-30 23:59:08
|
Hi, This is a bit off-topic, but is there a bit improvement in speed when using MKL vs ATLAS? Thanks Paul On 30 July 2013 23:29, Mailing list used for User Communication and Updates <kal...@li...> wrote: > That's right. They are working on it. > > --Mike > > > -----Original Message----- > From: Mailing list used for User Communication and Updates [mailto: > kal...@li...] > Sent: Monday, July 29, 2013 9:58 AM > To: kal...@li... > Subject: Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5 > > I previously spoke to Michael Deisher at Intel about this, he says he has > asked the relevant team. So I think they are aware of the problem. > Dan > > > On Mon, Jul 29, 2013 at 12:06 PM, Mailing list used for User Communication > and Updates <kal...@li...> wrote: > > It has been reported on intel dev forum. > > http://software.intel.com/en-us/forums/topic/402436 > > http://software.intel.com/en-us/forums/topic/402473 > > > > The problem is caused by the incorrect query result of workspace size > > given by dgesvd(). > > > > 11.0.5 seems to be a very buggy version. > > > > > > > > On Mon, Jul 29, 2013 at 11:39 PM, Mailing list used for User > > Communication and Updates <kal...@li...> wrote: > >> > >> I just checked different versions of MKL and indeed, this was > >> introduced in the update 5. Anyone with the commercial MKL support > >> cares to submit a ticket to Intel? > >> y. > >> > >> > >> On Wed, Jul 24, 2013 at 12:42 PM, Mailing list used for User > >> Communication and Updates <kal...@li...> wrote: > >>> > >>> You are right.... I just downgraded to MKL 11.0update1 and passed > >>> the test. > >>> > >>> > >>> > >>> On Thu, Jul 25, 2013 at 12:28 AM, Mailing list used for User > >>> Communication and Updates <kal...@li...> wrote: > >>>> > >>>> I saw some changes regarding improving performance of *GESVD in the > >>>> MKL > >>>> 11.0 update5 changelog. I wonder if they messed something up? How > >>>> about trying slightly "older" version of the MKL? If you are on the > >>>> subscription of Intel Software Network, you can download older > >>>> versions of the MKL at no charge... > >>>> > >>>> We are using Intel 11.0 and the tests pass without any problem... > >>>> > >>>> yenda > >>>> > >>>> > >>>> > >>>> On Wed, Jul 24, 2013 at 12:14 PM, Mailing list used for User > >>>> Communication and Updates <kal...@li...> wrote: > >>>>> > >>>>> I can't see any problems in the Kaldi code. Perhaps this is an > >>>>> older version of MKL? It could be a bug in MKL itself. MKL tests > >>>>> OK for me. > >>>>> Dan > >>>>> > >>>>> > >>>>> On Wed, Jul 24, 2013 at 11:55 AM, Mailing list used for User > >>>>> Communication and Updates <kal...@li...> wrote: > >>>>> > here's the dbg info copied from eclipse. > >>>>> > PS: I can pass the test with atlas provided by centos repo > >>>>> > atlas-sse3.x86_64 3.8.4-2.el6 > >>>>> > @base > >>>>> > atlas-sse3-devel.x86_64 3.8.4-2.el6 > >>>>> > @base > >>>>> > > >>>>> > > >>>>> > > >>>>> > stack: > >>>>> > > >>>>> > matrix-lib-test [C/C++ Application] matrix-lib-test [11312] > >>>>> > [cores: 1] Thread [1] 11312 [core: 1] (Suspended : Signal : > >>>>> > SIGABRT:Aborted) > >>>>> > raise() at 0x33090328a5 > >>>>> > abort() at 0x3309034085 > >>>>> > kaldi::KaldiAssertFailure_() at kaldi-error.cc:136 0x4cf401 > >>>>> > kaldi::MatrixBase<double>::LapackGesvd() at kaldi-matrix.cc:401 > >>>>> > 0x48fdd3 > >>>>> > kaldi::MatrixBase<double>::DestructiveSvd() at > >>>>> > kaldi-matrix.cc:1,330 > >>>>> > 0x48b902 > >>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.cc:1,352 > >>>>> > 0x48b9bb > >>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.h:315 0x468e50 > >>>>> > kaldi::MatrixBase<double>::Cond() at kaldi-matrix.cc:1,222 > >>>>> > 0x48bf47 > >>>>> > kaldi::InitRand<double>() at matrix-lib-test.cc:74 0x42dd7e > >>>>> > kaldi::UnitTestCompressedMatrix<double>() at > >>>>> > matrix-lib-test.cc:3,532 0x43055a > >>>>> > kaldi::MatrixUnitTest<double>() at matrix-lib-test.cc:3,705 > >>>>> > 0x42de8b > >>>>> > main() at matrix-lib-test.cc:3,829 0x42dac7 > >>>>> > > >>>>> > > >>>>> > > >>>>> > variable list after invoking the following @line393 > >>>>> > kaldi-matrix.cc > >>>>> > clapack_Xgesvd(v_job, u_job, > >>>>> > &M, &N, data_, &LDA, > >>>>> > s->Data(), > >>>>> > V->Data(), &V_stride, > >>>>> > U->Data(), &U_stride, > >>>>> > p_work, &l_work, > >>>>> > &result); > >>>>> > > >>>>> > > >>>>> > __func__ char [12] 0x4d5418 > >>>>> > M KaldiBlasInt 1 > >>>>> > LDA KaldiBlasInt 2 > >>>>> > U kaldi::MatrixBase<double> * 0x7ffffffface0 data_ double * > >>>>> > 0x7413b0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1 > >>>>> > num_rows_ kaldi::MatrixIndexT 10 stride_ kaldi::MatrixIndexT 2 > >>>>> > l_work KaldiBlasInt 5 p_work double * 0x741460 *p_work double > >>>>> > 1.0829823019173015e-312 temp void * 0x741460 N KaldiBlasInt 10 V > >>>>> > kaldi::MatrixBase<double> * 0x7fffffffacc0 data_ double * > >>>>> > 0x7406a0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1 > >>>>> > num_rows_ kaldi::MatrixIndexT 1 stride_ kaldi::MatrixIndexT 2 > >>>>> > result KaldiBlasInt -13 tmpU kaldi::Matrix<double> {...} > >>>>> > V_stride KaldiBlasInt 2 work_query double 5 u_job char * > >>>>> > 0x4d4fea "N" > >>>>> > tmpV kaldi::Matrix<double> {...} U_stride KaldiBlasInt 2 v_job > >>>>> > char * 0x4d4fea "N" > >>>>> > this kaldi::MatrixBase<double> * const 0x7fffffffaf80 data_ > >>>>> > double * 0x741300 *data_ double 0.78239572048187256 num_cols_ > >>>>> > kaldi::MatrixIndexT 1 num_rows_ kaldi::MatrixIndexT 10 stride_ > >>>>> > kaldi::MatrixIndexT 2 s kaldi::VectorBase<double> * > >>>>> > 0x7fffffffb190 data_ double * 0x7404a0 *data_ double 0 dim_ > >>>>> > kaldi::MatrixIndexT 1 U_in kaldi::MatrixBase<double> * 0x0 V_in > >>>>> > kaldi::MatrixBase<double> * 0x0 > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > On Wed, Jul 24, 2013 at 11:04 PM, Mailing list used for User > >>>>> > Communication and Updates <kal...@li...> > >>>>> > wrote: > >>>>> >> > >>>>> >> Could you please run this in a debugger and show me the values > >>>>> >> of p_work, l_work and result, e.g. > >>>>> >> > >>>>> >> gdb matrix-lib > >>>>> >> (gdb) r > >>>>> >> [wait till it crashes] > >>>>> >> (gdb) up > >>>>> >> (gdb) up # not sure how many times > >>>>> >> (gdb) up > >>>>> >> (gdb) p l_work > >>>>> >> (gdb) p p_work > >>>>> >> (gdb) p result > >>>>> >> > >>>>> >> Dan > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> On Wed, Jul 24, 2013 at 10:34 AM, Mailing list used for User > >>>>> >> Communication and Updates <kal...@li...> > wrote: > >>>>> >> > Hi, > >>>>> >> > > >>>>> >> > I tried latest trunk version with MKL. I have no problem when > >>>>> >> > running `make all`, but get a FAIL when testing. Can anyone > >>>>> >> > help? Thanks. > >>>>> >> > > >>>>> >> > > >>>>> >> > svn revision 2720 trunk > >>>>> >> > > >>>>> >> > > >>>>> >> > configuration line: > >>>>> >> > $ ./configure --threaded-math=yes --mathlib=MKL > >>>>> >> > --use-cuda=yes --mkl-root=$MKLROOT > >>>>> >> > --omp-libdir=/opt/intel/lib/intel64 > >>>>> >> > > >>>>> >> > > >>>>> >> > MKL version 11.0 update 5 (latest) > >>>>> >> > > >>>>> >> > > >>>>> >> > Linux version CentOS 6.4 > >>>>> >> > $ uname -a > >>>>> >> > Linux 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36 > >>>>> >> > UTC > >>>>> >> > 2013 > >>>>> >> > x86_64 > >>>>> >> > x86_64 x86_64 GNU/Linux > >>>>> >> > > >>>>> >> > > >>>>> >> > $ gcc -v > >>>>> >> > Using built-in specs. > >>>>> >> > Target: x86_64-redhat-linux > >>>>> >> > Configured with: ../configure --prefix=/usr > >>>>> >> > --mandir=/usr/share/man --infodir=/usr/share/info > >>>>> >> > --with-bugurl=http://bugzilla.redhat.com/bugzilla > >>>>> >> > --enable-bootstrap --enable-shared --enable-threads=posix > >>>>> >> > --enable-checking=release --with-system-zlib > >>>>> >> > --enable-__cxa_atexit --disable-libunwind-exceptions > >>>>> >> > --enable-gnu-unique-object > >>>>> >> > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada > >>>>> >> > --enable-java-awt=gtk > >>>>> >> > --disable-dssi > >>>>> >> > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre > >>>>> >> > --enable-libgcj-multifile --enable-java-maintainer-mode > >>>>> >> > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar > >>>>> >> > --disable-libjava-multilib > >>>>> >> > --with-ppl --with-cloog --with-tune=generic > >>>>> >> > --with-arch_32=i686 --build=x86_64-redhat-linux Thread model: > >>>>> >> > posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) > >>>>> >> > > >>>>> >> > > >>>>> >> > ~/kaldi/trunk/src/matrix > >>>>> >> > $ make test > >>>>> >> > Running matrix-lib-test .../bin/sh: line 1: 5561 Aborted > >>>>> >> > (core dumped) ./$x > /dev/null 2>&1 ... FAIL Running > >>>>> >> > kaldi-gpsr-test ...... SUCCESS > >>>>> >> > make: *** [test] Error 1 > >>>>> >> > > >>>>> >> > > >>>>> >> > $ ./matrix-lib-test > >>>>> >> > ... > >>>>> >> > ... > >>>>> >> > VLOG[2] (UnitTestLbfgs():matrix-lib-test.cc:2734) L-BFGS > >>>>> >> > finished after > >>>>> >> > 59 > >>>>> >> > function evaluations. > >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3623) M = > >>>>> >> > [ ] > >>>>> >> > > >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3624) M2 = > >>>>> >> > [ ] > >>>>> >> > > >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3626) > >>>>> >> > Compressed > >>>>> >> > matrix, > >>>>> >> > tot = 0, diff = 0 > >>>>> >> > > >>>>> >> > MKL ERROR: Parameter 13 was incorrect on entry to DGESVD. > >>>>> >> > KALDI_ASSERT: at LapackGesvd:kaldi-matrix.cc:401, failed: > >>>>> >> > result > >>>>> >> > >= 0 && > >>>>> >> > "Call to CLAPACK dgesvd_ called with wrong arguments" > >>>>> >> > Stack trace is: > >>>>> >> > kaldi::KaldiGetStackTrace() > >>>>> >> > kaldi::KaldiAssertFailure_(char const*, char const*, int, > >>>>> >> > char > >>>>> >> > const*) > >>>>> >> > kaldi::MatrixBase<double>::LapackGesvd(kaldi::VectorBase<doub > >>>>> >> > le>*, kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*) > >>>>> >> > > >>>>> >> > kaldi::MatrixBase<double>::DestructiveSvd(kaldi::VectorBase<d > >>>>> >> > ouble>*, kaldi::MatrixBase<double>*, > >>>>> >> > kaldi::MatrixBase<double>*) > >>>>> >> > kaldi::MatrixBase<double>::Svd(kaldi::VectorBase<double>*, > >>>>> >> > kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*) const > >>>>> >> > . > >>>>> >> > . > >>>>> >> > . > >>>>> >> > ./matrix-lib-test() [0x43055a] > >>>>> >> > ./matrix-lib-test() [0x42de8b] > >>>>> >> > ./matrix-lib-test(main+0x1d) [0x42dac7] > >>>>> >> > /lib64/libc.so.6(__libc_start_main+0xfd) [0x330901ecdd] > >>>>> >> > ./matrix-lib-test() [0x42ced9] Aborted (core dumped) > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > > >>>>> >> > ------------------------------------------------------------- > >>>>> >> > ----------------- See everything from the browser to the > >>>>> >> > database with AppDynamics Get end-to-end visibility with > >>>>> >> > application monitoring from AppDynamics Isolate bottlenecks > >>>>> >> > and diagnose root cause in seconds. > >>>>> >> > Start your free trial of AppDynamics Pro today! > >>>>> >> > > >>>>> >> > > >>>>> >> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/41 > >>>>> >> > 40/ostg.clktrk > >>>>> >> > _______________________________________________ > >>>>> >> > Kaldi-users mailing list > >>>>> >> > Kal...@li... > >>>>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users > >>>>> >> > > >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> --------------------------------------------------------------- > >>>>> >> --------------- See everything from the browser to the database > >>>>> >> with AppDynamics Get end-to-end visibility with application > >>>>> >> monitoring from AppDynamics Isolate bottlenecks and diagnose > >>>>> >> root cause in seconds. > >>>>> >> Start your free trial of AppDynamics Pro today! > >>>>> >> > >>>>> >> > >>>>> >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140 > >>>>> >> /ostg.clktrk _______________________________________________ > >>>>> >> Kaldi-users mailing list > >>>>> >> Kal...@li... > >>>>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > > >>>>> > ---------------------------------------------------------------- > >>>>> > -------------- See everything from the browser to the database > >>>>> > with AppDynamics Get end-to-end visibility with application > >>>>> > monitoring from AppDynamics Isolate bottlenecks and diagnose > >>>>> > root cause in seconds. > >>>>> > Start your free trial of AppDynamics Pro today! > >>>>> > > >>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ > >>>>> > ostg.clktrk _______________________________________________ > >>>>> > Kaldi-users mailing list > >>>>> > Kal...@li... > >>>>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users > >>>>> > > >>>>> > >>>>> > >>>>> ------------------------------------------------------------------ > >>>>> ------------ See everything from the browser to the database with > >>>>> AppDynamics Get end-to-end visibility with application monitoring > >>>>> from AppDynamics Isolate bottlenecks and diagnose root cause in > >>>>> seconds. > >>>>> Start your free trial of AppDynamics Pro today! > >>>>> > >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/os > >>>>> tg.clktrk _______________________________________________ > >>>>> Kaldi-users mailing list > >>>>> Kal...@li... > >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users > >>>> > >>>> > >>>> > >>>> > >>>> ------------------------------------------------------------------- > >>>> ----------- See everything from the browser to the database with > >>>> AppDynamics Get end-to-end visibility with application monitoring > >>>> from AppDynamics Isolate bottlenecks and diagnose root cause in > >>>> seconds. > >>>> Start your free trial of AppDynamics Pro today! > >>>> > >>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ost > >>>> g.clktrk _______________________________________________ > >>>> Kaldi-users mailing list > >>>> Kal...@li... > >>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users > >>>> > >>> > >>> > >>> > >>> > >>> > >>> -------------------------------------------------------------------- > >>> ---------- See everything from the browser to the database with > >>> AppDynamics Get end-to-end visibility with application monitoring > >>> from AppDynamics Isolate bottlenecks and diagnose root cause in > >>> seconds. > >>> Start your free trial of AppDynamics Pro today! > >>> > >>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg > >>> .clktrk _______________________________________________ > >>> Kaldi-users mailing list > >>> Kal...@li... > >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users > >>> > >> > >> > >> > >> --------------------------------------------------------------------- > >> --------- See everything from the browser to the database with > >> AppDynamics Get end-to-end visibility with application monitoring > >> from AppDynamics Isolate bottlenecks and diagnose root cause in > >> seconds. > >> Start your free trial of AppDynamics Pro today! > >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg. > >> clktrk _______________________________________________ > >> Kaldi-users mailing list > >> Kal...@li... > >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > >> > > > > > > > > ---------------------------------------------------------------------- > > -------- See everything from the browser to the database with > > AppDynamics Get end-to-end visibility with application monitoring from > > AppDynamics Isolate bottlenecks and diagnose root cause in seconds. > > Start your free trial of AppDynamics Pro today! > > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.c > > lktrk _______________________________________________ > > Kaldi-users mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > ------------------------------------------------------------------------------ > Get your SQL database under version control now! > Version control is standard for application code, but databases havent > caught up. So what steps can you take to put your SQL databases under > version control? Why should you start doing it? Read more to find out. > http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > ------------------------------------------------------------------------------ > Get your SQL database under version control now! > Version control is standard for application code, but databases havent > caught up. So what steps can you take to put your SQL databases under > version control? Why should you start doing it? Read more to find out. > http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
