Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

I just checked different versions of MKL and indeed, this was introduced in
the  update 5. Anyone with the commercial MKL support cares to submit a
ticket to Intel?
y.

On Wed, Jul 24, 2013 at 12:42 PM, Mailing list used for User Communication
and Updates <kal...@li...> wrote:

> You are right.... I just downgraded to MKL 11.0update1 and passed the test.
>
>
>
> On Thu, Jul 25, 2013 at 12:28 AM, Mailing list used for User Communication
> and Updates <kal...@li...> wrote:
>
>> I saw some changes regarding improving performance of *GESVD in the MKL
>> 11.0 update5  changelog. I wonder if they messed something up? How about
>> trying slightly "older" version of the MKL? If you are on the subscription
>> of Intel Software Network, you can download older versions of the MKL at no
>> charge...
>>
>> We are using Intel 11.0 and the tests pass without any problem...
>>
>> yenda
>>
>>
>>
>> On Wed, Jul 24, 2013 at 12:14 PM, Mailing list used for User
>> Communication and Updates <kal...@li...> wrote:
>>
>>> I can't see any problems in the Kaldi code.  Perhaps this is an older
>>> version of MKL?  It could be a bug in MKL itself.  MKL tests OK for
>>> me.
>>> Dan
>>>
>>>
>>> On Wed, Jul 24, 2013 at 11:55 AM, Mailing list used for User
>>> Communication and Updates <kal...@li...> wrote:
>>> > here's the dbg info copied from eclipse.
>>> > PS: I can pass the test with atlas provided by centos repo
>>> > atlas-sse3.x86_64                          3.8.4-2.el6
>>> > @base
>>> > atlas-sse3-devel.x86_64                    3.8.4-2.el6
>>> > @base
>>> >
>>> >
>>> >
>>> > stack:
>>> >
>>> > matrix-lib-test [C/C++ Application]
>>> > matrix-lib-test [11312] [cores: 1]
>>> > Thread [1] 11312 [core: 1] (Suspended : Signal : SIGABRT:Aborted)
>>> > raise() at 0x33090328a5
>>> > abort() at 0x3309034085
>>> > kaldi::KaldiAssertFailure_() at kaldi-error.cc:136 0x4cf401
>>> > kaldi::MatrixBase<double>::LapackGesvd() at kaldi-matrix.cc:401
>>> 0x48fdd3
>>> > kaldi::MatrixBase<double>::DestructiveSvd() at kaldi-matrix.cc:1,330
>>> > 0x48b902
>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.cc:1,352 0x48b9bb
>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.h:315 0x468e50
>>> > kaldi::MatrixBase<double>::Cond() at kaldi-matrix.cc:1,222 0x48bf47
>>> > kaldi::InitRand<double>() at matrix-lib-test.cc:74 0x42dd7e
>>> > kaldi::UnitTestCompressedMatrix<double>() at matrix-lib-test.cc:3,532
>>> > 0x43055a
>>> > kaldi::MatrixUnitTest<double>() at matrix-lib-test.cc:3,705 0x42de8b
>>> > main() at matrix-lib-test.cc:3,829 0x42dac7
>>> >
>>> >
>>> >
>>> > variable list after invoking the following    @line393 kaldi-matrix.cc
>>> >   clapack_Xgesvd(v_job, u_job,
>>> >                  &M, &N, data_, &LDA,
>>> >                  s->Data(),
>>> >                  V->Data(), &V_stride,
>>> >                  U->Data(), &U_stride,
>>> >                  p_work, &l_work,
>>> >                  &result);
>>> >
>>> >
>>> > __func__ char [12] 0x4d5418
>>> > M KaldiBlasInt 1
>>> > LDA KaldiBlasInt 2
>>> > U kaldi::MatrixBase<double> * 0x7ffffffface0
>>> > data_ double * 0x7413b0
>>> > *data_ double 0
>>> > num_cols_ kaldi::MatrixIndexT 1
>>> > num_rows_ kaldi::MatrixIndexT 10
>>> > stride_ kaldi::MatrixIndexT 2
>>> > l_work KaldiBlasInt 5
>>> > p_work double * 0x741460
>>> > *p_work double 1.0829823019173015e-312
>>> > temp void * 0x741460
>>> > N KaldiBlasInt 10
>>> > V kaldi::MatrixBase<double> * 0x7fffffffacc0
>>> > data_ double * 0x7406a0
>>> > *data_ double 0
>>> > num_cols_ kaldi::MatrixIndexT 1
>>> > num_rows_ kaldi::MatrixIndexT 1
>>> > stride_ kaldi::MatrixIndexT 2
>>> > result KaldiBlasInt -13
>>> > tmpU kaldi::Matrix<double> {...}
>>> > V_stride KaldiBlasInt 2
>>> > work_query double 5
>>> > u_job char * 0x4d4fea "N"
>>> > tmpV kaldi::Matrix<double> {...}
>>> > U_stride KaldiBlasInt 2
>>> > v_job char * 0x4d4fea "N"
>>> > this kaldi::MatrixBase<double> * const 0x7fffffffaf80
>>> > data_ double * 0x741300
>>> > *data_ double 0.78239572048187256
>>> > num_cols_ kaldi::MatrixIndexT 1
>>> > num_rows_ kaldi::MatrixIndexT 10
>>> > stride_ kaldi::MatrixIndexT 2
>>> > s kaldi::VectorBase<double> * 0x7fffffffb190
>>> > data_ double * 0x7404a0
>>> > *data_ double 0
>>> > dim_ kaldi::MatrixIndexT 1
>>> > U_in kaldi::MatrixBase<double> * 0x0
>>> > V_in kaldi::MatrixBase<double> * 0x0
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Jul 24, 2013 at 11:04 PM, Mailing list used for User
>>> Communication
>>> > and Updates <kal...@li...> wrote:
>>> >>
>>> >> Could you please run this in a debugger and show me the values of
>>> >> p_work, l_work and result, e.g.
>>> >>
>>> >> gdb matrix-lib
>>> >> (gdb) r
>>> >> [wait till it crashes]
>>> >> (gdb) up
>>> >> (gdb) up    # not sure how many times
>>> >> (gdb) up
>>> >> (gdb) p l_work
>>> >> (gdb) p p_work
>>> >> (gdb) p result
>>> >>
>>> >> Dan
>>> >>
>>> >>
>>> >>
>>> >> On Wed, Jul 24, 2013 at 10:34 AM, Mailing list used for User
>>> >> Communication and Updates <kal...@li...> wrote:
>>> >> > Hi,
>>> >> >
>>> >> > I tried latest trunk version with MKL. I have no problem when
>>> running
>>> >> > `make
>>> >> > all`, but get a FAIL when testing. Can anyone help? Thanks.
>>> >> >
>>> >> >
>>> >> > svn revision 2720 trunk
>>> >> >
>>> >> >
>>> >> > configuration line:
>>> >> > $ ./configure --threaded-math=yes --mathlib=MKL --use-cuda=yes
>>> >> > --mkl-root=$MKLROOT --omp-libdir=/opt/intel/lib/intel64
>>> >> >
>>> >> >
>>> >> > MKL version 11.0 update 5 (latest)
>>> >> >
>>> >> >
>>> >> > Linux version CentOS 6.4
>>> >> > $ uname -a
>>> >> > Linux 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36 UTC 2013
>>> >> > x86_64
>>> >> > x86_64 x86_64 GNU/Linux
>>> >> >
>>> >> >
>>> >> > $ gcc -v
>>> >> > Using built-in specs.
>>> >> > Target: x86_64-redhat-linux
>>> >> > Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
>>> >> > --infodir=/usr/share/info
>>> >> > --with-bugurl=http://bugzilla.redhat.com/bugzilla
>>> >> > --enable-bootstrap --enable-shared --enable-threads=posix
>>> >> > --enable-checking=release --with-system-zlib --enable-__cxa_atexit
>>> >> > --disable-libunwind-exceptions --enable-gnu-unique-object
>>> >> > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada
>>> >> > --enable-java-awt=gtk
>>> >> > --disable-dssi
>>> --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre
>>> >> > --enable-libgcj-multifile --enable-java-maintainer-mode
>>> >> > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar
>>> >> > --disable-libjava-multilib
>>> >> > --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686
>>> >> > --build=x86_64-redhat-linux
>>> >> > Thread model: posix
>>> >> > gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
>>> >> >
>>> >> >
>>> >> > ~/kaldi/trunk/src/matrix
>>> >> > $ make test
>>> >> > Running matrix-lib-test .../bin/sh: line 1:  5561 Aborted
>>> >> > (core dumped) ./$x > /dev/null 2>&1
>>> >> > ... FAIL
>>> >> > Running kaldi-gpsr-test ...... SUCCESS
>>> >> > make: *** [test] Error 1
>>> >> >
>>> >> >
>>> >> > $ ./matrix-lib-test
>>> >> > ...
>>> >> > ...
>>> >> > VLOG[2] (UnitTestLbfgs():matrix-lib-test.cc:2734) L-BFGS finished
>>> after
>>> >> > 59
>>> >> > function evaluations.
>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3623) M =  [ ]
>>> >> >
>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3624) M2 =  [ ]
>>> >> >
>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3626) Compressed
>>> >> > matrix,
>>> >> > tot = 0, diff = 0
>>> >> >
>>> >> > MKL ERROR: Parameter 13 was incorrect on entry to DGESVD.
>>> >> > KALDI_ASSERT: at LapackGesvd:kaldi-matrix.cc:401, failed: result >=
>>> 0 &&
>>> >> > "Call to CLAPACK dgesvd_ called with wrong arguments"
>>> >> > Stack trace is:
>>> >> > kaldi::KaldiGetStackTrace()
>>> >> > kaldi::KaldiAssertFailure_(char const*, char const*, int, char
>>> const*)
>>> >> > kaldi::MatrixBase<double>::LapackGesvd(kaldi::VectorBase<double>*,
>>> >> > kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*)
>>> >> >
>>> kaldi::MatrixBase<double>::DestructiveSvd(kaldi::VectorBase<double>*,
>>> >> > kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*)
>>> >> > kaldi::MatrixBase<double>::Svd(kaldi::VectorBase<double>*,
>>> >> > kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*) const
>>> >> > .
>>> >> > .
>>> >> > .
>>> >> > ./matrix-lib-test() [0x43055a]
>>> >> > ./matrix-lib-test() [0x42de8b]
>>> >> > ./matrix-lib-test(main+0x1d) [0x42dac7]
>>> >> > /lib64/libc.so.6(__libc_start_main+0xfd) [0x330901ecdd]
>>> >> > ./matrix-lib-test() [0x42ced9]
>>> >> > Aborted (core dumped)
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> ------------------------------------------------------------------------------
>>> >> > See everything from the browser to the database with AppDynamics
>>> >> > Get end-to-end visibility with application monitoring from
>>> AppDynamics
>>> >> > Isolate bottlenecks and diagnose root cause in seconds.
>>> >> > Start your free trial of AppDynamics Pro today!
>>> >> >
>>> >> >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>>> >> > _______________________________________________
>>> >> > Kaldi-users mailing list
>>> >> > Kal...@li...
>>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>> >> >
>>> >>
>>> >>
>>> >>
>>> ------------------------------------------------------------------------------
>>> >> See everything from the browser to the database with AppDynamics
>>> >> Get end-to-end visibility with application monitoring from AppDynamics
>>> >> Isolate bottlenecks and diagnose root cause in seconds.
>>> >> Start your free trial of AppDynamics Pro today!
>>> >>
>>> >>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>>> >> _______________________________________________
>>> >> Kaldi-users mailing list
>>> >> Kal...@li...
>>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>> >
>>> >
>>> >
>>> >
>>> >
>>> ------------------------------------------------------------------------------
>>> > See everything from the browser to the database with AppDynamics
>>> > Get end-to-end visibility with application monitoring from AppDynamics
>>> > Isolate bottlenecks and diagnose root cause in seconds.
>>> > Start your free trial of AppDynamics Pro today!
>>> >
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>>> > _______________________________________________
>>> > Kaldi-users mailing list
>>> > Kal...@li...
>>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>> >
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> See everything from the browser to the database with AppDynamics
>>> Get end-to-end visibility with application monitoring from AppDynamics
>>> Isolate bottlenecks and diagnose root cause in seconds.
>>> Start your free trial of AppDynamics Pro today!
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> Kaldi-users mailing list
>>> Kal...@li...
>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> See everything from the browser to the database with AppDynamics
>> Get end-to-end visibility with application monitoring from AppDynamics
>> Isolate bottlenecks and diagnose root cause in seconds.
>> Start your free trial of AppDynamics Pro today!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>
>>
>
>
>
>
> ------------------------------------------------------------------------------
> See everything from the browser to the database with AppDynamics
> Get end-to-end visibility with application monitoring from AppDynamics
> Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>
>