kaldi-users — Kaldi Users

Re: [Kaldi-users] DNN training on CPUs

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

num_jobs_nnet should be the same as the #machine (i.e. 4), but you may
want to decrease the learning rate a bit (e.g. by a factor of 2) if
you reducde the #machines from 16 to 4.
Dan


On Tue, Aug 20, 2013 at 9:21 AM, Mailing list used for User
Communication and Updates <kal...@li...> wrote:
> Hi All,
>
> I am planning to run the wsj/s5 set up for DNN training on 4 machines and
> each has 16 cores. Could anyone give me some pointers about how to change
> important parameters like num_jobs_nnet in the script for my setup?
>
> Thank you,
>
> Best Regards,
> Lahiru
>
> ------------------------------------------------------------------------------
> Introducing Performance Central, a new site from SourceForge and
> AppDynamics. Performance Central is your source for news, insights,
> analysis and resources for efficient Application Performance Management.
> Visit us today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>

[Kaldi-users] DNN training on CPUs

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Hi All,

I am planning to run the *wsj/s5* set up for DNN training on 4 machines and
each has 16 cores. Could anyone give me some pointers about how to change
important parameters like *num_jobs_nnet* in the script for my setup?

Thank you,

Best Regards,
Lahiru

Re: [Kaldi-users] Monophone DNN traning on TIMIT

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

I think tree-info may already exist.
Dan


On Thu, Aug 15, 2013 at 2:00 PM, Arnab Ghoshal <ar...@gm...> wrote:
> This is because all discriminative training setups (not just for DNNs)
> assume that they are using triphone models. The easiest way for you to
> fix it will be to add '--mono' option to where utils/mkgraph.sh is
> being called in make_denlats_nnet.sh
>
> As far as I know, there is currently no easy way to tell what kind of
> context-dependency the models use. It is assumed that the user knows
> what they are doing. For example, the setups won't work for quinphone
> models either without making the necessary modifications to the mkgrph
> options. Maybe one can write something like 'tree-info' and use that
> to automatically set the mkgraph options.
>
> -Arnab
>
> On Thu, Aug 15, 2013 at 10:17 AM, Mailing list used for User
> Communication and Updates <kal...@li...> wrote:
>> Hi All,
>>
>> I am trying to train a DNN for TIMIT as it is much smaller to play with. So,
>> I modified the run_dnn.sh file given in the wsj/s5 setup. However, while
>> trying to perform sMBR, following error occurred. Your advice is highly
>> appreciated.
>>
>> Thanks,
>> Lahiru
>>
>> steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
>> conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
>> exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
>> steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
>> conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
>> exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
>> Making unigram grammar FST in exp/mono_pretrain-dbn_dnn_denlats_train/lang
>> Compiling decoding graph in exp/mono_pretrain-dbn_dnn_denlats_train/dengraph
>> fstminimizeencoded
>> fsttablecompose exp/mono_pretrain-dbn_dnn_denlats_train/lang/L_disambig.fst
>> exp/mono_pretrain-dbn_dnn_denlats_train/lang/G.fst
>> fstdeterminizestar --use-log=true
>> fstisstochastic exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/LG.fst
>> 1.2886e-05 -0.000277955
>> fstcomposecontext --context-size=3 --central-position=1
>> --read-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/phones/disambig.int
>> --write-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/disambig_ilabels_3_1.int
>> exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
>> fstisstochastic exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/CLG_3_1.fst
>> 1.25878e-05 -0.000277955
>> make-h-transducer
>> --disambig-syms-out=exp/mono_pretrain-dbn_dnn_denlats_train/dengraph/disambig_tid.int
>> --transition-scale=1.0
>> exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
>> exp/mono_pretrain-dbn_dnn/tree exp/mono_pretrain-dbn_dnn/final.mdl
>> ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
>> mismatch, ilabel-info [from context FST is 3, context-dependency object
>> expects 1
>> ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
>> mismatch, ilabel-info [from context FST is 3, context-dependency object
>> expects 1
>>
>> [stack trace: ]
>> kaldi::KaldiGetStackTrace()
>> kaldi::KaldiErrorMessage::~KaldiErrorMessage()
>> kaldi::GetHmmAsFst(std::vector<int, std::allocator<int> >,
>> kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
>> kaldi::HTransducerConfig const&, std::tr1::unordered_map<std::pair<int,
>> std::vector<int, std::allocator<int> > >,
>> fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*,
>> kaldi::HmmCacheHash, std::equal_to<std::pair<int, std::vector<int,
>> std::allocator<int> > > >, std::allocator<std::pair<std::pair<int,
>> std::vector<int, std::allocator<int> > > const,
>> fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*> >, false>*)
>> kaldi::GetHTransducer(std::vector<std::vector<int, std::allocator<int> >,
>> std::allocator<std::vector<int, std::allocator<int> > > > const&,
>> kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
>> kaldi::HTransducerConfig const&, std::vector<int, std::allocator<int> >*)
>> make-h-transducer(main+0x4b5) [0x59bc31]
>> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3bafe1d994]
>> std::basic_filebuf<char, std::char_traits<char> >::xsputn(char const*, long)
>>
>>
>> ------------------------------------------------------------------------------
>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>> It's a free troubleshooting tool designed for production.
>> Get down to code-level detail for bottlenecks, with <2% overhead.
>> Download for free and get started troubleshooting in minutes.
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>

Re: [Kaldi-users] Monophone DNN traning on TIMIT

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Thanks for the info guys.


On Thu, Aug 15, 2013 at 8:00 PM, Mailing list used for User Communication
and Updates <kal...@li...> wrote:

> This is because all discriminative training setups (not just for DNNs)
> assume that they are using triphone models. The easiest way for you to
> fix it will be to add '--mono' option to where utils/mkgraph.sh is
> being called in make_denlats_nnet.sh
>
> As far as I know, there is currently no easy way to tell what kind of
> context-dependency the models use. It is assumed that the user knows
> what they are doing. For example, the setups won't work for quinphone
> models either without making the necessary modifications to the mkgrph
> options. Maybe one can write something like 'tree-info' and use that
> to automatically set the mkgraph options.
>
> -Arnab
>
> On Thu, Aug 15, 2013 at 10:17 AM, Mailing list used for User
> Communication and Updates <kal...@li...> wrote:
> > Hi All,
> >
> > I am trying to train a DNN for TIMIT as it is much smaller to play with.
> So,
> > I modified the run_dnn.sh file given in the wsj/s5 setup. However, while
> > trying to perform sMBR, following error occurred. Your advice is highly
> > appreciated.
> >
> > Thanks,
> > Lahiru
> >
> > steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
> > conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
> > exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
> > steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
> > conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
> > exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
> > Making unigram grammar FST in
> exp/mono_pretrain-dbn_dnn_denlats_train/lang
> > Compiling decoding graph in
> exp/mono_pretrain-dbn_dnn_denlats_train/dengraph
> > fstminimizeencoded
> > fsttablecompose
> exp/mono_pretrain-dbn_dnn_denlats_train/lang/L_disambig.fst
> > exp/mono_pretrain-dbn_dnn_denlats_train/lang/G.fst
> > fstdeterminizestar --use-log=true
> > fstisstochastic exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/LG.fst
> > 1.2886e-05 -0.000277955
> > fstcomposecontext --context-size=3 --central-position=1
> > --read-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/phones/
> disambig.int
> > --write-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/
> disambig_ilabels_3_1.int
> > exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
> > fstisstochastic
> exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/CLG_3_1.fst
> > 1.25878e-05 -0.000277955
> > make-h-transducer
> > --disambig-syms-out=exp/mono_pretrain-dbn_dnn_denlats_train/dengraph/
> disambig_tid.int
> > --transition-scale=1.0
> > exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
> > exp/mono_pretrain-dbn_dnn/tree exp/mono_pretrain-dbn_dnn/final.mdl
> > ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
> > mismatch, ilabel-info [from context FST is 3, context-dependency object
> > expects 1
> > ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
> > mismatch, ilabel-info [from context FST is 3, context-dependency object
> > expects 1
> >
> > [stack trace: ]
> > kaldi::KaldiGetStackTrace()
> > kaldi::KaldiErrorMessage::~KaldiErrorMessage()
> > kaldi::GetHmmAsFst(std::vector<int, std::allocator<int> >,
> > kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
> > kaldi::HTransducerConfig const&, std::tr1::unordered_map<std::pair<int,
> > std::vector<int, std::allocator<int> > >,
> > fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*,
> > kaldi::HmmCacheHash, std::equal_to<std::pair<int, std::vector<int,
> > std::allocator<int> > > >, std::allocator<std::pair<std::pair<int,
> > std::vector<int, std::allocator<int> > > const,
> > fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*> >,
> false>*)
> > kaldi::GetHTransducer(std::vector<std::vector<int, std::allocator<int> >,
> > std::allocator<std::vector<int, std::allocator<int> > > > const&,
> > kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
> > kaldi::HTransducerConfig const&, std::vector<int, std::allocator<int> >*)
> > make-h-transducer(main+0x4b5) [0x59bc31]
> > /lib64/libc.so.6(__libc_start_main+0xf4) [0x3bafe1d994]
> > std::basic_filebuf<char, std::char_traits<char> >::xsputn(char const*,
> long)
> >
> >
> >
> ------------------------------------------------------------------------------
> > Get 100% visibility into Java/.NET code with AppDynamics Lite!
> > It's a free troubleshooting tool designed for production.
> > Get down to code-level detail for bottlenecks, with <2% overhead.
> > Download for free and get started troubleshooting in minutes.
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Kaldi-users mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>

Re: [Kaldi-users] Monophone DNN traning on TIMIT

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

This is because all discriminative training setups (not just for DNNs)
assume that they are using triphone models. The easiest way for you to
fix it will be to add '--mono' option to where utils/mkgraph.sh is
being called in make_denlats_nnet.sh

As far as I know, there is currently no easy way to tell what kind of
context-dependency the models use. It is assumed that the user knows
what they are doing. For example, the setups won't work for quinphone
models either without making the necessary modifications to the mkgrph
options. Maybe one can write something like 'tree-info' and use that
to automatically set the mkgraph options.

-Arnab

On Thu, Aug 15, 2013 at 10:17 AM, Mailing list used for User
Communication and Updates <kal...@li...> wrote:
> Hi All,
>
> I am trying to train a DNN for TIMIT as it is much smaller to play with. So,
> I modified the run_dnn.sh file given in the wsj/s5 setup. However, while
> trying to perform sMBR, following error occurred. Your advice is highly
> appreciated.
>
> Thanks,
> Lahiru
>
> steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
> conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
> exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
> steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
> conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
> exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
> Making unigram grammar FST in exp/mono_pretrain-dbn_dnn_denlats_train/lang
> Compiling decoding graph in exp/mono_pretrain-dbn_dnn_denlats_train/dengraph
> fstminimizeencoded
> fsttablecompose exp/mono_pretrain-dbn_dnn_denlats_train/lang/L_disambig.fst
> exp/mono_pretrain-dbn_dnn_denlats_train/lang/G.fst
> fstdeterminizestar --use-log=true
> fstisstochastic exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/LG.fst
> 1.2886e-05 -0.000277955
> fstcomposecontext --context-size=3 --central-position=1
> --read-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/phones/disambig.int
> --write-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/disambig_ilabels_3_1.int
> exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
> fstisstochastic exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/CLG_3_1.fst
> 1.25878e-05 -0.000277955
> make-h-transducer
> --disambig-syms-out=exp/mono_pretrain-dbn_dnn_denlats_train/dengraph/disambig_tid.int
> --transition-scale=1.0
> exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
> exp/mono_pretrain-dbn_dnn/tree exp/mono_pretrain-dbn_dnn/final.mdl
> ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
> mismatch, ilabel-info [from context FST is 3, context-dependency object
> expects 1
> ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
> mismatch, ilabel-info [from context FST is 3, context-dependency object
> expects 1
>
> [stack trace: ]
> kaldi::KaldiGetStackTrace()
> kaldi::KaldiErrorMessage::~KaldiErrorMessage()
> kaldi::GetHmmAsFst(std::vector<int, std::allocator<int> >,
> kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
> kaldi::HTransducerConfig const&, std::tr1::unordered_map<std::pair<int,
> std::vector<int, std::allocator<int> > >,
> fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*,
> kaldi::HmmCacheHash, std::equal_to<std::pair<int, std::vector<int,
> std::allocator<int> > > >, std::allocator<std::pair<std::pair<int,
> std::vector<int, std::allocator<int> > > const,
> fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*> >, false>*)
> kaldi::GetHTransducer(std::vector<std::vector<int, std::allocator<int> >,
> std::allocator<std::vector<int, std::allocator<int> > > > const&,
> kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
> kaldi::HTransducerConfig const&, std::vector<int, std::allocator<int> >*)
> make-h-transducer(main+0x4b5) [0x59bc31]
> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3bafe1d994]
> std::basic_filebuf<char, std::char_traits<char> >::xsputn(char const*, long)
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>

Re: [Kaldi-users] Monophone DNN traning on TIMIT

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Probably the options
--context-size=3 --central-position=1
should be changed to
--context-size=1 --central-position=0
[but ideally the script would automatically do the right thing].
Dan

On Thu, Aug 15, 2013 at 11:17 AM, Mailing list used for User
Communication and Updates <kal...@li...> wrote:
> Hi All,
>
> I am trying to train a DNN for TIMIT as it is much smaller to play with. So,
> I modified the run_dnn.sh file given in the wsj/s5 setup. However, while
> trying to perform sMBR, following error occurred. Your advice is highly
> appreciated.
>
> Thanks,
> Lahiru
>
> steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
> conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
> exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
> steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
> conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
> exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
> Making unigram grammar FST in exp/mono_pretrain-dbn_dnn_denlats_train/lang
> Compiling decoding graph in exp/mono_pretrain-dbn_dnn_denlats_train/dengraph
> fstminimizeencoded
> fsttablecompose exp/mono_pretrain-dbn_dnn_denlats_train/lang/L_disambig.fst
> exp/mono_pretrain-dbn_dnn_denlats_train/lang/G.fst
> fstdeterminizestar --use-log=true
> fstisstochastic exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/LG.fst
> 1.2886e-05 -0.000277955
> fstcomposecontext --context-size=3 --central-position=1
> --read-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/phones/disambig.int
> --write-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/disambig_ilabels_3_1.int
> exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
> fstisstochastic exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/CLG_3_1.fst
> 1.25878e-05 -0.000277955
> make-h-transducer
> --disambig-syms-out=exp/mono_pretrain-dbn_dnn_denlats_train/dengraph/disambig_tid.int
> --transition-scale=1.0
> exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
> exp/mono_pretrain-dbn_dnn/tree exp/mono_pretrain-dbn_dnn/final.mdl
> ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
> mismatch, ilabel-info [from context FST is 3, context-dependency object
> expects 1
> ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
> mismatch, ilabel-info [from context FST is 3, context-dependency object
> expects 1
>
> [stack trace: ]
> kaldi::KaldiGetStackTrace()
> kaldi::KaldiErrorMessage::~KaldiErrorMessage()
> kaldi::GetHmmAsFst(std::vector<int, std::allocator<int> >,
> kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
> kaldi::HTransducerConfig const&, std::tr1::unordered_map<std::pair<int,
> std::vector<int, std::allocator<int> > >,
> fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*,
> kaldi::HmmCacheHash, std::equal_to<std::pair<int, std::vector<int,
> std::allocator<int> > > >, std::allocator<std::pair<std::pair<int,
> std::vector<int, std::allocator<int> > > const,
> fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*> >, false>*)
> kaldi::GetHTransducer(std::vector<std::vector<int, std::allocator<int> >,
> std::allocator<std::vector<int, std::allocator<int> > > > const&,
> kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
> kaldi::HTransducerConfig const&, std::vector<int, std::allocator<int> >*)
> make-h-transducer(main+0x4b5) [0x59bc31]
> /lib64/libc.so.6(__libc_start_main+0xf4) [0x3bafe1d994]
> std::basic_filebuf<char, std::char_traits<char> >::xsputn(char const*, long)
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>

[Kaldi-users] Monophone DNN traning on TIMIT

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Hi All,

I am trying to train a DNN for TIMIT as it is much smaller to play with.
So, I modified the run_dnn.sh file given in the wsj/s5 setup. However,
while trying to perform sMBR, following error occurred. Your advice is
highly appreciated.

Thanks,
Lahiru

*steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train*
steps/make_denlats_nnet.sh --nj 100 --cmd run.pl --config
conf/decode_dnn.config --acwt 0.10 data-fmllr-mono/train data/lang
exp/mono_pretrain-dbn_dnn exp/mono_pretrain-dbn_dnn_denlats_train
Making unigram grammar FST in exp/mono_pretrain-dbn_dnn_denlats_train/lang
Compiling decoding graph in exp/mono_pretrain-dbn_dnn_denlats_train/dengraph
fstminimizeencoded
fsttablecompose exp/mono_pretrain-dbn_dnn_denlats_train/lang/L_disambig.fst
exp/mono_pretrain-dbn_dnn_denlats_train/lang/G.fst
fstdeterminizestar --use-log=true
fstisstochastic exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/LG.fst
1.2886e-05 -0.000277955
fstcomposecontext --context-size=3 --central-position=1
--read-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/phones/
disambig.int--write-disambig-syms=exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/
disambig_ilabels_3_1.intexp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
fstisstochastic
exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/CLG_3_1.fst
1.25878e-05 -0.000277955
make-h-transducer
--disambig-syms-out=exp/mono_pretrain-dbn_dnn_denlats_train/dengraph/
disambig_tid.int --transition-scale=1.0
exp/mono_pretrain-dbn_dnn_denlats_train/lang/tmp/ilabels_3_1
exp/mono_pretrain-dbn_dnn/tree exp/mono_pretrain-dbn_dnn/final.mdl
*ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
mismatch, ilabel-info [from context FST is 3, context-dependency object
expects 1
ERROR (make-h-transducer:GetHmmAsFst():hmm-utils.cc:41) Context size
mismatch, ilabel-info [from context FST is 3, context-dependency object
expects 1*

[stack trace: ]
kaldi::KaldiGetStackTrace()
kaldi::KaldiErrorMessage::~KaldiErrorMessage()
kaldi::GetHmmAsFst(std::vector<int, std::allocator<int> >,
kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
kaldi::HTransducerConfig const&, std::tr1::unordered_map<std::pair<int,
std::vector<int, std::allocator<int> > >,
fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*,
kaldi::HmmCacheHash, std::equal_to<std::pair<int, std::vector<int,
std::allocator<int> > > >, std::allocator<std::pair<std::pair<int,
std::vector<int, std::allocator<int> > > const,
fst::VectorFst<fst::ArcTpl<fst::TropicalWeightTpl<float> > >*> >, false>*)
kaldi::GetHTransducer(std::vector<std::vector<int, std::allocator<int> >,
std::allocator<std::vector<int, std::allocator<int> > > > const&,
kaldi::ContextDependencyInterface const&, kaldi::TransitionModel const&,
kaldi::HTransducerConfig const&, std::vector<int, std::allocator<int> >*)
make-h-transducer(main+0x4b5) [0x59bc31]
/lib64/libc.so.6(__libc_start_main+0xf4) [0x3bafe1d994]
std::basic_filebuf<char, std::char_traits<char> >::xsputn(char const*, long)

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

>From what you described, something seems to be not matching up with
something else.  I am thinking that both of these somethings perhaps
derive from Kaldi lattices and maybe one of them had had
lattice-align-words run on it, and one had not, and that might be
responsible for the mismatch.
Dan


On Thu, Aug 1, 2013 at 7:42 PM, Nathan Dunn <nd...@uo...> wrote:
>
> On Aug 1, 2013, at 2:04 PM, Daniel Povey wrote:
>
>> Your stuff based on lattice-align-words seemed like it could give the
>> right answer.  The mismatch you described is what I would expect and
>> is not a problem, it's because you didn't run lattice-align-words in
>> the baseline.
>
> I'm trying to understand what you mean by "run lattice-align-words in the baseline".   Are you saying that I should align the lattices prior to running them?
>
>
> in wsj/s5/steps/word_align_lattices.sh , I see:
>
> $cmd JOB=1:$nj $outdir/log/align.JOB.log \
>   lattice-align-words --silence-label=$silence_label --test=true \
>    $wbfile $mdl "ark:gunzip -c $indir/lat.JOB.gz|" "ark,t:|gzip -c >$outdir/lat.JOB.gz" || exit 1;
>
> I'm assuming this aligns the lattice that can later be used below:
>
>> lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>
>
> Am I on the right track, or is there a better place for you to point me.
>
>>  I don't know exactly what the problem is-- perhaps the
>> times are too far from the human-generated reference?
>
> We can look into that.
>
>> In future if you ask questions about this, please paste the output of
>> the programs concerned and the corresponding command lines.
>
> Sorry, I'll do that in the future.
>
> Thanks,
>
> Nathan
>
>>
>> Dan
>>
>>
>>
>>> So, this was base on this thread:
>>>
>>> https://sourceforge.net/p/kaldi/mailman/message/31160057/
>>>
>>> Our "goal" is to determine what the last word spoken is at 1 minute for a given audio file.   How would you recommend doing that?
>>>
>>> This seems very close to what we need, but not quite there.
>>>
>>> Thanks,
>>>
>>> Nathan
>>>
>>> On Aug 1, 2013, at 1:12 PM, Daniel Povey wrote:
>>>
>>>> I wasn't aware we had any decoder that prints out per-word timings.
>>>> Anyway, even if we did, those timings would not be accurate because of
>>>> the word-symbols being "pushed around" in the graph.
>>>> Dan
>>>>
>>>>
>>>>
>>>> On Thu, Aug 1, 2013 at 4:08 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>>
>>>>> Sorry for my misuse of terminology.  Let me know if there are better words for these.
>>>>>
>>>>> What I am calling the "decoding file" is the file generated during the decoding process.  Maybe the transcription or hypothesis would be more accurate.  This is what I use for that.    I have it start at a beam of 5 and go up to 20.  I get pretty good results for the most part.
>>>>>
>>>>>>>>> steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>>>
>>>>>
>>>>> The "timings" file is what I call the file that shows the time each word starts for a decoded phrase.   It is based largely on swbd/s5/local/score_sclite.sh
>>>>>
>>>>>>>>> lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>>>
>>>>> For some reason the "timings file" and "decoding file" do not match.   I'm wonder if they should match or is there likely an upstream error.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Nathan
>>>>>
>>>>> On Aug 1, 2013, at 12:38 PM, Daniel Povey wrote:
>>>>>
>>>>>> I am not sure what you mean by "timing file" and "decoding file".
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 1, 2013 at 3:37 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>>>> I'm wondering why the timing file doesn't match my decoding file.
>>>>>>>
>>>>>>> They should match right?
>>>>>>>
>>>>>>> Nathan
>>>>>>>
>>>>>>> On Aug 1, 2013, at 12:15 PM, Daniel Povey <dp...@gm...> wrote:
>>>>>>>
>>>>>>>> Nathan-- I don't really understand what you are saying or what you are asking.
>>>>>>>> Dan
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>>>>>>
>>>>>>>>> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>>>>>>>>>
>>>>>>>>> in the decode file (20.txt , decoding with a beam of 20) :
>>>>>>>>>
>>>>>>>>> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>>>>>>>>>
>>>>>>>>> in the timings.all.txt file:
>>>>>>>>> 02.cut1 1 2.64 0.03 <UNK>
>>>>>>>>> 02.cut1 1 3.06 0.11 YOU
>>>>>>>>> 02.cut1 1 3.17 0.22 ARE
>>>>>>>>> 02.cut1 1 3.39 0.40 STANDING
>>>>>>>>> 02.cut1 1 4.06 0.23 ON
>>>>>>>>> 02.cut1 1 4.29 0.06 A
>>>>>>>>> 02.cut1 1 4.35 0.12 IS
>>>>>>>>> 02.cut1 1 4.57 0.75 INVIOLATE
>>>>>>>>> 02.cut1 1 5.43 1.24 ECOSYSTEM
>>>>>>>>> 02.cut1 1 6.88 0.35 NESTS
>>>>>>>>> 02.cut1 1 7.64 0.65 ASSISTED(2)
>>>>>>>>> 02.cut1 1 8.29 0.19 OUT
>>>>>>>>> 02.cut1 1 9.76 1.06 ENLISTED(2)
>>>>>>>>> 02.cut1 1 10.82 0.41 WOULD
>>>>>>>>> 02.cut1 1 11.23 0.59 AN
>>>>>>>>> 02.cut1 1 11.82 0.79 ALAN'S
>>>>>>>>> 02.cut1 1 12.67 1.03 NETTLESOME
>>>>>>>>> 02.cut1 1 13.84 1.00 INSISTED
>>>>>>>>> 02.cut1 1 14.84 0.21 AND
>>>>>>>>>
>>>>>>>>> Here are the commands:
>>>>>>>>>
>>>>>>>>> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
>>>>>>>>> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>>>>>>> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>>>>>>> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
>>>>>>>>> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
>>>>>>>>> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>>>>>>>>>
>>>>>>>>> Anyway, any help is appreciated.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Nathan
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

On Aug 1, 2013, at 2:04 PM, Daniel Povey wrote:

> Your stuff based on lattice-align-words seemed like it could give the
> right answer.  The mismatch you described is what I would expect and
> is not a problem, it's because you didn't run lattice-align-words in
> the baseline.

I'm trying to understand what you mean by "run lattice-align-words in the baseline".   Are you saying that I should align the lattices prior to running them?  


in wsj/s5/steps/word_align_lattices.sh , I see:

$cmd JOB=1:$nj $outdir/log/align.JOB.log \
  lattice-align-words --silence-label=$silence_label --test=true \
   $wbfile $mdl "ark:gunzip -c $indir/lat.JOB.gz|" "ark,t:|gzip -c >$outdir/lat.JOB.gz" || exit 1;

I'm assuming this aligns the lattice that can later be used below:

> lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt


Am I on the right track, or is there a better place for you to point me.  

>  I don't know exactly what the problem is-- perhaps the
> times are too far from the human-generated reference?

We can look into that. 

> In future if you ask questions about this, please paste the output of
> the programs concerned and the corresponding command lines.

Sorry, I'll do that in the future. 

Thanks,

Nathan

> 
> Dan
> 
> 
> 
>> So, this was base on this thread:
>> 
>> https://sourceforge.net/p/kaldi/mailman/message/31160057/
>> 
>> Our "goal" is to determine what the last word spoken is at 1 minute for a given audio file.   How would you recommend doing that?
>> 
>> This seems very close to what we need, but not quite there.
>> 
>> Thanks,
>> 
>> Nathan
>> 
>> On Aug 1, 2013, at 1:12 PM, Daniel Povey wrote:
>> 
>>> I wasn't aware we had any decoder that prints out per-word timings.
>>> Anyway, even if we did, those timings would not be accurate because of
>>> the word-symbols being "pushed around" in the graph.
>>> Dan
>>> 
>>> 
>>> 
>>> On Thu, Aug 1, 2013 at 4:08 PM, Nathan Dunn <nd...@uo...> wrote:
>>>> 
>>>> Sorry for my misuse of terminology.  Let me know if there are better words for these.
>>>> 
>>>> What I am calling the "decoding file" is the file generated during the decoding process.  Maybe the transcription or hypothesis would be more accurate.  This is what I use for that.    I have it start at a beam of 5 and go up to 20.  I get pretty good results for the most part.
>>>> 
>>>>>>>> steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>> 
>>>> 
>>>> The "timings" file is what I call the file that shows the time each word starts for a decoded phrase.   It is based largely on swbd/s5/local/score_sclite.sh
>>>> 
>>>>>>>> lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>> 
>>>> For some reason the "timings file" and "decoding file" do not match.   I'm wonder if they should match or is there likely an upstream error.
>>>> 
>>>> Thanks,
>>>> 
>>>> Nathan
>>>> 
>>>> On Aug 1, 2013, at 12:38 PM, Daniel Povey wrote:
>>>> 
>>>>> I am not sure what you mean by "timing file" and "decoding file".
>>>>> Dan
>>>>> 
>>>>> 
>>>>> On Thu, Aug 1, 2013 at 3:37 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>>> I'm wondering why the timing file doesn't match my decoding file.
>>>>>> 
>>>>>> They should match right?
>>>>>> 
>>>>>> Nathan
>>>>>> 
>>>>>> On Aug 1, 2013, at 12:15 PM, Daniel Povey <dp...@gm...> wrote:
>>>>>> 
>>>>>>> Nathan-- I don't really understand what you are saying or what you are asking.
>>>>>>> Dan
>>>>>>> 
>>>>>>> 
>>>>>>> On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>>>>> 
>>>>>>>> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>>>>>>>> 
>>>>>>>> in the decode file (20.txt , decoding with a beam of 20) :
>>>>>>>> 
>>>>>>>> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>>>>>>>> 
>>>>>>>> in the timings.all.txt file:
>>>>>>>> 02.cut1 1 2.64 0.03 <UNK>
>>>>>>>> 02.cut1 1 3.06 0.11 YOU
>>>>>>>> 02.cut1 1 3.17 0.22 ARE
>>>>>>>> 02.cut1 1 3.39 0.40 STANDING
>>>>>>>> 02.cut1 1 4.06 0.23 ON
>>>>>>>> 02.cut1 1 4.29 0.06 A
>>>>>>>> 02.cut1 1 4.35 0.12 IS
>>>>>>>> 02.cut1 1 4.57 0.75 INVIOLATE
>>>>>>>> 02.cut1 1 5.43 1.24 ECOSYSTEM
>>>>>>>> 02.cut1 1 6.88 0.35 NESTS
>>>>>>>> 02.cut1 1 7.64 0.65 ASSISTED(2)
>>>>>>>> 02.cut1 1 8.29 0.19 OUT
>>>>>>>> 02.cut1 1 9.76 1.06 ENLISTED(2)
>>>>>>>> 02.cut1 1 10.82 0.41 WOULD
>>>>>>>> 02.cut1 1 11.23 0.59 AN
>>>>>>>> 02.cut1 1 11.82 0.79 ALAN'S
>>>>>>>> 02.cut1 1 12.67 1.03 NETTLESOME
>>>>>>>> 02.cut1 1 13.84 1.00 INSISTED
>>>>>>>> 02.cut1 1 14.84 0.21 AND
>>>>>>>> 
>>>>>>>> Here are the commands:
>>>>>>>> 
>>>>>>>> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
>>>>>>>> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>>>>>> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>>>>>> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
>>>>>>>> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
>>>>>>>> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>>>>>>>> 
>>>>>>>> Anyway, any help is appreciated.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Nathan
>>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Your stuff based on lattice-align-words seemed like it could give the
right answer.  The mismatch you described is what I would expect and
is not a problem, it's because you didn't run lattice-align-words in
the baseline.  I don't know exactly what the problem is-- perhaps the
times are too far from the human-generated reference?
In future if you ask questions about this, please paste the output of
the programs concerned and the corresponding command lines.

Dan



> So, this was base on this thread:
>
> https://sourceforge.net/p/kaldi/mailman/message/31160057/
>
> Our "goal" is to determine what the last word spoken is at 1 minute for a given audio file.   How would you recommend doing that?
>
> This seems very close to what we need, but not quite there.
>
> Thanks,
>
> Nathan
>
> On Aug 1, 2013, at 1:12 PM, Daniel Povey wrote:
>
>> I wasn't aware we had any decoder that prints out per-word timings.
>> Anyway, even if we did, those timings would not be accurate because of
>> the word-symbols being "pushed around" in the graph.
>> Dan
>>
>>
>>
>> On Thu, Aug 1, 2013 at 4:08 PM, Nathan Dunn <nd...@uo...> wrote:
>>>
>>> Sorry for my misuse of terminology.  Let me know if there are better words for these.
>>>
>>> What I am calling the "decoding file" is the file generated during the decoding process.  Maybe the transcription or hypothesis would be more accurate.  This is what I use for that.    I have it start at a beam of 5 and go up to 20.  I get pretty good results for the most part.
>>>
>>>>>>> steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>
>>>
>>> The "timings" file is what I call the file that shows the time each word starts for a decoded phrase.   It is based largely on swbd/s5/local/score_sclite.sh
>>>
>>>>>>> lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>
>>> For some reason the "timings file" and "decoding file" do not match.   I'm wonder if they should match or is there likely an upstream error.
>>>
>>> Thanks,
>>>
>>> Nathan
>>>
>>> On Aug 1, 2013, at 12:38 PM, Daniel Povey wrote:
>>>
>>>> I am not sure what you mean by "timing file" and "decoding file".
>>>> Dan
>>>>
>>>>
>>>> On Thu, Aug 1, 2013 at 3:37 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>> I'm wondering why the timing file doesn't match my decoding file.
>>>>>
>>>>> They should match right?
>>>>>
>>>>> Nathan
>>>>>
>>>>> On Aug 1, 2013, at 12:15 PM, Daniel Povey <dp...@gm...> wrote:
>>>>>
>>>>>> Nathan-- I don't really understand what you are saying or what you are asking.
>>>>>> Dan
>>>>>>
>>>>>>
>>>>>> On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>>>>
>>>>>>> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>>>>>>>
>>>>>>> in the decode file (20.txt , decoding with a beam of 20) :
>>>>>>>
>>>>>>> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>>>>>>>
>>>>>>> in the timings.all.txt file:
>>>>>>> 02.cut1 1 2.64 0.03 <UNK>
>>>>>>> 02.cut1 1 3.06 0.11 YOU
>>>>>>> 02.cut1 1 3.17 0.22 ARE
>>>>>>> 02.cut1 1 3.39 0.40 STANDING
>>>>>>> 02.cut1 1 4.06 0.23 ON
>>>>>>> 02.cut1 1 4.29 0.06 A
>>>>>>> 02.cut1 1 4.35 0.12 IS
>>>>>>> 02.cut1 1 4.57 0.75 INVIOLATE
>>>>>>> 02.cut1 1 5.43 1.24 ECOSYSTEM
>>>>>>> 02.cut1 1 6.88 0.35 NESTS
>>>>>>> 02.cut1 1 7.64 0.65 ASSISTED(2)
>>>>>>> 02.cut1 1 8.29 0.19 OUT
>>>>>>> 02.cut1 1 9.76 1.06 ENLISTED(2)
>>>>>>> 02.cut1 1 10.82 0.41 WOULD
>>>>>>> 02.cut1 1 11.23 0.59 AN
>>>>>>> 02.cut1 1 11.82 0.79 ALAN'S
>>>>>>> 02.cut1 1 12.67 1.03 NETTLESOME
>>>>>>> 02.cut1 1 13.84 1.00 INSISTED
>>>>>>> 02.cut1 1 14.84 0.21 AND
>>>>>>>
>>>>>>> Here are the commands:
>>>>>>>
>>>>>>> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
>>>>>>> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>>>>> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>>>>> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
>>>>>>> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
>>>>>>> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>>>>>>>
>>>>>>>
>>>>>>> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>>>>>>>
>>>>>>> Anyway, any help is appreciated.
>>>>>>>
>>>>>>>
>>>>>>> Nathan
>>>>>>
>>>>
>>>
>>
>

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

So, this was base on this thread:

https://sourceforge.net/p/kaldi/mailman/message/31160057/

Our "goal" is to determine what the last word spoken is at 1 minute for a given audio file.   How would you recommend doing that?   

This seems very close to what we need, but not quite there.  

Thanks,

Nathan

On Aug 1, 2013, at 1:12 PM, Daniel Povey wrote:

> I wasn't aware we had any decoder that prints out per-word timings.
> Anyway, even if we did, those timings would not be accurate because of
> the word-symbols being "pushed around" in the graph.
> Dan
> 
> 
> 
> On Thu, Aug 1, 2013 at 4:08 PM, Nathan Dunn <nd...@uo...> wrote:
>> 
>> Sorry for my misuse of terminology.  Let me know if there are better words for these.
>> 
>> What I am calling the "decoding file" is the file generated during the decoding process.  Maybe the transcription or hypothesis would be more accurate.  This is what I use for that.    I have it start at a beam of 5 and go up to 20.  I get pretty good results for the most part.
>> 
>>>>>> steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>> 
>> 
>> The "timings" file is what I call the file that shows the time each word starts for a decoded phrase.   It is based largely on swbd/s5/local/score_sclite.sh
>> 
>>>>>> lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>> 
>> For some reason the "timings file" and "decoding file" do not match.   I'm wonder if they should match or is there likely an upstream error.
>> 
>> Thanks,
>> 
>> Nathan
>> 
>> On Aug 1, 2013, at 12:38 PM, Daniel Povey wrote:
>> 
>>> I am not sure what you mean by "timing file" and "decoding file".
>>> Dan
>>> 
>>> 
>>> On Thu, Aug 1, 2013 at 3:37 PM, Nathan Dunn <nd...@uo...> wrote:
>>>> I'm wondering why the timing file doesn't match my decoding file.
>>>> 
>>>> They should match right?
>>>> 
>>>> Nathan
>>>> 
>>>> On Aug 1, 2013, at 12:15 PM, Daniel Povey <dp...@gm...> wrote:
>>>> 
>>>>> Nathan-- I don't really understand what you are saying or what you are asking.
>>>>> Dan
>>>>> 
>>>>> 
>>>>> On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>>> 
>>>>>> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>>>>>> 
>>>>>> in the decode file (20.txt , decoding with a beam of 20) :
>>>>>> 
>>>>>> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>>>>>> 
>>>>>> in the timings.all.txt file:
>>>>>> 02.cut1 1 2.64 0.03 <UNK>
>>>>>> 02.cut1 1 3.06 0.11 YOU
>>>>>> 02.cut1 1 3.17 0.22 ARE
>>>>>> 02.cut1 1 3.39 0.40 STANDING
>>>>>> 02.cut1 1 4.06 0.23 ON
>>>>>> 02.cut1 1 4.29 0.06 A
>>>>>> 02.cut1 1 4.35 0.12 IS
>>>>>> 02.cut1 1 4.57 0.75 INVIOLATE
>>>>>> 02.cut1 1 5.43 1.24 ECOSYSTEM
>>>>>> 02.cut1 1 6.88 0.35 NESTS
>>>>>> 02.cut1 1 7.64 0.65 ASSISTED(2)
>>>>>> 02.cut1 1 8.29 0.19 OUT
>>>>>> 02.cut1 1 9.76 1.06 ENLISTED(2)
>>>>>> 02.cut1 1 10.82 0.41 WOULD
>>>>>> 02.cut1 1 11.23 0.59 AN
>>>>>> 02.cut1 1 11.82 0.79 ALAN'S
>>>>>> 02.cut1 1 12.67 1.03 NETTLESOME
>>>>>> 02.cut1 1 13.84 1.00 INSISTED
>>>>>> 02.cut1 1 14.84 0.21 AND
>>>>>> 
>>>>>> Here are the commands:
>>>>>> 
>>>>>> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
>>>>>> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>>>> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>>>> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
>>>>>> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
>>>>>> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>>>>>> 
>>>>>> 
>>>>>> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>>>>>> 
>>>>>> Anyway, any help is appreciated.
>>>>>> 
>>>>>> 
>>>>>> Nathan
>>>>> 
>>> 
>> 
> 

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

I wasn't aware we had any decoder that prints out per-word timings.
Anyway, even if we did, those timings would not be accurate because of
the word-symbols being "pushed around" in the graph.
Dan



On Thu, Aug 1, 2013 at 4:08 PM, Nathan Dunn <nd...@uo...> wrote:
>
> Sorry for my misuse of terminology.  Let me know if there are better words for these.
>
> What I am calling the "decoding file" is the file generated during the decoding process.  Maybe the transcription or hypothesis would be more accurate.  This is what I use for that.    I have it start at a beam of 5 and go up to 20.  I get pretty good results for the most part.
>
>>>>> steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>
>
> The "timings" file is what I call the file that shows the time each word starts for a decoded phrase.   It is based largely on swbd/s5/local/score_sclite.sh
>
>>>>> lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>
> For some reason the "timings file" and "decoding file" do not match.   I'm wonder if they should match or is there likely an upstream error.
>
> Thanks,
>
> Nathan
>
> On Aug 1, 2013, at 12:38 PM, Daniel Povey wrote:
>
>> I am not sure what you mean by "timing file" and "decoding file".
>> Dan
>>
>>
>> On Thu, Aug 1, 2013 at 3:37 PM, Nathan Dunn <nd...@uo...> wrote:
>>> I'm wondering why the timing file doesn't match my decoding file.
>>>
>>> They should match right?
>>>
>>> Nathan
>>>
>>> On Aug 1, 2013, at 12:15 PM, Daniel Povey <dp...@gm...> wrote:
>>>
>>>> Nathan-- I don't really understand what you are saying or what you are asking.
>>>> Dan
>>>>
>>>>
>>>> On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>>>>>
>>>>> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>>>>>
>>>>> in the decode file (20.txt , decoding with a beam of 20) :
>>>>>
>>>>> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>>>>>
>>>>> in the timings.all.txt file:
>>>>> 02.cut1 1 2.64 0.03 <UNK>
>>>>> 02.cut1 1 3.06 0.11 YOU
>>>>> 02.cut1 1 3.17 0.22 ARE
>>>>> 02.cut1 1 3.39 0.40 STANDING
>>>>> 02.cut1 1 4.06 0.23 ON
>>>>> 02.cut1 1 4.29 0.06 A
>>>>> 02.cut1 1 4.35 0.12 IS
>>>>> 02.cut1 1 4.57 0.75 INVIOLATE
>>>>> 02.cut1 1 5.43 1.24 ECOSYSTEM
>>>>> 02.cut1 1 6.88 0.35 NESTS
>>>>> 02.cut1 1 7.64 0.65 ASSISTED(2)
>>>>> 02.cut1 1 8.29 0.19 OUT
>>>>> 02.cut1 1 9.76 1.06 ENLISTED(2)
>>>>> 02.cut1 1 10.82 0.41 WOULD
>>>>> 02.cut1 1 11.23 0.59 AN
>>>>> 02.cut1 1 11.82 0.79 ALAN'S
>>>>> 02.cut1 1 12.67 1.03 NETTLESOME
>>>>> 02.cut1 1 13.84 1.00 INSISTED
>>>>> 02.cut1 1 14.84 0.21 AND
>>>>>
>>>>> Here are the commands:
>>>>>
>>>>> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
>>>>> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>>> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>>> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
>>>>> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
>>>>> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>>>>>
>>>>>
>>>>> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>>>>>
>>>>> Anyway, any help is appreciated.
>>>>>
>>>>>
>>>>> Nathan
>>>>
>>
>

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Sorry for my misuse of terminology.  Let me know if there are better words for these.

What I am calling the "decoding file" is the file generated during the decoding process.  Maybe the transcription or hypothesis would be more accurate.  This is what I use for that.    I have it start at a beam of 5 and go up to 20.  I get pretty good results for the most part.  

>>>> steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test


The "timings" file is what I call the file that shows the time each word starts for a decoded phrase.   It is based largely on swbd/s5/local/score_sclite.sh

>>>> lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt

For some reason the "timings file" and "decoding file" do not match.   I'm wonder if they should match or is there likely an upstream error.  

Thanks,

Nathan

On Aug 1, 2013, at 12:38 PM, Daniel Povey wrote:

> I am not sure what you mean by "timing file" and "decoding file".
> Dan
> 
> 
> On Thu, Aug 1, 2013 at 3:37 PM, Nathan Dunn <nd...@uo...> wrote:
>> I'm wondering why the timing file doesn't match my decoding file.
>> 
>> They should match right?
>> 
>> Nathan
>> 
>> On Aug 1, 2013, at 12:15 PM, Daniel Povey <dp...@gm...> wrote:
>> 
>>> Nathan-- I don't really understand what you are saying or what you are asking.
>>> Dan
>>> 
>>> 
>>> On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>>>> 
>>>> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>>>> 
>>>> in the decode file (20.txt , decoding with a beam of 20) :
>>>> 
>>>> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>>>> 
>>>> in the timings.all.txt file:
>>>> 02.cut1 1 2.64 0.03 <UNK>
>>>> 02.cut1 1 3.06 0.11 YOU
>>>> 02.cut1 1 3.17 0.22 ARE
>>>> 02.cut1 1 3.39 0.40 STANDING
>>>> 02.cut1 1 4.06 0.23 ON
>>>> 02.cut1 1 4.29 0.06 A
>>>> 02.cut1 1 4.35 0.12 IS
>>>> 02.cut1 1 4.57 0.75 INVIOLATE
>>>> 02.cut1 1 5.43 1.24 ECOSYSTEM
>>>> 02.cut1 1 6.88 0.35 NESTS
>>>> 02.cut1 1 7.64 0.65 ASSISTED(2)
>>>> 02.cut1 1 8.29 0.19 OUT
>>>> 02.cut1 1 9.76 1.06 ENLISTED(2)
>>>> 02.cut1 1 10.82 0.41 WOULD
>>>> 02.cut1 1 11.23 0.59 AN
>>>> 02.cut1 1 11.82 0.79 ALAN'S
>>>> 02.cut1 1 12.67 1.03 NETTLESOME
>>>> 02.cut1 1 13.84 1.00 INSISTED
>>>> 02.cut1 1 14.84 0.21 AND
>>>> 
>>>> Here are the commands:
>>>> 
>>>> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
>>>> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>>> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>>> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
>>>> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
>>>> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>>>> 
>>>> 
>>>> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>>>> 
>>>> Anyway, any help is appreciated.
>>>> 
>>>> 
>>>> Nathan
>>> 
> 

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

I am not sure what you mean by "timing file" and "decoding file".
Dan


On Thu, Aug 1, 2013 at 3:37 PM, Nathan Dunn <nd...@uo...> wrote:
> I'm wondering why the timing file doesn't match my decoding file.
>
> They should match right?
>
> Nathan
>
> On Aug 1, 2013, at 12:15 PM, Daniel Povey <dp...@gm...> wrote:
>
>> Nathan-- I don't really understand what you are saying or what you are asking.
>> Dan
>>
>>
>> On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>>>
>>> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>>>
>>> in the decode file (20.txt , decoding with a beam of 20) :
>>>
>>> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>>>
>>> in the timings.all.txt file:
>>> 02.cut1 1 2.64 0.03 <UNK>
>>> 02.cut1 1 3.06 0.11 YOU
>>> 02.cut1 1 3.17 0.22 ARE
>>> 02.cut1 1 3.39 0.40 STANDING
>>> 02.cut1 1 4.06 0.23 ON
>>> 02.cut1 1 4.29 0.06 A
>>> 02.cut1 1 4.35 0.12 IS
>>> 02.cut1 1 4.57 0.75 INVIOLATE
>>> 02.cut1 1 5.43 1.24 ECOSYSTEM
>>> 02.cut1 1 6.88 0.35 NESTS
>>> 02.cut1 1 7.64 0.65 ASSISTED(2)
>>> 02.cut1 1 8.29 0.19 OUT
>>> 02.cut1 1 9.76 1.06 ENLISTED(2)
>>> 02.cut1 1 10.82 0.41 WOULD
>>> 02.cut1 1 11.23 0.59 AN
>>> 02.cut1 1 11.82 0.79 ALAN'S
>>> 02.cut1 1 12.67 1.03 NETTLESOME
>>> 02.cut1 1 13.84 1.00 INSISTED
>>> 02.cut1 1 14.84 0.21 AND
>>>
>>> Here are the commands:
>>>
>>> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
>>> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>>> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>>> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
>>> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
>>> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>>>
>>>
>>> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>>>
>>> Anyway, any help is appreciated.
>>>
>>>
>>> Nathan
>>

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

I'm wondering why the timing file doesn't match my decoding file.   

They should match right? 

Nathan

On Aug 1, 2013, at 12:15 PM, Daniel Povey <dp...@gm...> wrote:

> Nathan-- I don't really understand what you are saying or what you are asking.
> Dan
> 
> 
> On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>> 
>> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>> 
>> in the decode file (20.txt , decoding with a beam of 20) :
>> 
>> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>> 
>> in the timings.all.txt file:
>> 02.cut1 1 2.64 0.03 <UNK>
>> 02.cut1 1 3.06 0.11 YOU
>> 02.cut1 1 3.17 0.22 ARE
>> 02.cut1 1 3.39 0.40 STANDING
>> 02.cut1 1 4.06 0.23 ON
>> 02.cut1 1 4.29 0.06 A
>> 02.cut1 1 4.35 0.12 IS
>> 02.cut1 1 4.57 0.75 INVIOLATE
>> 02.cut1 1 5.43 1.24 ECOSYSTEM
>> 02.cut1 1 6.88 0.35 NESTS
>> 02.cut1 1 7.64 0.65 ASSISTED(2)
>> 02.cut1 1 8.29 0.19 OUT
>> 02.cut1 1 9.76 1.06 ENLISTED(2)
>> 02.cut1 1 10.82 0.41 WOULD
>> 02.cut1 1 11.23 0.59 AN
>> 02.cut1 1 11.82 0.79 ALAN'S
>> 02.cut1 1 12.67 1.03 NETTLESOME
>> 02.cut1 1 13.84 1.00 INSISTED
>> 02.cut1 1 14.84 0.21 AND
>> 
>> Here are the commands:
>> 
>> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
>> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
>> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
>> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
>> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
>> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>> 
>> 
>> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>> 
>> Anyway, any help is appreciated.
>> 
>> 
>> Nathan
> 

Re: [Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Nathan-- I don't really understand what you are saying or what you are asking.
Dan


On Thu, Aug 1, 2013 at 2:45 PM, Nathan Dunn <nd...@uo...> wrote:
>
> Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.
>
> in the decode file (20.txt , decoding with a beam of 20) :
>
> 02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . .
>
> in the timings.all.txt file:
> 02.cut1 1 2.64 0.03 <UNK>
> 02.cut1 1 3.06 0.11 YOU
> 02.cut1 1 3.17 0.22 ARE
> 02.cut1 1 3.39 0.40 STANDING
> 02.cut1 1 4.06 0.23 ON
> 02.cut1 1 4.29 0.06 A
> 02.cut1 1 4.35 0.12 IS
> 02.cut1 1 4.57 0.75 INVIOLATE
> 02.cut1 1 5.43 1.24 ECOSYSTEM
> 02.cut1 1 6.88 0.35 NESTS
> 02.cut1 1 7.64 0.65 ASSISTED(2)
> 02.cut1 1 8.29 0.19 OUT
> 02.cut1 1 9.76 1.06 ENLISTED(2)
> 02.cut1 1 10.82 0.41 WOULD
> 02.cut1 1 11.23 0.59 AN
> 02.cut1 1 11.82 0.79 ALAN'S
> 02.cut1 1 12.67 1.03 NETTLESOME
> 02.cut1 1 13.84 1.00 INSISTED
> 02.cut1 1 14.84 0.21 AND
>
> Here are the commands:
>
> %utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
> %steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
> %lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
> LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
> LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
> LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.
>
>
> It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.
>
> Anyway, any help is appreciated.
>
>
> Nathan
>
>

[Kaldi-users] timings don't match decoding

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Following the formal s5 scripts in wsj and the directions below, I was able to get word timings that ROUGHLY matched the decoding values I was getting.   

in the decode file (20.txt , decoding with a beam of 20) :

02.cut1 YOU ARE STANDING ON A SANDY WHITE BEACH OF THE ASSISTED AND END A THAN ANY OTHER KIND CONSISTED . . . 

in the timings.all.txt file:
02.cut1 1 2.64 0.03 <UNK> 
02.cut1 1 3.06 0.11 YOU 
02.cut1 1 3.17 0.22 ARE 
02.cut1 1 3.39 0.40 STANDING 
02.cut1 1 4.06 0.23 ON 
02.cut1 1 4.29 0.06 A 
02.cut1 1 4.35 0.12 IS 
02.cut1 1 4.57 0.75 INVIOLATE 
02.cut1 1 5.43 1.24 ECOSYSTEM 
02.cut1 1 6.88 0.35 NESTS 
02.cut1 1 7.64 0.65 ASSISTED(2) 
02.cut1 1 8.29 0.19 OUT 
02.cut1 1 9.76 1.06 ENLISTED(2) 
02.cut1 1 10.82 0.41 WOULD 
02.cut1 1 11.23 0.59 AN 
02.cut1 1 11.82 0.79 ALAN'S 
02.cut1 1 12.67 1.03 NETTLESOME 
02.cut1 1 13.84 1.00 INSISTED 
02.cut1 1 14.84 0.21 AND 

Here are the commands:

%utils/mkgraph.sh data/local/g300_test/lang exp/tri1 exp/tri1/graph
%steps/decode.sh --nj 10 --model exp/tri1/final.mdl --num-threads 1 --acwt 0.1 --cmd "$decode_cmd" --config conf/decode.config exp/tri1/graph data/local/g300_test exp/tri1/decode_g300_test
%lattice-1best "ark:gunzip -c exp/tri1/decode_g300_test/lat.*.gz|" ark:- | lattice-align-words ./data/local/g300_test/lang/phones/word_boundary.int exp/tri1/final.mdl ark:- ark:- | nbest-to-ctm ark:- - | ./utils/int2sym.pl -f 5 ./data/local/g300_test/lang/words.txt > exp/tri1/decode_g300_test/timings.all.txt
LOG (lattice-1best:main():lattice-1best.cc:88) Done converting 339 to best path, 0 had errors.
LOG (lattice-align-words:main():lattice-align-words.cc:117) Successfully aligned 339 lattices; 0 had errors.
LOG (nbest-to-ctm:main():nbest-to-ctm.cc:95) Converted 339 linear lattices to ctm format; 0 had errors.


It should use the lattices generated from the decoding and the word_boundary and word files.   I can apply weights for language model and acoustic model, but I doubt that will have a great effect.   The words.txt file must be correct if I am getting "similar" results.   

Anyway, any help is appreciated.  


Nathan

Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Thanks


On 31 July 2013 09:01, Mailing list used for User Communication and Updates
<kal...@li...> wrote:

> Hi-- yes there is but it depends what you're doing.  I don't recall
> even the approximate percentages of improvement though.  You can get a
> temporary research license for MKL for free I think, so you could try
> it out.
> Dan
>
>
> On Tue, Jul 30, 2013 at 7:59 PM, Mailing list used for User
> Communication and Updates <kal...@li...> wrote:
> > Hi,
> > This is a bit off-topic, but is there a bit improvement in speed when
> using
> > MKL vs ATLAS?
> >
> > Thanks
> >
> > Paul
> >
> >
> > On 30 July 2013 23:29, Mailing list used for User Communication and
> Updates
> > <kal...@li...> wrote:
> >>
> >> That's right.  They are working on it.
> >>
> >> --Mike
> >>
> >>
> >> -----Original Message-----
> >> From: Mailing list used for User Communication and Updates
> >> [mailto:kal...@li...]
> >> Sent: Monday, July 29, 2013 9:58 AM
> >> To: kal...@li...
> >> Subject: Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0
> update5
> >>
> >> I previously spoke to Michael Deisher at Intel about this, he says he
> has
> >> asked the relevant team.  So I think they are aware of the problem.
> >> Dan
> >>
> >>
> >> On Mon, Jul 29, 2013 at 12:06 PM, Mailing list used for User
> Communication
> >> and Updates <kal...@li...> wrote:
> >> > It has been reported on intel dev forum.
> >> > http://software.intel.com/en-us/forums/topic/402436
> >> > http://software.intel.com/en-us/forums/topic/402473
> >> >
> >> > The problem is caused by  the incorrect query result of workspace size
> >> > given by dgesvd().
> >> >
> >> > 11.0.5 seems to be a very buggy version.
> >> >
> >> >
> >> >
> >> > On Mon, Jul 29, 2013 at 11:39 PM, Mailing list used for User
> >> > Communication and Updates <kal...@li...> wrote:
> >> >>
> >> >> I just checked different versions of MKL and indeed, this was
> >> >> introduced in the  update 5. Anyone with the commercial MKL support
> >> >> cares to submit a ticket to Intel?
> >> >> y.
> >> >>
> >> >>
> >> >> On Wed, Jul 24, 2013 at 12:42 PM, Mailing list used for User
> >> >> Communication and Updates <kal...@li...> wrote:
> >> >>>
> >> >>> You are right.... I just downgraded to MKL 11.0update1 and passed
> >> >>> the test.
> >> >>>
> >> >>>
> >> >>>
> >> >>> On Thu, Jul 25, 2013 at 12:28 AM, Mailing list used for User
> >> >>> Communication and Updates <kal...@li...>
> wrote:
> >> >>>>
> >> >>>> I saw some changes regarding improving performance of *GESVD in the
> >> >>>> MKL
> >> >>>> 11.0 update5  changelog. I wonder if they messed something up? How
> >> >>>> about trying slightly "older" version of the MKL? If you are on the
> >> >>>> subscription of Intel Software Network, you can download older
> >> >>>> versions of the MKL at no charge...
> >> >>>>
> >> >>>> We are using Intel 11.0 and the tests pass without any problem...
> >> >>>>
> >> >>>> yenda
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> On Wed, Jul 24, 2013 at 12:14 PM, Mailing list used for User
> >> >>>> Communication and Updates <kal...@li...>
> wrote:
> >> >>>>>
> >> >>>>> I can't see any problems in the Kaldi code.  Perhaps this is an
> >> >>>>> older version of MKL?  It could be a bug in MKL itself.  MKL tests
> >> >>>>> OK for me.
> >> >>>>> Dan
> >> >>>>>
> >> >>>>>
> >> >>>>> On Wed, Jul 24, 2013 at 11:55 AM, Mailing list used for User
> >> >>>>> Communication and Updates <kal...@li...>
> wrote:
> >> >>>>> > here's the dbg info copied from eclipse.
> >> >>>>> > PS: I can pass the test with atlas provided by centos repo
> >> >>>>> > atlas-sse3.x86_64                          3.8.4-2.el6
> >> >>>>> > @base
> >> >>>>> > atlas-sse3-devel.x86_64                    3.8.4-2.el6
> >> >>>>> > @base
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> > stack:
> >> >>>>> >
> >> >>>>> > matrix-lib-test [C/C++ Application] matrix-lib-test [11312]
> >> >>>>> > [cores: 1] Thread [1] 11312 [core: 1] (Suspended : Signal :
> >> >>>>> > SIGABRT:Aborted)
> >> >>>>> > raise() at 0x33090328a5
> >> >>>>> > abort() at 0x3309034085
> >> >>>>> > kaldi::KaldiAssertFailure_() at kaldi-error.cc:136 0x4cf401
> >> >>>>> > kaldi::MatrixBase<double>::LapackGesvd() at kaldi-matrix.cc:401
> >> >>>>> > 0x48fdd3
> >> >>>>> > kaldi::MatrixBase<double>::DestructiveSvd() at
> >> >>>>> > kaldi-matrix.cc:1,330
> >> >>>>> > 0x48b902
> >> >>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.cc:1,352
> >> >>>>> > 0x48b9bb
> >> >>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.h:315 0x468e50
> >> >>>>> > kaldi::MatrixBase<double>::Cond() at kaldi-matrix.cc:1,222
> >> >>>>> > 0x48bf47
> >> >>>>> > kaldi::InitRand<double>() at matrix-lib-test.cc:74 0x42dd7e
> >> >>>>> > kaldi::UnitTestCompressedMatrix<double>() at
> >> >>>>> > matrix-lib-test.cc:3,532 0x43055a
> >> >>>>> > kaldi::MatrixUnitTest<double>() at matrix-lib-test.cc:3,705
> >> >>>>> > 0x42de8b
> >> >>>>> > main() at matrix-lib-test.cc:3,829 0x42dac7
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> > variable list after invoking the following    @line393
> >> >>>>> > kaldi-matrix.cc
> >> >>>>> >   clapack_Xgesvd(v_job, u_job,
> >> >>>>> >                  &M, &N, data_, &LDA,
> >> >>>>> >                  s->Data(),
> >> >>>>> >                  V->Data(), &V_stride,
> >> >>>>> >                  U->Data(), &U_stride,
> >> >>>>> >                  p_work, &l_work,
> >> >>>>> >                  &result);
> >> >>>>> >
> >> >>>>> >
> >> >>>>> > __func__ char [12] 0x4d5418
> >> >>>>> > M KaldiBlasInt 1
> >> >>>>> > LDA KaldiBlasInt 2
> >> >>>>> > U kaldi::MatrixBase<double> * 0x7ffffffface0 data_ double *
> >> >>>>> > 0x7413b0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1
> >> >>>>> > num_rows_ kaldi::MatrixIndexT 10 stride_ kaldi::MatrixIndexT 2
> >> >>>>> > l_work KaldiBlasInt 5 p_work double * 0x741460 *p_work double
> >> >>>>> > 1.0829823019173015e-312 temp void * 0x741460 N KaldiBlasInt 10 V
> >> >>>>> > kaldi::MatrixBase<double> * 0x7fffffffacc0 data_ double *
> >> >>>>> > 0x7406a0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1
> >> >>>>> > num_rows_ kaldi::MatrixIndexT 1 stride_ kaldi::MatrixIndexT 2
> >> >>>>> > result KaldiBlasInt -13 tmpU kaldi::Matrix<double> {...}
> >> >>>>> > V_stride KaldiBlasInt 2 work_query double 5 u_job char *
> >> >>>>> > 0x4d4fea "N"
> >> >>>>> > tmpV kaldi::Matrix<double> {...} U_stride KaldiBlasInt 2 v_job
> >> >>>>> > char * 0x4d4fea "N"
> >> >>>>> > this kaldi::MatrixBase<double> * const 0x7fffffffaf80 data_
> >> >>>>> > double * 0x741300 *data_ double 0.78239572048187256 num_cols_
> >> >>>>> > kaldi::MatrixIndexT 1 num_rows_ kaldi::MatrixIndexT 10 stride_
> >> >>>>> > kaldi::MatrixIndexT 2 s kaldi::VectorBase<double> *
> >> >>>>> > 0x7fffffffb190 data_ double * 0x7404a0 *data_ double 0 dim_
> >> >>>>> > kaldi::MatrixIndexT 1 U_in kaldi::MatrixBase<double> * 0x0 V_in
> >> >>>>> > kaldi::MatrixBase<double> * 0x0
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> > On Wed, Jul 24, 2013 at 11:04 PM, Mailing list used for User
> >> >>>>> > Communication and Updates <kal...@li...>
> >> >>>>> > wrote:
> >> >>>>> >>
> >> >>>>> >> Could you please run this in a debugger and show me the values
> >> >>>>> >> of p_work, l_work and result, e.g.
> >> >>>>> >>
> >> >>>>> >> gdb matrix-lib
> >> >>>>> >> (gdb) r
> >> >>>>> >> [wait till it crashes]
> >> >>>>> >> (gdb) up
> >> >>>>> >> (gdb) up    # not sure how many times
> >> >>>>> >> (gdb) up
> >> >>>>> >> (gdb) p l_work
> >> >>>>> >> (gdb) p p_work
> >> >>>>> >> (gdb) p result
> >> >>>>> >>
> >> >>>>> >> Dan
> >> >>>>> >>
> >> >>>>> >>
> >> >>>>> >>
> >> >>>>> >> On Wed, Jul 24, 2013 at 10:34 AM, Mailing list used for User
> >> >>>>> >> Communication and Updates <kal...@li...>
> >> >>>>> >> wrote:
> >> >>>>> >> > Hi,
> >> >>>>> >> >
> >> >>>>> >> > I tried latest trunk version with MKL. I have no problem when
> >> >>>>> >> > running `make all`, but get a FAIL when testing. Can anyone
> >> >>>>> >> > help? Thanks.
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > svn revision 2720 trunk
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > configuration line:
> >> >>>>> >> > $ ./configure --threaded-math=yes --mathlib=MKL
> >> >>>>> >> > --use-cuda=yes --mkl-root=$MKLROOT
> >> >>>>> >> > --omp-libdir=/opt/intel/lib/intel64
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > MKL version 11.0 update 5 (latest)
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > Linux version CentOS 6.4
> >> >>>>> >> > $ uname -a
> >> >>>>> >> > Linux 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36
> >> >>>>> >> > UTC
> >> >>>>> >> > 2013
> >> >>>>> >> > x86_64
> >> >>>>> >> > x86_64 x86_64 GNU/Linux
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > $ gcc -v
> >> >>>>> >> > Using built-in specs.
> >> >>>>> >> > Target: x86_64-redhat-linux
> >> >>>>> >> > Configured with: ../configure --prefix=/usr
> >> >>>>> >> > --mandir=/usr/share/man --infodir=/usr/share/info
> >> >>>>> >> > --with-bugurl=http://bugzilla.redhat.com/bugzilla
> >> >>>>> >> > --enable-bootstrap --enable-shared --enable-threads=posix
> >> >>>>> >> > --enable-checking=release --with-system-zlib
> >> >>>>> >> > --enable-__cxa_atexit --disable-libunwind-exceptions
> >> >>>>> >> > --enable-gnu-unique-object
> >> >>>>> >> > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada
> >> >>>>> >> > --enable-java-awt=gtk
> >> >>>>> >> > --disable-dssi
> >> >>>>> >> > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre
> >> >>>>> >> > --enable-libgcj-multifile --enable-java-maintainer-mode
> >> >>>>> >> > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar
> >> >>>>> >> > --disable-libjava-multilib
> >> >>>>> >> > --with-ppl --with-cloog --with-tune=generic
> >> >>>>> >> > --with-arch_32=i686 --build=x86_64-redhat-linux Thread model:
> >> >>>>> >> > posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > ~/kaldi/trunk/src/matrix
> >> >>>>> >> > $ make test
> >> >>>>> >> > Running matrix-lib-test .../bin/sh: line 1:  5561 Aborted
> >> >>>>> >> > (core dumped) ./$x > /dev/null 2>&1 ... FAIL Running
> >> >>>>> >> > kaldi-gpsr-test ...... SUCCESS
> >> >>>>> >> > make: *** [test] Error 1
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > $ ./matrix-lib-test
> >> >>>>> >> > ...
> >> >>>>> >> > ...
> >> >>>>> >> > VLOG[2] (UnitTestLbfgs():matrix-lib-test.cc:2734) L-BFGS
> >> >>>>> >> > finished after
> >> >>>>> >> > 59
> >> >>>>> >> > function evaluations.
> >> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3623) M =
> >> >>>>> >> > [ ]
> >> >>>>> >> >
> >> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3624) M2 =
> >> >>>>> >> > [ ]
> >> >>>>> >> >
> >> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3626)
> >> >>>>> >> > Compressed
> >> >>>>> >> > matrix,
> >> >>>>> >> > tot = 0, diff = 0
> >> >>>>> >> >
> >> >>>>> >> > MKL ERROR: Parameter 13 was incorrect on entry to DGESVD.
> >> >>>>> >> > KALDI_ASSERT: at LapackGesvd:kaldi-matrix.cc:401, failed:
> >> >>>>> >> > result
> >> >>>>> >> > >= 0 &&
> >> >>>>> >> > "Call to CLAPACK dgesvd_ called with wrong arguments"
> >> >>>>> >> > Stack trace is:
> >> >>>>> >> > kaldi::KaldiGetStackTrace()
> >> >>>>> >> > kaldi::KaldiAssertFailure_(char const*, char const*, int,
> >> >>>>> >> > char
> >> >>>>> >> > const*)
> >> >>>>> >> > kaldi::MatrixBase<double>::LapackGesvd(kaldi::VectorBase<doub
> >> >>>>> >> > le>*, kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*)
> >> >>>>> >> >
> >> >>>>> >> > kaldi::MatrixBase<double>::DestructiveSvd(kaldi::VectorBase<d
> >> >>>>> >> > ouble>*, kaldi::MatrixBase<double>*,
> >> >>>>> >> > kaldi::MatrixBase<double>*)
> >> >>>>> >> > kaldi::MatrixBase<double>::Svd(kaldi::VectorBase<double>*,
> >> >>>>> >> > kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*) const
> >> >>>>> >> > .
> >> >>>>> >> > .
> >> >>>>> >> > .
> >> >>>>> >> > ./matrix-lib-test() [0x43055a]
> >> >>>>> >> > ./matrix-lib-test() [0x42de8b]
> >> >>>>> >> > ./matrix-lib-test(main+0x1d) [0x42dac7]
> >> >>>>> >> > /lib64/libc.so.6(__libc_start_main+0xfd) [0x330901ecdd]
> >> >>>>> >> > ./matrix-lib-test() [0x42ced9] Aborted (core dumped)
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> > -------------------------------------------------------------
> >> >>>>> >> > ----------------- See everything from the browser to the
> >> >>>>> >> > database with AppDynamics Get end-to-end visibility with
> >> >>>>> >> > application monitoring from AppDynamics Isolate bottlenecks
> >> >>>>> >> > and diagnose root cause in seconds.
> >> >>>>> >> > Start your free trial of AppDynamics Pro today!
> >> >>>>> >> >
> >> >>>>> >> >
> >> >>>>> >> >
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/41
> >> >>>>> >> > 40/ostg.clktrk
> >> >>>>> >> > _______________________________________________
> >> >>>>> >> > Kaldi-users mailing list
> >> >>>>> >> > Kal...@li...
> >> >>>>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >>>>> >> >
> >> >>>>> >>
> >> >>>>> >>
> >> >>>>> >>
> >> >>>>> >> ---------------------------------------------------------------
> >> >>>>> >> --------------- See everything from the browser to the database
> >> >>>>> >> with AppDynamics Get end-to-end visibility with application
> >> >>>>> >> monitoring from AppDynamics Isolate bottlenecks and diagnose
> >> >>>>> >> root cause in seconds.
> >> >>>>> >> Start your free trial of AppDynamics Pro today!
> >> >>>>> >>
> >> >>>>> >>
> >> >>>>> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140
> >> >>>>> >> /ostg.clktrk _______________________________________________
> >> >>>>> >> Kaldi-users mailing list
> >> >>>>> >> Kal...@li...
> >> >>>>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> >
> >> >>>>> > ----------------------------------------------------------------
> >> >>>>> > -------------- See everything from the browser to the database
> >> >>>>> > with AppDynamics Get end-to-end visibility with application
> >> >>>>> > monitoring from AppDynamics Isolate bottlenecks and diagnose
> >> >>>>> > root cause in seconds.
> >> >>>>> > Start your free trial of AppDynamics Pro today!
> >> >>>>> >
> >> >>>>> >
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/
> >> >>>>> > ostg.clktrk _______________________________________________
> >> >>>>> > Kaldi-users mailing list
> >> >>>>> > Kal...@li...
> >> >>>>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >>>>> >
> >> >>>>>
> >> >>>>>
> >> >>>>> ------------------------------------------------------------------
> >> >>>>> ------------ See everything from the browser to the database with
> >> >>>>> AppDynamics Get end-to-end visibility with application monitoring
> >> >>>>> from AppDynamics Isolate bottlenecks and diagnose root cause in
> >> >>>>> seconds.
> >> >>>>> Start your free trial of AppDynamics Pro today!
> >> >>>>>
> >> >>>>>
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/os
> >> >>>>> tg.clktrk _______________________________________________
> >> >>>>> Kaldi-users mailing list
> >> >>>>> Kal...@li...
> >> >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>> -------------------------------------------------------------------
> >> >>>> ----------- See everything from the browser to the database with
> >> >>>> AppDynamics Get end-to-end visibility with application monitoring
> >> >>>> from AppDynamics Isolate bottlenecks and diagnose root cause in
> >> >>>> seconds.
> >> >>>> Start your free trial of AppDynamics Pro today!
> >> >>>>
> >> >>>>
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ost
> >> >>>> g.clktrk _______________________________________________
> >> >>>> Kaldi-users mailing list
> >> >>>> Kal...@li...
> >> >>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >>>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>> --------------------------------------------------------------------
> >> >>> ---------- See everything from the browser to the database with
> >> >>> AppDynamics Get end-to-end visibility with application monitoring
> >> >>> from AppDynamics Isolate bottlenecks and diagnose root cause in
> >> >>> seconds.
> >> >>> Start your free trial of AppDynamics Pro today!
> >> >>>
> >> >>>
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg
> >> >>> .clktrk _______________________________________________
> >> >>> Kaldi-users mailing list
> >> >>> Kal...@li...
> >> >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >> ---------------------------------------------------------------------
> >> >> --------- See everything from the browser to the database with
> >> >> AppDynamics Get end-to-end visibility with application monitoring
> >> >> from AppDynamics Isolate bottlenecks and diagnose root cause in
> >> >> seconds.
> >> >> Start your free trial of AppDynamics Pro today!
> >> >>
> >> >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg
> .
> >> >> clktrk _______________________________________________
> >> >> Kaldi-users mailing list
> >> >> Kal...@li...
> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >>
> >> >
> >> >
> >> >
> >> > ----------------------------------------------------------------------
> >> > -------- See everything from the browser to the database with
> >> > AppDynamics Get end-to-end visibility with application monitoring from
> >> > AppDynamics Isolate bottlenecks and diagnose root cause in seconds.
> >> > Start your free trial of AppDynamics Pro today!
> >> >
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.c
> >> > lktrk _______________________________________________
> >> > Kaldi-users mailing list
> >> > Kal...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Get your SQL database under version control now!
> >> Version control is standard for application code, but databases havent
> >> caught up. So what steps can you take to put your SQL databases under
> >> version control? Why should you start doing it? Read more to find out.
> >>
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> Kaldi-users mailing list
> >> Kal...@li...
> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Get your SQL database under version control now!
> >> Version control is standard for application code, but databases havent
> >> caught up. So what steps can you take to put your SQL databases under
> >> version control? Why should you start doing it? Read more to find out.
> >>
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> >> _______________________________________________
> >> Kaldi-users mailing list
> >> Kal...@li...
> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > Get your SQL database under version control now!
> > Version control is standard for application code, but databases havent
> > caught up. So what steps can you take to put your SQL databases under
> > version control? Why should you start doing it? Read more to find out.
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Kaldi-users mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >
>
>
> ------------------------------------------------------------------------------
> Get your SQL database under version control now!
> Version control is standard for application code, but databases havent
> caught up. So what steps can you take to put your SQL databases under
> version control? Why should you start doing it? Read more to find out.
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>

Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Hi-- yes there is but it depends what you're doing.  I don't recall
even the approximate percentages of improvement though.  You can get a
temporary research license for MKL for free I think, so you could try
it out.
Dan


On Tue, Jul 30, 2013 at 7:59 PM, Mailing list used for User
Communication and Updates <kal...@li...> wrote:
> Hi,
> This is a bit off-topic, but is there a bit improvement in speed when using
> MKL vs ATLAS?
>
> Thanks
>
> Paul
>
>
> On 30 July 2013 23:29, Mailing list used for User Communication and Updates
> <kal...@li...> wrote:
>>
>> That's right.  They are working on it.
>>
>> --Mike
>>
>>
>> -----Original Message-----
>> From: Mailing list used for User Communication and Updates
>> [mailto:kal...@li...]
>> Sent: Monday, July 29, 2013 9:58 AM
>> To: kal...@li...
>> Subject: Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5
>>
>> I previously spoke to Michael Deisher at Intel about this, he says he has
>> asked the relevant team.  So I think they are aware of the problem.
>> Dan
>>
>>
>> On Mon, Jul 29, 2013 at 12:06 PM, Mailing list used for User Communication
>> and Updates <kal...@li...> wrote:
>> > It has been reported on intel dev forum.
>> > http://software.intel.com/en-us/forums/topic/402436
>> > http://software.intel.com/en-us/forums/topic/402473
>> >
>> > The problem is caused by  the incorrect query result of workspace size
>> > given by dgesvd().
>> >
>> > 11.0.5 seems to be a very buggy version.
>> >
>> >
>> >
>> > On Mon, Jul 29, 2013 at 11:39 PM, Mailing list used for User
>> > Communication and Updates <kal...@li...> wrote:
>> >>
>> >> I just checked different versions of MKL and indeed, this was
>> >> introduced in the  update 5. Anyone with the commercial MKL support
>> >> cares to submit a ticket to Intel?
>> >> y.
>> >>
>> >>
>> >> On Wed, Jul 24, 2013 at 12:42 PM, Mailing list used for User
>> >> Communication and Updates <kal...@li...> wrote:
>> >>>
>> >>> You are right.... I just downgraded to MKL 11.0update1 and passed
>> >>> the test.
>> >>>
>> >>>
>> >>>
>> >>> On Thu, Jul 25, 2013 at 12:28 AM, Mailing list used for User
>> >>> Communication and Updates <kal...@li...> wrote:
>> >>>>
>> >>>> I saw some changes regarding improving performance of *GESVD in the
>> >>>> MKL
>> >>>> 11.0 update5  changelog. I wonder if they messed something up? How
>> >>>> about trying slightly "older" version of the MKL? If you are on the
>> >>>> subscription of Intel Software Network, you can download older
>> >>>> versions of the MKL at no charge...
>> >>>>
>> >>>> We are using Intel 11.0 and the tests pass without any problem...
>> >>>>
>> >>>> yenda
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Wed, Jul 24, 2013 at 12:14 PM, Mailing list used for User
>> >>>> Communication and Updates <kal...@li...> wrote:
>> >>>>>
>> >>>>> I can't see any problems in the Kaldi code.  Perhaps this is an
>> >>>>> older version of MKL?  It could be a bug in MKL itself.  MKL tests
>> >>>>> OK for me.
>> >>>>> Dan
>> >>>>>
>> >>>>>
>> >>>>> On Wed, Jul 24, 2013 at 11:55 AM, Mailing list used for User
>> >>>>> Communication and Updates <kal...@li...> wrote:
>> >>>>> > here's the dbg info copied from eclipse.
>> >>>>> > PS: I can pass the test with atlas provided by centos repo
>> >>>>> > atlas-sse3.x86_64                          3.8.4-2.el6
>> >>>>> > @base
>> >>>>> > atlas-sse3-devel.x86_64                    3.8.4-2.el6
>> >>>>> > @base
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > stack:
>> >>>>> >
>> >>>>> > matrix-lib-test [C/C++ Application] matrix-lib-test [11312]
>> >>>>> > [cores: 1] Thread [1] 11312 [core: 1] (Suspended : Signal :
>> >>>>> > SIGABRT:Aborted)
>> >>>>> > raise() at 0x33090328a5
>> >>>>> > abort() at 0x3309034085
>> >>>>> > kaldi::KaldiAssertFailure_() at kaldi-error.cc:136 0x4cf401
>> >>>>> > kaldi::MatrixBase<double>::LapackGesvd() at kaldi-matrix.cc:401
>> >>>>> > 0x48fdd3
>> >>>>> > kaldi::MatrixBase<double>::DestructiveSvd() at
>> >>>>> > kaldi-matrix.cc:1,330
>> >>>>> > 0x48b902
>> >>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.cc:1,352
>> >>>>> > 0x48b9bb
>> >>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.h:315 0x468e50
>> >>>>> > kaldi::MatrixBase<double>::Cond() at kaldi-matrix.cc:1,222
>> >>>>> > 0x48bf47
>> >>>>> > kaldi::InitRand<double>() at matrix-lib-test.cc:74 0x42dd7e
>> >>>>> > kaldi::UnitTestCompressedMatrix<double>() at
>> >>>>> > matrix-lib-test.cc:3,532 0x43055a
>> >>>>> > kaldi::MatrixUnitTest<double>() at matrix-lib-test.cc:3,705
>> >>>>> > 0x42de8b
>> >>>>> > main() at matrix-lib-test.cc:3,829 0x42dac7
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > variable list after invoking the following    @line393
>> >>>>> > kaldi-matrix.cc
>> >>>>> >   clapack_Xgesvd(v_job, u_job,
>> >>>>> >                  &M, &N, data_, &LDA,
>> >>>>> >                  s->Data(),
>> >>>>> >                  V->Data(), &V_stride,
>> >>>>> >                  U->Data(), &U_stride,
>> >>>>> >                  p_work, &l_work,
>> >>>>> >                  &result);
>> >>>>> >
>> >>>>> >
>> >>>>> > __func__ char [12] 0x4d5418
>> >>>>> > M KaldiBlasInt 1
>> >>>>> > LDA KaldiBlasInt 2
>> >>>>> > U kaldi::MatrixBase<double> * 0x7ffffffface0 data_ double *
>> >>>>> > 0x7413b0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1
>> >>>>> > num_rows_ kaldi::MatrixIndexT 10 stride_ kaldi::MatrixIndexT 2
>> >>>>> > l_work KaldiBlasInt 5 p_work double * 0x741460 *p_work double
>> >>>>> > 1.0829823019173015e-312 temp void * 0x741460 N KaldiBlasInt 10 V
>> >>>>> > kaldi::MatrixBase<double> * 0x7fffffffacc0 data_ double *
>> >>>>> > 0x7406a0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1
>> >>>>> > num_rows_ kaldi::MatrixIndexT 1 stride_ kaldi::MatrixIndexT 2
>> >>>>> > result KaldiBlasInt -13 tmpU kaldi::Matrix<double> {...}
>> >>>>> > V_stride KaldiBlasInt 2 work_query double 5 u_job char *
>> >>>>> > 0x4d4fea "N"
>> >>>>> > tmpV kaldi::Matrix<double> {...} U_stride KaldiBlasInt 2 v_job
>> >>>>> > char * 0x4d4fea "N"
>> >>>>> > this kaldi::MatrixBase<double> * const 0x7fffffffaf80 data_
>> >>>>> > double * 0x741300 *data_ double 0.78239572048187256 num_cols_
>> >>>>> > kaldi::MatrixIndexT 1 num_rows_ kaldi::MatrixIndexT 10 stride_
>> >>>>> > kaldi::MatrixIndexT 2 s kaldi::VectorBase<double> *
>> >>>>> > 0x7fffffffb190 data_ double * 0x7404a0 *data_ double 0 dim_
>> >>>>> > kaldi::MatrixIndexT 1 U_in kaldi::MatrixBase<double> * 0x0 V_in
>> >>>>> > kaldi::MatrixBase<double> * 0x0
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > On Wed, Jul 24, 2013 at 11:04 PM, Mailing list used for User
>> >>>>> > Communication and Updates <kal...@li...>
>> >>>>> > wrote:
>> >>>>> >>
>> >>>>> >> Could you please run this in a debugger and show me the values
>> >>>>> >> of p_work, l_work and result, e.g.
>> >>>>> >>
>> >>>>> >> gdb matrix-lib
>> >>>>> >> (gdb) r
>> >>>>> >> [wait till it crashes]
>> >>>>> >> (gdb) up
>> >>>>> >> (gdb) up    # not sure how many times
>> >>>>> >> (gdb) up
>> >>>>> >> (gdb) p l_work
>> >>>>> >> (gdb) p p_work
>> >>>>> >> (gdb) p result
>> >>>>> >>
>> >>>>> >> Dan
>> >>>>> >>
>> >>>>> >>
>> >>>>> >>
>> >>>>> >> On Wed, Jul 24, 2013 at 10:34 AM, Mailing list used for User
>> >>>>> >> Communication and Updates <kal...@li...>
>> >>>>> >> wrote:
>> >>>>> >> > Hi,
>> >>>>> >> >
>> >>>>> >> > I tried latest trunk version with MKL. I have no problem when
>> >>>>> >> > running `make all`, but get a FAIL when testing. Can anyone
>> >>>>> >> > help? Thanks.
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > svn revision 2720 trunk
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > configuration line:
>> >>>>> >> > $ ./configure --threaded-math=yes --mathlib=MKL
>> >>>>> >> > --use-cuda=yes --mkl-root=$MKLROOT
>> >>>>> >> > --omp-libdir=/opt/intel/lib/intel64
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > MKL version 11.0 update 5 (latest)
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > Linux version CentOS 6.4
>> >>>>> >> > $ uname -a
>> >>>>> >> > Linux 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36
>> >>>>> >> > UTC
>> >>>>> >> > 2013
>> >>>>> >> > x86_64
>> >>>>> >> > x86_64 x86_64 GNU/Linux
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > $ gcc -v
>> >>>>> >> > Using built-in specs.
>> >>>>> >> > Target: x86_64-redhat-linux
>> >>>>> >> > Configured with: ../configure --prefix=/usr
>> >>>>> >> > --mandir=/usr/share/man --infodir=/usr/share/info
>> >>>>> >> > --with-bugurl=http://bugzilla.redhat.com/bugzilla
>> >>>>> >> > --enable-bootstrap --enable-shared --enable-threads=posix
>> >>>>> >> > --enable-checking=release --with-system-zlib
>> >>>>> >> > --enable-__cxa_atexit --disable-libunwind-exceptions
>> >>>>> >> > --enable-gnu-unique-object
>> >>>>> >> > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada
>> >>>>> >> > --enable-java-awt=gtk
>> >>>>> >> > --disable-dssi
>> >>>>> >> > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre
>> >>>>> >> > --enable-libgcj-multifile --enable-java-maintainer-mode
>> >>>>> >> > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar
>> >>>>> >> > --disable-libjava-multilib
>> >>>>> >> > --with-ppl --with-cloog --with-tune=generic
>> >>>>> >> > --with-arch_32=i686 --build=x86_64-redhat-linux Thread model:
>> >>>>> >> > posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > ~/kaldi/trunk/src/matrix
>> >>>>> >> > $ make test
>> >>>>> >> > Running matrix-lib-test .../bin/sh: line 1:  5561 Aborted
>> >>>>> >> > (core dumped) ./$x > /dev/null 2>&1 ... FAIL Running
>> >>>>> >> > kaldi-gpsr-test ...... SUCCESS
>> >>>>> >> > make: *** [test] Error 1
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > $ ./matrix-lib-test
>> >>>>> >> > ...
>> >>>>> >> > ...
>> >>>>> >> > VLOG[2] (UnitTestLbfgs():matrix-lib-test.cc:2734) L-BFGS
>> >>>>> >> > finished after
>> >>>>> >> > 59
>> >>>>> >> > function evaluations.
>> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3623) M =
>> >>>>> >> > [ ]
>> >>>>> >> >
>> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3624) M2 =
>> >>>>> >> > [ ]
>> >>>>> >> >
>> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3626)
>> >>>>> >> > Compressed
>> >>>>> >> > matrix,
>> >>>>> >> > tot = 0, diff = 0
>> >>>>> >> >
>> >>>>> >> > MKL ERROR: Parameter 13 was incorrect on entry to DGESVD.
>> >>>>> >> > KALDI_ASSERT: at LapackGesvd:kaldi-matrix.cc:401, failed:
>> >>>>> >> > result
>> >>>>> >> > >= 0 &&
>> >>>>> >> > "Call to CLAPACK dgesvd_ called with wrong arguments"
>> >>>>> >> > Stack trace is:
>> >>>>> >> > kaldi::KaldiGetStackTrace()
>> >>>>> >> > kaldi::KaldiAssertFailure_(char const*, char const*, int,
>> >>>>> >> > char
>> >>>>> >> > const*)
>> >>>>> >> > kaldi::MatrixBase<double>::LapackGesvd(kaldi::VectorBase<doub
>> >>>>> >> > le>*, kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*)
>> >>>>> >> >
>> >>>>> >> > kaldi::MatrixBase<double>::DestructiveSvd(kaldi::VectorBase<d
>> >>>>> >> > ouble>*, kaldi::MatrixBase<double>*,
>> >>>>> >> > kaldi::MatrixBase<double>*)
>> >>>>> >> > kaldi::MatrixBase<double>::Svd(kaldi::VectorBase<double>*,
>> >>>>> >> > kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*) const
>> >>>>> >> > .
>> >>>>> >> > .
>> >>>>> >> > .
>> >>>>> >> > ./matrix-lib-test() [0x43055a]
>> >>>>> >> > ./matrix-lib-test() [0x42de8b]
>> >>>>> >> > ./matrix-lib-test(main+0x1d) [0x42dac7]
>> >>>>> >> > /lib64/libc.so.6(__libc_start_main+0xfd) [0x330901ecdd]
>> >>>>> >> > ./matrix-lib-test() [0x42ced9] Aborted (core dumped)
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > -------------------------------------------------------------
>> >>>>> >> > ----------------- See everything from the browser to the
>> >>>>> >> > database with AppDynamics Get end-to-end visibility with
>> >>>>> >> > application monitoring from AppDynamics Isolate bottlenecks
>> >>>>> >> > and diagnose root cause in seconds.
>> >>>>> >> > Start your free trial of AppDynamics Pro today!
>> >>>>> >> >
>> >>>>> >> >
>> >>>>> >> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/41
>> >>>>> >> > 40/ostg.clktrk
>> >>>>> >> > _______________________________________________
>> >>>>> >> > Kaldi-users mailing list
>> >>>>> >> > Kal...@li...
>> >>>>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >>>>> >> >
>> >>>>> >>
>> >>>>> >>
>> >>>>> >>
>> >>>>> >> ---------------------------------------------------------------
>> >>>>> >> --------------- See everything from the browser to the database
>> >>>>> >> with AppDynamics Get end-to-end visibility with application
>> >>>>> >> monitoring from AppDynamics Isolate bottlenecks and diagnose
>> >>>>> >> root cause in seconds.
>> >>>>> >> Start your free trial of AppDynamics Pro today!
>> >>>>> >>
>> >>>>> >>
>> >>>>> >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140
>> >>>>> >> /ostg.clktrk _______________________________________________
>> >>>>> >> Kaldi-users mailing list
>> >>>>> >> Kal...@li...
>> >>>>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> >
>> >>>>> > ----------------------------------------------------------------
>> >>>>> > -------------- See everything from the browser to the database
>> >>>>> > with AppDynamics Get end-to-end visibility with application
>> >>>>> > monitoring from AppDynamics Isolate bottlenecks and diagnose
>> >>>>> > root cause in seconds.
>> >>>>> > Start your free trial of AppDynamics Pro today!
>> >>>>> >
>> >>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/
>> >>>>> > ostg.clktrk _______________________________________________
>> >>>>> > Kaldi-users mailing list
>> >>>>> > Kal...@li...
>> >>>>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >>>>> >
>> >>>>>
>> >>>>>
>> >>>>> ------------------------------------------------------------------
>> >>>>> ------------ See everything from the browser to the database with
>> >>>>> AppDynamics Get end-to-end visibility with application monitoring
>> >>>>> from AppDynamics Isolate bottlenecks and diagnose root cause in
>> >>>>> seconds.
>> >>>>> Start your free trial of AppDynamics Pro today!
>> >>>>>
>> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/os
>> >>>>> tg.clktrk _______________________________________________
>> >>>>> Kaldi-users mailing list
>> >>>>> Kal...@li...
>> >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> -------------------------------------------------------------------
>> >>>> ----------- See everything from the browser to the database with
>> >>>> AppDynamics Get end-to-end visibility with application monitoring
>> >>>> from AppDynamics Isolate bottlenecks and diagnose root cause in
>> >>>> seconds.
>> >>>> Start your free trial of AppDynamics Pro today!
>> >>>>
>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ost
>> >>>> g.clktrk _______________________________________________
>> >>>> Kaldi-users mailing list
>> >>>> Kal...@li...
>> >>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >>>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> --------------------------------------------------------------------
>> >>> ---------- See everything from the browser to the database with
>> >>> AppDynamics Get end-to-end visibility with application monitoring
>> >>> from AppDynamics Isolate bottlenecks and diagnose root cause in
>> >>> seconds.
>> >>> Start your free trial of AppDynamics Pro today!
>> >>>
>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg
>> >>> .clktrk _______________________________________________
>> >>> Kaldi-users mailing list
>> >>> Kal...@li...
>> >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >>>
>> >>
>> >>
>> >>
>> >> ---------------------------------------------------------------------
>> >> --------- See everything from the browser to the database with
>> >> AppDynamics Get end-to-end visibility with application monitoring
>> >> from AppDynamics Isolate bottlenecks and diagnose root cause in
>> >> seconds.
>> >> Start your free trial of AppDynamics Pro today!
>> >>
>> >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.
>> >> clktrk _______________________________________________
>> >> Kaldi-users mailing list
>> >> Kal...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >>
>> >
>> >
>> >
>> > ----------------------------------------------------------------------
>> > -------- See everything from the browser to the database with
>> > AppDynamics Get end-to-end visibility with application monitoring from
>> > AppDynamics Isolate bottlenecks and diagnose root cause in seconds.
>> > Start your free trial of AppDynamics Pro today!
>> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.c
>> > lktrk _______________________________________________
>> > Kaldi-users mailing list
>> > Kal...@li...
>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Get your SQL database under version control now!
>> Version control is standard for application code, but databases havent
>> caught up. So what steps can you take to put your SQL databases under
>> version control? Why should you start doing it? Read more to find out.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>
>>
>> ------------------------------------------------------------------------------
>> Get your SQL database under version control now!
>> Version control is standard for application code, but databases havent
>> caught up. So what steps can you take to put your SQL databases under
>> version control? Why should you start doing it? Read more to find out.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>
>
>
> ------------------------------------------------------------------------------
> Get your SQL database under version control now!
> Version control is standard for application code, but databases havent
> caught up. So what steps can you take to put your SQL databases under
> version control? Why should you start doing it? Read more to find out.
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>

Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Hi,
This is a bit off-topic, but is there a bit improvement in speed when using
MKL vs ATLAS?

Thanks

Paul


On 30 July 2013 23:29, Mailing list used for User Communication and Updates
<kal...@li...> wrote:

> That's right.  They are working on it.
>
> --Mike
>
>
> -----Original Message-----
> From: Mailing list used for User Communication and Updates [mailto:
> kal...@li...]
> Sent: Monday, July 29, 2013 9:58 AM
> To: kal...@li...
> Subject: Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5
>
> I previously spoke to Michael Deisher at Intel about this, he says he has
> asked the relevant team.  So I think they are aware of the problem.
> Dan
>
>
> On Mon, Jul 29, 2013 at 12:06 PM, Mailing list used for User Communication
> and Updates <kal...@li...> wrote:
> > It has been reported on intel dev forum.
> > http://software.intel.com/en-us/forums/topic/402436
> > http://software.intel.com/en-us/forums/topic/402473
> >
> > The problem is caused by  the incorrect query result of workspace size
> > given by dgesvd().
> >
> > 11.0.5 seems to be a very buggy version.
> >
> >
> >
> > On Mon, Jul 29, 2013 at 11:39 PM, Mailing list used for User
> > Communication and Updates <kal...@li...> wrote:
> >>
> >> I just checked different versions of MKL and indeed, this was
> >> introduced in the  update 5. Anyone with the commercial MKL support
> >> cares to submit a ticket to Intel?
> >> y.
> >>
> >>
> >> On Wed, Jul 24, 2013 at 12:42 PM, Mailing list used for User
> >> Communication and Updates <kal...@li...> wrote:
> >>>
> >>> You are right.... I just downgraded to MKL 11.0update1 and passed
> >>> the test.
> >>>
> >>>
> >>>
> >>> On Thu, Jul 25, 2013 at 12:28 AM, Mailing list used for User
> >>> Communication and Updates <kal...@li...> wrote:
> >>>>
> >>>> I saw some changes regarding improving performance of *GESVD in the
> >>>> MKL
> >>>> 11.0 update5  changelog. I wonder if they messed something up? How
> >>>> about trying slightly "older" version of the MKL? If you are on the
> >>>> subscription of Intel Software Network, you can download older
> >>>> versions of the MKL at no charge...
> >>>>
> >>>> We are using Intel 11.0 and the tests pass without any problem...
> >>>>
> >>>> yenda
> >>>>
> >>>>
> >>>>
> >>>> On Wed, Jul 24, 2013 at 12:14 PM, Mailing list used for User
> >>>> Communication and Updates <kal...@li...> wrote:
> >>>>>
> >>>>> I can't see any problems in the Kaldi code.  Perhaps this is an
> >>>>> older version of MKL?  It could be a bug in MKL itself.  MKL tests
> >>>>> OK for me.
> >>>>> Dan
> >>>>>
> >>>>>
> >>>>> On Wed, Jul 24, 2013 at 11:55 AM, Mailing list used for User
> >>>>> Communication and Updates <kal...@li...> wrote:
> >>>>> > here's the dbg info copied from eclipse.
> >>>>> > PS: I can pass the test with atlas provided by centos repo
> >>>>> > atlas-sse3.x86_64                          3.8.4-2.el6
> >>>>> > @base
> >>>>> > atlas-sse3-devel.x86_64                    3.8.4-2.el6
> >>>>> > @base
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > stack:
> >>>>> >
> >>>>> > matrix-lib-test [C/C++ Application] matrix-lib-test [11312]
> >>>>> > [cores: 1] Thread [1] 11312 [core: 1] (Suspended : Signal :
> >>>>> > SIGABRT:Aborted)
> >>>>> > raise() at 0x33090328a5
> >>>>> > abort() at 0x3309034085
> >>>>> > kaldi::KaldiAssertFailure_() at kaldi-error.cc:136 0x4cf401
> >>>>> > kaldi::MatrixBase<double>::LapackGesvd() at kaldi-matrix.cc:401
> >>>>> > 0x48fdd3
> >>>>> > kaldi::MatrixBase<double>::DestructiveSvd() at
> >>>>> > kaldi-matrix.cc:1,330
> >>>>> > 0x48b902
> >>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.cc:1,352
> >>>>> > 0x48b9bb
> >>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.h:315 0x468e50
> >>>>> > kaldi::MatrixBase<double>::Cond() at kaldi-matrix.cc:1,222
> >>>>> > 0x48bf47
> >>>>> > kaldi::InitRand<double>() at matrix-lib-test.cc:74 0x42dd7e
> >>>>> > kaldi::UnitTestCompressedMatrix<double>() at
> >>>>> > matrix-lib-test.cc:3,532 0x43055a
> >>>>> > kaldi::MatrixUnitTest<double>() at matrix-lib-test.cc:3,705
> >>>>> > 0x42de8b
> >>>>> > main() at matrix-lib-test.cc:3,829 0x42dac7
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > variable list after invoking the following    @line393
> >>>>> > kaldi-matrix.cc
> >>>>> >   clapack_Xgesvd(v_job, u_job,
> >>>>> >                  &M, &N, data_, &LDA,
> >>>>> >                  s->Data(),
> >>>>> >                  V->Data(), &V_stride,
> >>>>> >                  U->Data(), &U_stride,
> >>>>> >                  p_work, &l_work,
> >>>>> >                  &result);
> >>>>> >
> >>>>> >
> >>>>> > __func__ char [12] 0x4d5418
> >>>>> > M KaldiBlasInt 1
> >>>>> > LDA KaldiBlasInt 2
> >>>>> > U kaldi::MatrixBase<double> * 0x7ffffffface0 data_ double *
> >>>>> > 0x7413b0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1
> >>>>> > num_rows_ kaldi::MatrixIndexT 10 stride_ kaldi::MatrixIndexT 2
> >>>>> > l_work KaldiBlasInt 5 p_work double * 0x741460 *p_work double
> >>>>> > 1.0829823019173015e-312 temp void * 0x741460 N KaldiBlasInt 10 V
> >>>>> > kaldi::MatrixBase<double> * 0x7fffffffacc0 data_ double *
> >>>>> > 0x7406a0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1
> >>>>> > num_rows_ kaldi::MatrixIndexT 1 stride_ kaldi::MatrixIndexT 2
> >>>>> > result KaldiBlasInt -13 tmpU kaldi::Matrix<double> {...}
> >>>>> > V_stride KaldiBlasInt 2 work_query double 5 u_job char *
> >>>>> > 0x4d4fea "N"
> >>>>> > tmpV kaldi::Matrix<double> {...} U_stride KaldiBlasInt 2 v_job
> >>>>> > char * 0x4d4fea "N"
> >>>>> > this kaldi::MatrixBase<double> * const 0x7fffffffaf80 data_
> >>>>> > double * 0x741300 *data_ double 0.78239572048187256 num_cols_
> >>>>> > kaldi::MatrixIndexT 1 num_rows_ kaldi::MatrixIndexT 10 stride_
> >>>>> > kaldi::MatrixIndexT 2 s kaldi::VectorBase<double> *
> >>>>> > 0x7fffffffb190 data_ double * 0x7404a0 *data_ double 0 dim_
> >>>>> > kaldi::MatrixIndexT 1 U_in kaldi::MatrixBase<double> * 0x0 V_in
> >>>>> > kaldi::MatrixBase<double> * 0x0
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > On Wed, Jul 24, 2013 at 11:04 PM, Mailing list used for User
> >>>>> > Communication and Updates <kal...@li...>
> >>>>> > wrote:
> >>>>> >>
> >>>>> >> Could you please run this in a debugger and show me the values
> >>>>> >> of p_work, l_work and result, e.g.
> >>>>> >>
> >>>>> >> gdb matrix-lib
> >>>>> >> (gdb) r
> >>>>> >> [wait till it crashes]
> >>>>> >> (gdb) up
> >>>>> >> (gdb) up    # not sure how many times
> >>>>> >> (gdb) up
> >>>>> >> (gdb) p l_work
> >>>>> >> (gdb) p p_work
> >>>>> >> (gdb) p result
> >>>>> >>
> >>>>> >> Dan
> >>>>> >>
> >>>>> >>
> >>>>> >>
> >>>>> >> On Wed, Jul 24, 2013 at 10:34 AM, Mailing list used for User
> >>>>> >> Communication and Updates <kal...@li...>
> wrote:
> >>>>> >> > Hi,
> >>>>> >> >
> >>>>> >> > I tried latest trunk version with MKL. I have no problem when
> >>>>> >> > running `make all`, but get a FAIL when testing. Can anyone
> >>>>> >> > help? Thanks.
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > svn revision 2720 trunk
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > configuration line:
> >>>>> >> > $ ./configure --threaded-math=yes --mathlib=MKL
> >>>>> >> > --use-cuda=yes --mkl-root=$MKLROOT
> >>>>> >> > --omp-libdir=/opt/intel/lib/intel64
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > MKL version 11.0 update 5 (latest)
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > Linux version CentOS 6.4
> >>>>> >> > $ uname -a
> >>>>> >> > Linux 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36
> >>>>> >> > UTC
> >>>>> >> > 2013
> >>>>> >> > x86_64
> >>>>> >> > x86_64 x86_64 GNU/Linux
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > $ gcc -v
> >>>>> >> > Using built-in specs.
> >>>>> >> > Target: x86_64-redhat-linux
> >>>>> >> > Configured with: ../configure --prefix=/usr
> >>>>> >> > --mandir=/usr/share/man --infodir=/usr/share/info
> >>>>> >> > --with-bugurl=http://bugzilla.redhat.com/bugzilla
> >>>>> >> > --enable-bootstrap --enable-shared --enable-threads=posix
> >>>>> >> > --enable-checking=release --with-system-zlib
> >>>>> >> > --enable-__cxa_atexit --disable-libunwind-exceptions
> >>>>> >> > --enable-gnu-unique-object
> >>>>> >> > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada
> >>>>> >> > --enable-java-awt=gtk
> >>>>> >> > --disable-dssi
> >>>>> >> > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre
> >>>>> >> > --enable-libgcj-multifile --enable-java-maintainer-mode
> >>>>> >> > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar
> >>>>> >> > --disable-libjava-multilib
> >>>>> >> > --with-ppl --with-cloog --with-tune=generic
> >>>>> >> > --with-arch_32=i686 --build=x86_64-redhat-linux Thread model:
> >>>>> >> > posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > ~/kaldi/trunk/src/matrix
> >>>>> >> > $ make test
> >>>>> >> > Running matrix-lib-test .../bin/sh: line 1:  5561 Aborted
> >>>>> >> > (core dumped) ./$x > /dev/null 2>&1 ... FAIL Running
> >>>>> >> > kaldi-gpsr-test ...... SUCCESS
> >>>>> >> > make: *** [test] Error 1
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > $ ./matrix-lib-test
> >>>>> >> > ...
> >>>>> >> > ...
> >>>>> >> > VLOG[2] (UnitTestLbfgs():matrix-lib-test.cc:2734) L-BFGS
> >>>>> >> > finished after
> >>>>> >> > 59
> >>>>> >> > function evaluations.
> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3623) M =
> >>>>> >> > [ ]
> >>>>> >> >
> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3624) M2 =
> >>>>> >> > [ ]
> >>>>> >> >
> >>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3626)
> >>>>> >> > Compressed
> >>>>> >> > matrix,
> >>>>> >> > tot = 0, diff = 0
> >>>>> >> >
> >>>>> >> > MKL ERROR: Parameter 13 was incorrect on entry to DGESVD.
> >>>>> >> > KALDI_ASSERT: at LapackGesvd:kaldi-matrix.cc:401, failed:
> >>>>> >> > result
> >>>>> >> > >= 0 &&
> >>>>> >> > "Call to CLAPACK dgesvd_ called with wrong arguments"
> >>>>> >> > Stack trace is:
> >>>>> >> > kaldi::KaldiGetStackTrace()
> >>>>> >> > kaldi::KaldiAssertFailure_(char const*, char const*, int,
> >>>>> >> > char
> >>>>> >> > const*)
> >>>>> >> > kaldi::MatrixBase<double>::LapackGesvd(kaldi::VectorBase<doub
> >>>>> >> > le>*, kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*)
> >>>>> >> >
> >>>>> >> > kaldi::MatrixBase<double>::DestructiveSvd(kaldi::VectorBase<d
> >>>>> >> > ouble>*, kaldi::MatrixBase<double>*,
> >>>>> >> > kaldi::MatrixBase<double>*)
> >>>>> >> > kaldi::MatrixBase<double>::Svd(kaldi::VectorBase<double>*,
> >>>>> >> > kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*) const
> >>>>> >> > .
> >>>>> >> > .
> >>>>> >> > .
> >>>>> >> > ./matrix-lib-test() [0x43055a]
> >>>>> >> > ./matrix-lib-test() [0x42de8b]
> >>>>> >> > ./matrix-lib-test(main+0x1d) [0x42dac7]
> >>>>> >> > /lib64/libc.so.6(__libc_start_main+0xfd) [0x330901ecdd]
> >>>>> >> > ./matrix-lib-test() [0x42ced9] Aborted (core dumped)
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > -------------------------------------------------------------
> >>>>> >> > ----------------- See everything from the browser to the
> >>>>> >> > database with AppDynamics Get end-to-end visibility with
> >>>>> >> > application monitoring from AppDynamics Isolate bottlenecks
> >>>>> >> > and diagnose root cause in seconds.
> >>>>> >> > Start your free trial of AppDynamics Pro today!
> >>>>> >> >
> >>>>> >> >
> >>>>> >> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/41
> >>>>> >> > 40/ostg.clktrk
> >>>>> >> > _______________________________________________
> >>>>> >> > Kaldi-users mailing list
> >>>>> >> > Kal...@li...
> >>>>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >>>>> >> >
> >>>>> >>
> >>>>> >>
> >>>>> >>
> >>>>> >> ---------------------------------------------------------------
> >>>>> >> --------------- See everything from the browser to the database
> >>>>> >> with AppDynamics Get end-to-end visibility with application
> >>>>> >> monitoring from AppDynamics Isolate bottlenecks and diagnose
> >>>>> >> root cause in seconds.
> >>>>> >> Start your free trial of AppDynamics Pro today!
> >>>>> >>
> >>>>> >>
> >>>>> >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140
> >>>>> >> /ostg.clktrk _______________________________________________
> >>>>> >> Kaldi-users mailing list
> >>>>> >> Kal...@li...
> >>>>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> >
> >>>>> > ----------------------------------------------------------------
> >>>>> > -------------- See everything from the browser to the database
> >>>>> > with AppDynamics Get end-to-end visibility with application
> >>>>> > monitoring from AppDynamics Isolate bottlenecks and diagnose
> >>>>> > root cause in seconds.
> >>>>> > Start your free trial of AppDynamics Pro today!
> >>>>> >
> >>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/
> >>>>> > ostg.clktrk _______________________________________________
> >>>>> > Kaldi-users mailing list
> >>>>> > Kal...@li...
> >>>>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >>>>> >
> >>>>>
> >>>>>
> >>>>> ------------------------------------------------------------------
> >>>>> ------------ See everything from the browser to the database with
> >>>>> AppDynamics Get end-to-end visibility with application monitoring
> >>>>> from AppDynamics Isolate bottlenecks and diagnose root cause in
> >>>>> seconds.
> >>>>> Start your free trial of AppDynamics Pro today!
> >>>>>
> >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/os
> >>>>> tg.clktrk _______________________________________________
> >>>>> Kaldi-users mailing list
> >>>>> Kal...@li...
> >>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> -------------------------------------------------------------------
> >>>> ----------- See everything from the browser to the database with
> >>>> AppDynamics Get end-to-end visibility with application monitoring
> >>>> from AppDynamics Isolate bottlenecks and diagnose root cause in
> >>>> seconds.
> >>>> Start your free trial of AppDynamics Pro today!
> >>>>
> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ost
> >>>> g.clktrk _______________________________________________
> >>>> Kaldi-users mailing list
> >>>> Kal...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> --------------------------------------------------------------------
> >>> ---------- See everything from the browser to the database with
> >>> AppDynamics Get end-to-end visibility with application monitoring
> >>> from AppDynamics Isolate bottlenecks and diagnose root cause in
> >>> seconds.
> >>> Start your free trial of AppDynamics Pro today!
> >>>
> >>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg
> >>> .clktrk _______________________________________________
> >>> Kaldi-users mailing list
> >>> Kal...@li...
> >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >>>
> >>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> --------- See everything from the browser to the database with
> >> AppDynamics Get end-to-end visibility with application monitoring
> >> from AppDynamics Isolate bottlenecks and diagnose root cause in
> >> seconds.
> >> Start your free trial of AppDynamics Pro today!
> >>
> >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.
> >> clktrk _______________________________________________
> >> Kaldi-users mailing list
> >> Kal...@li...
> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >>
> >
> >
> >
> > ----------------------------------------------------------------------
> > -------- See everything from the browser to the database with
> > AppDynamics Get end-to-end visibility with application monitoring from
> > AppDynamics Isolate bottlenecks and diagnose root cause in seconds.
> > Start your free trial of AppDynamics Pro today!
> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.c
> > lktrk _______________________________________________
> > Kaldi-users mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >
>
>
> ------------------------------------------------------------------------------
> Get your SQL database under version control now!
> Version control is standard for application code, but databases havent
> caught up. So what steps can you take to put your SQL databases under
> version control? Why should you start doing it? Read more to find out.
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>
>
> ------------------------------------------------------------------------------
> Get your SQL database under version control now!
> Version control is standard for application code, but databases havent
> caught up. So what steps can you take to put your SQL databases under
> version control? Why should you start doing it? Read more to find out.
> http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>

Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

That's right.  They are working on it.

--Mike


-----Original Message-----
From: Mailing list used for User Communication and Updates [mailto:kal...@li...] 
Sent: Monday, July 29, 2013 9:58 AM
To: kal...@li...
Subject: Re: [Kaldi-users] matrix-lib-test test fail with MKL 11.0 update5

I previously spoke to Michael Deisher at Intel about this, he says he has asked the relevant team.  So I think they are aware of the problem.
Dan


On Mon, Jul 29, 2013 at 12:06 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote:
> It has been reported on intel dev forum.
> http://software.intel.com/en-us/forums/topic/402436
> http://software.intel.com/en-us/forums/topic/402473
>
> The problem is caused by  the incorrect query result of workspace size 
> given by dgesvd().
>
> 11.0.5 seems to be a very buggy version.
>
>
>
> On Mon, Jul 29, 2013 at 11:39 PM, Mailing list used for User 
> Communication and Updates <kal...@li...> wrote:
>>
>> I just checked different versions of MKL and indeed, this was 
>> introduced in the  update 5. Anyone with the commercial MKL support 
>> cares to submit a ticket to Intel?
>> y.
>>
>>
>> On Wed, Jul 24, 2013 at 12:42 PM, Mailing list used for User 
>> Communication and Updates <kal...@li...> wrote:
>>>
>>> You are right.... I just downgraded to MKL 11.0update1 and passed 
>>> the test.
>>>
>>>
>>>
>>> On Thu, Jul 25, 2013 at 12:28 AM, Mailing list used for User 
>>> Communication and Updates <kal...@li...> wrote:
>>>>
>>>> I saw some changes regarding improving performance of *GESVD in the 
>>>> MKL
>>>> 11.0 update5  changelog. I wonder if they messed something up? How 
>>>> about trying slightly "older" version of the MKL? If you are on the 
>>>> subscription of Intel Software Network, you can download older 
>>>> versions of the MKL at no charge...
>>>>
>>>> We are using Intel 11.0 and the tests pass without any problem...
>>>>
>>>> yenda
>>>>
>>>>
>>>>
>>>> On Wed, Jul 24, 2013 at 12:14 PM, Mailing list used for User 
>>>> Communication and Updates <kal...@li...> wrote:
>>>>>
>>>>> I can't see any problems in the Kaldi code.  Perhaps this is an 
>>>>> older version of MKL?  It could be a bug in MKL itself.  MKL tests 
>>>>> OK for me.
>>>>> Dan
>>>>>
>>>>>
>>>>> On Wed, Jul 24, 2013 at 11:55 AM, Mailing list used for User 
>>>>> Communication and Updates <kal...@li...> wrote:
>>>>> > here's the dbg info copied from eclipse.
>>>>> > PS: I can pass the test with atlas provided by centos repo
>>>>> > atlas-sse3.x86_64                          3.8.4-2.el6
>>>>> > @base
>>>>> > atlas-sse3-devel.x86_64                    3.8.4-2.el6
>>>>> > @base
>>>>> >
>>>>> >
>>>>> >
>>>>> > stack:
>>>>> >
>>>>> > matrix-lib-test [C/C++ Application] matrix-lib-test [11312] 
>>>>> > [cores: 1] Thread [1] 11312 [core: 1] (Suspended : Signal : 
>>>>> > SIGABRT:Aborted)
>>>>> > raise() at 0x33090328a5
>>>>> > abort() at 0x3309034085
>>>>> > kaldi::KaldiAssertFailure_() at kaldi-error.cc:136 0x4cf401
>>>>> > kaldi::MatrixBase<double>::LapackGesvd() at kaldi-matrix.cc:401
>>>>> > 0x48fdd3
>>>>> > kaldi::MatrixBase<double>::DestructiveSvd() at 
>>>>> > kaldi-matrix.cc:1,330
>>>>> > 0x48b902
>>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.cc:1,352 
>>>>> > 0x48b9bb
>>>>> > kaldi::MatrixBase<double>::Svd() at kaldi-matrix.h:315 0x468e50
>>>>> > kaldi::MatrixBase<double>::Cond() at kaldi-matrix.cc:1,222 
>>>>> > 0x48bf47
>>>>> > kaldi::InitRand<double>() at matrix-lib-test.cc:74 0x42dd7e
>>>>> > kaldi::UnitTestCompressedMatrix<double>() at 
>>>>> > matrix-lib-test.cc:3,532 0x43055a
>>>>> > kaldi::MatrixUnitTest<double>() at matrix-lib-test.cc:3,705 
>>>>> > 0x42de8b
>>>>> > main() at matrix-lib-test.cc:3,829 0x42dac7
>>>>> >
>>>>> >
>>>>> >
>>>>> > variable list after invoking the following    @line393
>>>>> > kaldi-matrix.cc
>>>>> >   clapack_Xgesvd(v_job, u_job,
>>>>> >                  &M, &N, data_, &LDA,
>>>>> >                  s->Data(),
>>>>> >                  V->Data(), &V_stride,
>>>>> >                  U->Data(), &U_stride,
>>>>> >                  p_work, &l_work,
>>>>> >                  &result);
>>>>> >
>>>>> >
>>>>> > __func__ char [12] 0x4d5418
>>>>> > M KaldiBlasInt 1
>>>>> > LDA KaldiBlasInt 2
>>>>> > U kaldi::MatrixBase<double> * 0x7ffffffface0 data_ double * 
>>>>> > 0x7413b0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1 
>>>>> > num_rows_ kaldi::MatrixIndexT 10 stride_ kaldi::MatrixIndexT 2 
>>>>> > l_work KaldiBlasInt 5 p_work double * 0x741460 *p_work double 
>>>>> > 1.0829823019173015e-312 temp void * 0x741460 N KaldiBlasInt 10 V 
>>>>> > kaldi::MatrixBase<double> * 0x7fffffffacc0 data_ double * 
>>>>> > 0x7406a0 *data_ double 0 num_cols_ kaldi::MatrixIndexT 1 
>>>>> > num_rows_ kaldi::MatrixIndexT 1 stride_ kaldi::MatrixIndexT 2 
>>>>> > result KaldiBlasInt -13 tmpU kaldi::Matrix<double> {...} 
>>>>> > V_stride KaldiBlasInt 2 work_query double 5 u_job char * 
>>>>> > 0x4d4fea "N"
>>>>> > tmpV kaldi::Matrix<double> {...} U_stride KaldiBlasInt 2 v_job 
>>>>> > char * 0x4d4fea "N"
>>>>> > this kaldi::MatrixBase<double> * const 0x7fffffffaf80 data_ 
>>>>> > double * 0x741300 *data_ double 0.78239572048187256 num_cols_ 
>>>>> > kaldi::MatrixIndexT 1 num_rows_ kaldi::MatrixIndexT 10 stride_ 
>>>>> > kaldi::MatrixIndexT 2 s kaldi::VectorBase<double> * 
>>>>> > 0x7fffffffb190 data_ double * 0x7404a0 *data_ double 0 dim_ 
>>>>> > kaldi::MatrixIndexT 1 U_in kaldi::MatrixBase<double> * 0x0 V_in 
>>>>> > kaldi::MatrixBase<double> * 0x0
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Wed, Jul 24, 2013 at 11:04 PM, Mailing list used for User 
>>>>> > Communication and Updates <kal...@li...> 
>>>>> > wrote:
>>>>> >>
>>>>> >> Could you please run this in a debugger and show me the values 
>>>>> >> of p_work, l_work and result, e.g.
>>>>> >>
>>>>> >> gdb matrix-lib
>>>>> >> (gdb) r
>>>>> >> [wait till it crashes]
>>>>> >> (gdb) up
>>>>> >> (gdb) up    # not sure how many times
>>>>> >> (gdb) up
>>>>> >> (gdb) p l_work
>>>>> >> (gdb) p p_work
>>>>> >> (gdb) p result
>>>>> >>
>>>>> >> Dan
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> On Wed, Jul 24, 2013 at 10:34 AM, Mailing list used for User 
>>>>> >> Communication and Updates <kal...@li...> wrote:
>>>>> >> > Hi,
>>>>> >> >
>>>>> >> > I tried latest trunk version with MKL. I have no problem when 
>>>>> >> > running `make all`, but get a FAIL when testing. Can anyone 
>>>>> >> > help? Thanks.
>>>>> >> >
>>>>> >> >
>>>>> >> > svn revision 2720 trunk
>>>>> >> >
>>>>> >> >
>>>>> >> > configuration line:
>>>>> >> > $ ./configure --threaded-math=yes --mathlib=MKL 
>>>>> >> > --use-cuda=yes --mkl-root=$MKLROOT 
>>>>> >> > --omp-libdir=/opt/intel/lib/intel64
>>>>> >> >
>>>>> >> >
>>>>> >> > MKL version 11.0 update 5 (latest)
>>>>> >> >
>>>>> >> >
>>>>> >> > Linux version CentOS 6.4
>>>>> >> > $ uname -a
>>>>> >> > Linux 2.6.32-358.6.2.el6.x86_64 #1 SMP Thu May 16 20:59:36 
>>>>> >> > UTC
>>>>> >> > 2013
>>>>> >> > x86_64
>>>>> >> > x86_64 x86_64 GNU/Linux
>>>>> >> >
>>>>> >> >
>>>>> >> > $ gcc -v
>>>>> >> > Using built-in specs.
>>>>> >> > Target: x86_64-redhat-linux
>>>>> >> > Configured with: ../configure --prefix=/usr 
>>>>> >> > --mandir=/usr/share/man --infodir=/usr/share/info 
>>>>> >> > --with-bugurl=http://bugzilla.redhat.com/bugzilla
>>>>> >> > --enable-bootstrap --enable-shared --enable-threads=posix 
>>>>> >> > --enable-checking=release --with-system-zlib 
>>>>> >> > --enable-__cxa_atexit --disable-libunwind-exceptions 
>>>>> >> > --enable-gnu-unique-object 
>>>>> >> > --enable-languages=c,c++,objc,obj-c++,java,fortran,ada
>>>>> >> > --enable-java-awt=gtk
>>>>> >> > --disable-dssi
>>>>> >> > --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre
>>>>> >> > --enable-libgcj-multifile --enable-java-maintainer-mode 
>>>>> >> > --with-ecj-jar=/usr/share/java/eclipse-ecj.jar
>>>>> >> > --disable-libjava-multilib
>>>>> >> > --with-ppl --with-cloog --with-tune=generic 
>>>>> >> > --with-arch_32=i686 --build=x86_64-redhat-linux Thread model: 
>>>>> >> > posix gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC)
>>>>> >> >
>>>>> >> >
>>>>> >> > ~/kaldi/trunk/src/matrix
>>>>> >> > $ make test
>>>>> >> > Running matrix-lib-test .../bin/sh: line 1:  5561 Aborted 
>>>>> >> > (core dumped) ./$x > /dev/null 2>&1 ... FAIL Running 
>>>>> >> > kaldi-gpsr-test ...... SUCCESS
>>>>> >> > make: *** [test] Error 1
>>>>> >> >
>>>>> >> >
>>>>> >> > $ ./matrix-lib-test
>>>>> >> > ...
>>>>> >> > ...
>>>>> >> > VLOG[2] (UnitTestLbfgs():matrix-lib-test.cc:2734) L-BFGS 
>>>>> >> > finished after
>>>>> >> > 59
>>>>> >> > function evaluations.
>>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3623) M =  
>>>>> >> > [ ]
>>>>> >> >
>>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3624) M2 =  
>>>>> >> > [ ]
>>>>> >> >
>>>>> >> > LOG (UnitTestCompressedMatrix():matrix-lib-test.cc:3626)
>>>>> >> > Compressed
>>>>> >> > matrix,
>>>>> >> > tot = 0, diff = 0
>>>>> >> >
>>>>> >> > MKL ERROR: Parameter 13 was incorrect on entry to DGESVD.
>>>>> >> > KALDI_ASSERT: at LapackGesvd:kaldi-matrix.cc:401, failed: 
>>>>> >> > result
>>>>> >> > >= 0 &&
>>>>> >> > "Call to CLAPACK dgesvd_ called with wrong arguments"
>>>>> >> > Stack trace is:
>>>>> >> > kaldi::KaldiGetStackTrace()
>>>>> >> > kaldi::KaldiAssertFailure_(char const*, char const*, int, 
>>>>> >> > char
>>>>> >> > const*)
>>>>> >> > kaldi::MatrixBase<double>::LapackGesvd(kaldi::VectorBase<doub
>>>>> >> > le>*, kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*)
>>>>> >> >
>>>>> >> > kaldi::MatrixBase<double>::DestructiveSvd(kaldi::VectorBase<d
>>>>> >> > ouble>*, kaldi::MatrixBase<double>*, 
>>>>> >> > kaldi::MatrixBase<double>*) 
>>>>> >> > kaldi::MatrixBase<double>::Svd(kaldi::VectorBase<double>*,
>>>>> >> > kaldi::MatrixBase<double>*, kaldi::MatrixBase<double>*) const 
>>>>> >> > .
>>>>> >> > .
>>>>> >> > .
>>>>> >> > ./matrix-lib-test() [0x43055a]
>>>>> >> > ./matrix-lib-test() [0x42de8b]
>>>>> >> > ./matrix-lib-test(main+0x1d) [0x42dac7]
>>>>> >> > /lib64/libc.so.6(__libc_start_main+0xfd) [0x330901ecdd]
>>>>> >> > ./matrix-lib-test() [0x42ced9] Aborted (core dumped)
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> > -------------------------------------------------------------
>>>>> >> > ----------------- See everything from the browser to the 
>>>>> >> > database with AppDynamics Get end-to-end visibility with 
>>>>> >> > application monitoring from AppDynamics Isolate bottlenecks 
>>>>> >> > and diagnose root cause in seconds.
>>>>> >> > Start your free trial of AppDynamics Pro today!
>>>>> >> >
>>>>> >> >
>>>>> >> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/41
>>>>> >> > 40/ostg.clktrk 
>>>>> >> > _______________________________________________
>>>>> >> > Kaldi-users mailing list
>>>>> >> > Kal...@li... 
>>>>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>> >> >
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> ---------------------------------------------------------------
>>>>> >> --------------- See everything from the browser to the database 
>>>>> >> with AppDynamics Get end-to-end visibility with application 
>>>>> >> monitoring from AppDynamics Isolate bottlenecks and diagnose 
>>>>> >> root cause in seconds.
>>>>> >> Start your free trial of AppDynamics Pro today!
>>>>> >>
>>>>> >>
>>>>> >> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140
>>>>> >> /ostg.clktrk _______________________________________________
>>>>> >> Kaldi-users mailing list
>>>>> >> Kal...@li... 
>>>>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > ----------------------------------------------------------------
>>>>> > -------------- See everything from the browser to the database 
>>>>> > with AppDynamics Get end-to-end visibility with application 
>>>>> > monitoring from AppDynamics Isolate bottlenecks and diagnose 
>>>>> > root cause in seconds.
>>>>> > Start your free trial of AppDynamics Pro today!
>>>>> >
>>>>> > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/
>>>>> > ostg.clktrk _______________________________________________
>>>>> > Kaldi-users mailing list
>>>>> > Kal...@li... 
>>>>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>> >
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------
>>>>> ------------ See everything from the browser to the database with 
>>>>> AppDynamics Get end-to-end visibility with application monitoring 
>>>>> from AppDynamics Isolate bottlenecks and diagnose root cause in 
>>>>> seconds.
>>>>> Start your free trial of AppDynamics Pro today!
>>>>>
>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/os
>>>>> tg.clktrk _______________________________________________
>>>>> Kaldi-users mailing list
>>>>> Kal...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------------------
>>>> ----------- See everything from the browser to the database with 
>>>> AppDynamics Get end-to-end visibility with application monitoring 
>>>> from AppDynamics Isolate bottlenecks and diagnose root cause in 
>>>> seconds.
>>>> Start your free trial of AppDynamics Pro today!
>>>>
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ost
>>>> g.clktrk _______________________________________________
>>>> Kaldi-users mailing list
>>>> Kal...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>>
>>>
>>>
>>>
>>>
>>>
>>> --------------------------------------------------------------------
>>> ---------- See everything from the browser to the database with 
>>> AppDynamics Get end-to-end visibility with application monitoring 
>>> from AppDynamics Isolate bottlenecks and diagnose root cause in 
>>> seconds.
>>> Start your free trial of AppDynamics Pro today!
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg
>>> .clktrk _______________________________________________
>>> Kaldi-users mailing list
>>> Kal...@li...
>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> --------- See everything from the browser to the database with 
>> AppDynamics Get end-to-end visibility with application monitoring 
>> from AppDynamics Isolate bottlenecks and diagnose root cause in 
>> seconds.
>> Start your free trial of AppDynamics Pro today!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.
>> clktrk _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>
>
>
>
> ----------------------------------------------------------------------
> -------- See everything from the browser to the database with 
> AppDynamics Get end-to-end visibility with application monitoring from 
> AppDynamics Isolate bottlenecks and diagnose root cause in seconds.
> Start your free trial of AppDynamics Pro today!
> http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.c
> lktrk _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>

------------------------------------------------------------------------------
Get your SQL database under version control now!
Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out.
http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk
_______________________________________________
Kaldi-users mailing list
Kal...@li...
https://lists.sourceforge.net/lists/listinfo/kaldi-users

Re: [Kaldi-users] training recommendataions

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Okay.  Thanks,

Nathan

On Jul 29, 2013, at 3:42 PM, Daniel Povey wrote:

> There's no special thing we do for this.  Just play with the #leaves
> and #Gaussians.
> Dan
> 
> 
> On Mon, Jul 29, 2013 at 6:38 PM, Nathan Dunn <nd...@ca...> wrote:
>> 
>> Thanks.
>> 
>> It's unusual that the later stages of training are not better.
>> Normally you get a substantial improvement.
>> 
>> 
>> I wonder if this is due to the very small amount of my training data.
>> 
>> Is there a recommended recipe that I should follow for this type of data
>> (20K in training data, decoding 1 min long passages)?  I tried to use swbd,
>> but ended up going back to using the settings that more closely matched
>> resource management.
>> 
>> 
>> Nathan
>> 
>> On Jul 29, 2013, at 3:27 PM, Daniel Povey wrote:
>> 
>> 1 - I have a training set of around 5K words, though I could bring that up
>> 
>> to around 20K
>> 
>> 
>> More language model training data will definitely help.
>> 
>> 2 - I am using the kaldi_lm, though I could use SRILM . . not sure if it
>> 
>> would necessarily improve results
>> 
>> 
>> Probably would make no difference-- more a usability issue.
>> 
>> 3 - I am decoding about 1 minute of text, though training data is in 10
>> 
>> second epochs.  I can mix some of the test data in if that would help.
>> 
>> 
>> It's not considered good form to mix the test data in with training--
>> this will give you unrealistically good results.
>> 
>> 4 - When I am training deltas I use a very small # of leaves / gauss (100 /
>> 
>> 1000) to get the best results.   The best results are with tri1.  Further
>> 
>> training yields worse results.
>> 
>> 
>> It's unusual that the later stages of training are not better.
>> Normally you get a substantial improvement.
>> 
>> Dan
>> 
>> 5 - I use the same lexicon for the training and decoding (though a more
>> 
>> restrictive language model for decoding).
>> 
>> 
>> Any help / thoughts are appreciated.
>> 
>> 
>> Thanks,
>> 
>> 
>> Nathan
>> 
>> 
>> 
>> 

Re: [Kaldi-users] training recommendataions

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

There's no special thing we do for this.  Just play with the #leaves
and #Gaussians.
Dan


On Mon, Jul 29, 2013 at 6:38 PM, Nathan Dunn <nd...@ca...> wrote:
>
> Thanks.
>
> It's unusual that the later stages of training are not better.
> Normally you get a substantial improvement.
>
>
> I wonder if this is due to the very small amount of my training data.
>
> Is there a recommended recipe that I should follow for this type of data
> (20K in training data, decoding 1 min long passages)?  I tried to use swbd,
> but ended up going back to using the settings that more closely matched
> resource management.
>
>
> Nathan
>
> On Jul 29, 2013, at 3:27 PM, Daniel Povey wrote:
>
> 1 - I have a training set of around 5K words, though I could bring that up
>
> to around 20K
>
>
> More language model training data will definitely help.
>
> 2 - I am using the kaldi_lm, though I could use SRILM . . not sure if it
>
> would necessarily improve results
>
>
> Probably would make no difference-- more a usability issue.
>
> 3 - I am decoding about 1 minute of text, though training data is in 10
>
> second epochs.  I can mix some of the test data in if that would help.
>
>
> It's not considered good form to mix the test data in with training--
> this will give you unrealistically good results.
>
> 4 - When I am training deltas I use a very small # of leaves / gauss (100 /
>
> 1000) to get the best results.   The best results are with tri1.  Further
>
> training yields worse results.
>
>
> It's unusual that the later stages of training are not better.
> Normally you get a substantial improvement.
>
> Dan
>
> 5 - I use the same lexicon for the training and decoding (though a more
>
> restrictive language model for decoding).
>
>
> Any help / thoughts are appreciated.
>
>
> Thanks,
>
>
> Nathan
>
>
>
>

Re: [Kaldi-users] training recommendataions

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

Thanks.  

> It's unusual that the later stages of training are not better.
> Normally you get a substantial improvement.

I wonder if this is due to the very small amount of my training data.   

Is there a recommended recipe that I should follow for this type of data (20K in training data, decoding 1 min long passages)?  I tried to use swbd, but ended up going back to using the settings that more closely matched resource management.  

Nathan

On Jul 29, 2013, at 3:27 PM, Daniel Povey wrote:

>> 1 - I have a training set of around 5K words, though I could bring that up
>> to around 20K
> 
> More language model training data will definitely help.
> 
>> 2 - I am using the kaldi_lm, though I could use SRILM . . not sure if it
>> would necessarily improve results
> 
> Probably would make no difference-- more a usability issue.
> 
>> 3 - I am decoding about 1 minute of text, though training data is in 10
>> second epochs.  I can mix some of the test data in if that would help.
> 
> It's not considered good form to mix the test data in with training--
> this will give you unrealistically good results.
> 
>> 4 - When I am training deltas I use a very small # of leaves / gauss (100 /
>> 1000) to get the best results.   The best results are with tri1.  Further
>> training yields worse results.
> 
> It's unusual that the later stages of training are not better.
> Normally you get a substantial improvement.
> 
> Dan
> 
>> 5 - I use the same lexicon for the training and decoding (though a more
>> restrictive language model for decoding).
> 
>> Any help / thoughts are appreciated.
>> 
>> Thanks,
>> 
>> Nathan
>> 
>> 

Re: [Kaldi-users] training recommendataions

[Mailing l. u. f. U. C. a. U. <kal...@li...>]

> 1 - I have a training set of around 5K words, though I could bring that up
> to around 20K

More language model training data will definitely help.

> 2 - I am using the kaldi_lm, though I could use SRILM . . not sure if it
> would necessarily improve results

Probably would make no difference-- more a usability issue.

> 3 - I am decoding about 1 minute of text, though training data is in 10
> second epochs.  I can mix some of the test data in if that would help.

It's not considered good form to mix the test data in with training--
this will give you unrealistically good results.

> 4 - When I am training deltas I use a very small # of leaves / gauss (100 /
> 1000) to get the best results.   The best results are with tri1.  Further
> training yields worse results.

It's unusual that the later stages of training are not better.
Normally you get a substantial improvement.

Dan

> 5 - I use the same lexicon for the training and decoding (though a more
> restrictive language model for decoding).

> Any help / thoughts are appreciated.
>
> Thanks,
>
> Nathan
>
>