Menu
▾
▴
kaldi-users — Kaldi Users
You can subscribe to this list here.
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
(2) |
Sep
(1) |
Oct
(1) |
Nov
|
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2012 |
Jan
|
Feb
|
Mar
(8) |
Apr
(4) |
May
(2) |
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2013 |
Jan
|
Feb
(2) |
Mar
(2) |
Apr
(7) |
May
(31) |
Jun
(40) |
Jul
(65) |
Aug
(37) |
Sep
(12) |
Oct
(57) |
Nov
(15) |
Dec
(35) |
| 2014 |
Jan
(3) |
Feb
(30) |
Mar
(57) |
Apr
(26) |
May
(49) |
Jun
(26) |
Jul
(63) |
Aug
(33) |
Sep
(20) |
Oct
(153) |
Nov
(62) |
Dec
(20) |
| 2015 |
Jan
(6) |
Feb
(21) |
Mar
(42) |
Apr
(33) |
May
(76) |
Jun
(102) |
Jul
(39) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-07-02 13:22:50
|
Sorry, I was wrong. It selects the GPU automatically, I found the error in *exp/tri4b_pretrain-dbn/log/cmvn_glob_fwd.log *file. ERROR (nnet-forward:PdfPrior():nnet-pdf-prior.cc:26) --class-frame-counts is empty: Cannot initialize priors without the counts. ERROR (nnet-forward:main():nnet-forward.cc:196) ERROR (nnet-forward:PdfPrior():nnet-pdf-prior.cc:26) --class-frame-counts is empty: Cannot initialize priors without the counts. Thanks Lahiru On Tue, Jul 2, 2013 at 9:10 PM, Lahiru Samarakoon <lah...@gm...>wrote: > Hi All, > > When running DNN training on GPUs, I am getting following error. > > *Log File : exp/tri4b_pretrain-dbn/_pretrain_dbn.log* > > *# PRE-TRAINING RBM LAYER 1 > Initializing 'exp/tri4b_pretrain-dbn/1.rbm.init' > Traceback (most recent call last): > File "utils/nnet/gen_rbm_init.py", line 40, in ? > dimL.append(int(dimStrL[i])) > ValueError: invalid literal for int(): * > > > I am running this in a GPU cluster which assigns the job to a GPU > dynamically, So I cannot configure the *_gpu_id= # manually select GPU id > to run on, (-1 disables GPU)*. > Can this be the cause? > > Thanks, > Lahiru > > > On Fri, Jun 28, 2013 at 11:06 PM, Mailing list used for User Communication > and Updates <kal...@li...> wrote: > >> It's not the same as that. Each machine does SGD separately and, >> periodically, the parameters are averaged across machines. >> Dan >> >> >> On Fri, Jun 28, 2013 at 11:03 AM, Mailing list used for User >> Communication and Updates <kal...@li...> wrote: >> > Wow, nice. >> > Does the implementation similar to the Jeff Dean's paper Large Scale >> > Distributed Deep Networks >> > ( >> http://www.cs.toronto.edu/~ranzato/publications/DistBeliefNIPS2012_withAppendix.pdf >> ) >> > ? >> > Does Kaldi use Asynchronous SGD? >> > >> > Please give me a brief description. >> > >> > Thanks, >> > Lahiru >> > >> > >> > On Fri, Jun 28, 2013 at 10:28 PM, Mailing list used for User >> Communication >> > and Updates <kal...@li...> wrote: >> >> >> >> It's on multiple machines and also multiple threads per machine. >> >> Dan >> >> >> >> >> >> On Fri, Jun 28, 2013 at 2:05 AM, Mailing list used for User >> >> Communication and Updates <kal...@li...> wrote: >> >> > Thanks guys :-) >> >> > >> >> > Dan, is your setup for distributed training? Or is it only >> parallelize >> >> > with >> >> > in a single machine? >> >> > >> >> > Thanks, >> >> > Lahiru >> >> > >> >> > >> >> > >> >> > On Fri, Jun 28, 2013 at 5:29 AM, Mailing list used for User >> >> > Communication >> >> > and Updates <kal...@li...> wrote: >> >> >> >> >> >> In my setup there is RBM pre-training: >> >> >> http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf >> >> >> <http://www.cs.toronto.edu/%7Ehinton/absps/guideTR.pdf> >> >> >> followed by per-frame cross entropy training and sMBR training: >> >> >> http://www.danielpovey.com/files/2013_interspeech_dnn.pdf >> >> >> >> >> >> >> >> >> Dne 27.6.2013 13:21, Mailing list used for User Communication and >> >> >> Updates napsal(a): >> >> >> > There are basically two setups there: Karel's setup, generally >> called >> >> >> > run_dnn.sh or run_nnet.sh, which is for GPUs, and my setup, called >> >> >> > run_nnet_cpu.sh, which is for CPUs in parallel. Karel's setup may >> >> >> > have an ICASSP paper, Karel can tell you. Mine is mostly >> >> >> > unpublished. >> >> >> > >> >> >> > Dan >> >> >> > >> >> >> > >> >> >> > On Thu, Jun 27, 2013 at 5:31 AM, Mailing list used for User >> >> >> > Communication and Updates <kal...@li...> >> wrote: >> >> >> >> Hi All, >> >> >> >> >> >> >> >> I am in the process of running the wsj/s5 recipe. Now I am about >> the >> >> >> >> run DNN >> >> >> >> experiments and specifically interested in the DNN training. I >> am >> >> >> >> planning >> >> >> >> to look into the DNN code for more understanding. Since there are >> >> >> >> many >> >> >> >> DNN >> >> >> >> variants, could anyone tell me the papers Kalid DNN >> implementation >> >> >> >> represents? >> >> >> >> >> >> >> >> Thanks, >> >> >> >> Lahiru >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> >> This SF.net email is sponsored by Windows: >> >> >> >> >> >> >> >> Build for Windows Store. >> >> >> >> >> >> >> >> http://p.sf.net/sfu/windows-dev2dev >> >> >> >> _______________________________________________ >> >> >> >> Kaldi-users mailing list >> >> >> >> Kal...@li... >> >> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> >> >> >> >> >> > >> >> >> > >> >> >> > >> ------------------------------------------------------------------------------ >> >> >> > This SF.net email is sponsored by Windows: >> >> >> > >> >> >> > Build for Windows Store. >> >> >> > >> >> >> > http://p.sf.net/sfu/windows-dev2dev >> >> >> > _______________________________________________ >> >> >> > Kaldi-users mailing list >> >> >> > Kal...@li... >> >> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> This SF.net email is sponsored by Windows: >> >> >> >> >> >> Build for Windows Store. >> >> >> >> >> >> http://p.sf.net/sfu/windows-dev2dev >> >> >> _______________________________________________ >> >> >> Kaldi-users mailing list >> >> >> Kal...@li... >> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> > >> >> > >> >> > >> >> > >> >> > >> ------------------------------------------------------------------------------ >> >> > This SF.net email is sponsored by Windows: >> >> > >> >> > Build for Windows Store. >> >> > >> >> > http://p.sf.net/sfu/windows-dev2dev >> >> > _______________________________________________ >> >> > Kaldi-users mailing list >> >> > Kal...@li... >> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> > >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> This SF.net email is sponsored by Windows: >> >> >> >> Build for Windows Store. >> >> >> >> http://p.sf.net/sfu/windows-dev2dev >> >> _______________________________________________ >> >> Kaldi-users mailing list >> >> Kal...@li... >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> > >> > >> > >> > >> ------------------------------------------------------------------------------ >> > This SF.net email is sponsored by Windows: >> > >> > Build for Windows Store. >> > >> > http://p.sf.net/sfu/windows-dev2dev >> > _______________________________________________ >> > Kaldi-users mailing list >> > Kal...@li... >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >> > >> >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> > > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-07-02 13:10:56
|
Hi All,
When running DNN training on GPUs, I am getting following error.
*Log File : exp/tri4b_pretrain-dbn/_pretrain_dbn.log*
*# PRE-TRAINING RBM LAYER 1
Initializing 'exp/tri4b_pretrain-dbn/1.rbm.init'
Traceback (most recent call last):
File "utils/nnet/gen_rbm_init.py", line 40, in ?
dimL.append(int(dimStrL[i]))
ValueError: invalid literal for int(): *
I am running this in a GPU cluster which assigns the job to a GPU
dynamically, So I cannot configure the *_gpu_id= # manually select GPU id
to run on, (-1 disables GPU)*.
Can this be the cause?
Thanks,
Lahiru
On Fri, Jun 28, 2013 at 11:06 PM, Mailing list used for User Communication
and Updates <kal...@li...> wrote:
> It's not the same as that. Each machine does SGD separately and,
> periodically, the parameters are averaged across machines.
> Dan
>
>
> On Fri, Jun 28, 2013 at 11:03 AM, Mailing list used for User
> Communication and Updates <kal...@li...> wrote:
> > Wow, nice.
> > Does the implementation similar to the Jeff Dean's paper Large Scale
> > Distributed Deep Networks
> > (
> http://www.cs.toronto.edu/~ranzato/publications/DistBeliefNIPS2012_withAppendix.pdf
> )
> > ?
> > Does Kaldi use Asynchronous SGD?
> >
> > Please give me a brief description.
> >
> > Thanks,
> > Lahiru
> >
> >
> > On Fri, Jun 28, 2013 at 10:28 PM, Mailing list used for User
> Communication
> > and Updates <kal...@li...> wrote:
> >>
> >> It's on multiple machines and also multiple threads per machine.
> >> Dan
> >>
> >>
> >> On Fri, Jun 28, 2013 at 2:05 AM, Mailing list used for User
> >> Communication and Updates <kal...@li...> wrote:
> >> > Thanks guys :-)
> >> >
> >> > Dan, is your setup for distributed training? Or is it only parallelize
> >> > with
> >> > in a single machine?
> >> >
> >> > Thanks,
> >> > Lahiru
> >> >
> >> >
> >> >
> >> > On Fri, Jun 28, 2013 at 5:29 AM, Mailing list used for User
> >> > Communication
> >> > and Updates <kal...@li...> wrote:
> >> >>
> >> >> In my setup there is RBM pre-training:
> >> >> http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf
> >> >> <http://www.cs.toronto.edu/%7Ehinton/absps/guideTR.pdf>
> >> >> followed by per-frame cross entropy training and sMBR training:
> >> >> http://www.danielpovey.com/files/2013_interspeech_dnn.pdf
> >> >>
> >> >>
> >> >> Dne 27.6.2013 13:21, Mailing list used for User Communication and
> >> >> Updates napsal(a):
> >> >> > There are basically two setups there: Karel's setup, generally
> called
> >> >> > run_dnn.sh or run_nnet.sh, which is for GPUs, and my setup, called
> >> >> > run_nnet_cpu.sh, which is for CPUs in parallel. Karel's setup may
> >> >> > have an ICASSP paper, Karel can tell you. Mine is mostly
> >> >> > unpublished.
> >> >> >
> >> >> > Dan
> >> >> >
> >> >> >
> >> >> > On Thu, Jun 27, 2013 at 5:31 AM, Mailing list used for User
> >> >> > Communication and Updates <kal...@li...>
> wrote:
> >> >> >> Hi All,
> >> >> >>
> >> >> >> I am in the process of running the wsj/s5 recipe. Now I am about
> the
> >> >> >> run DNN
> >> >> >> experiments and specifically interested in the DNN training. I am
> >> >> >> planning
> >> >> >> to look into the DNN code for more understanding. Since there are
> >> >> >> many
> >> >> >> DNN
> >> >> >> variants, could anyone tell me the papers Kalid DNN implementation
> >> >> >> represents?
> >> >> >>
> >> >> >> Thanks,
> >> >> >> Lahiru
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> ------------------------------------------------------------------------------
> >> >> >> This SF.net email is sponsored by Windows:
> >> >> >>
> >> >> >> Build for Windows Store.
> >> >> >>
> >> >> >> http://p.sf.net/sfu/windows-dev2dev
> >> >> >> _______________________________________________
> >> >> >> Kaldi-users mailing list
> >> >> >> Kal...@li...
> >> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >> >>
> >> >> >
> >> >> >
> >> >> >
> ------------------------------------------------------------------------------
> >> >> > This SF.net email is sponsored by Windows:
> >> >> >
> >> >> > Build for Windows Store.
> >> >> >
> >> >> > http://p.sf.net/sfu/windows-dev2dev
> >> >> > _______________________________________________
> >> >> > Kaldi-users mailing list
> >> >> > Kal...@li...
> >> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >>
> >> >>
> >> >>
> >> >>
> >> >>
> ------------------------------------------------------------------------------
> >> >> This SF.net email is sponsored by Windows:
> >> >>
> >> >> Build for Windows Store.
> >> >>
> >> >> http://p.sf.net/sfu/windows-dev2dev
> >> >> _______________________________________________
> >> >> Kaldi-users mailing list
> >> >> Kal...@li...
> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >
> >> >
> >> >
> >> >
> >> >
> ------------------------------------------------------------------------------
> >> > This SF.net email is sponsored by Windows:
> >> >
> >> > Build for Windows Store.
> >> >
> >> > http://p.sf.net/sfu/windows-dev2dev
> >> > _______________________________________________
> >> > Kaldi-users mailing list
> >> > Kal...@li...
> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >> >
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> This SF.net email is sponsored by Windows:
> >>
> >> Build for Windows Store.
> >>
> >> http://p.sf.net/sfu/windows-dev2dev
> >> _______________________________________________
> >> Kaldi-users mailing list
> >> Kal...@li...
> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >
> >
> >
> >
> ------------------------------------------------------------------------------
> > This SF.net email is sponsored by Windows:
> >
> > Build for Windows Store.
> >
> > http://p.sf.net/sfu/windows-dev2dev
> > _______________________________________________
> > Kaldi-users mailing list
> > Kal...@li...
> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
> >
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>
|
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-07-01 06:01:40
|
This seemed to do the trick. Would you also recommend lower the beam, as well. Thanks, Nathan On Jun 30, 2013, at 9:44 PM, Daniel Povey wrote: > You could try gmm-latgen-faster which should be faster. > You can set the lattice beam using e.g. --lattice-beam=5.0 > That should stop this error. > Dan > > > On Mon, Jul 1, 2013 at 12:40 AM, Nathan Dunn <nd...@ca...> wrote: >> >> Dan, >> >> Thanks for quick reply. I am using an updated resource management script. The beam is 20 and an acoustic scale of 0.1. I'm not sure how I would specify the lattice-beam. I am in the stable branch. >> >> gmm-latgen-simple --beam=20.0 --acoustic-scale=0.1 --word-symbol-table=$lang/words.txt \ >> $srcdir/final.mdl $graphdir/HCLG.fst "$feats" "ark:|gzip -c > $dir/lat.gz" \ >> ark,t:$dir/test.tra ark,t:$dir/test.ali \ >> 2> $dir/decode.log || exit 1; >> >> I generated the language model using the CMU toolkit (not sure if that is best practices). For decoding its a little unusual. >> >> The language model is (currently) a set of the same 3 canonical passages. Each reader (100) reads the same 3 passages. We use these to build the language model, which is not ideal as it would be better to use the correct transcript (we are currently in the process of getting this), though it would surprise me if it failed because of this (and it works surprisingly well when it doesn't fail). >> >> So, after reading the script and taking your advice, I'm trying going to reduce the beam to 13 and an acoustic value of 0.07 (though I think it tries to fit this anyway). However, if gives me a similar failure. >> >> So, here are a few observations: >> 1 - it looks like I am getting this from poor quality audio (reader max's out input gain) >> 2 - should I use another method other than: gmm-latgen-simple . . . looked like there were quite a few other options >> 3 - are there other good parameters you would recommend? >> >> Thanks, >> >> Nathan >> >> >> On Jun 30, 2013, at 12:37 PM, Daniel Povey wrote: >> >>> In case you are not on the list or did not get the reply. >>> Please cc the list if you reply. >>> >>> >>> ---------- Forwarded message ---------- >>> From: Daniel Povey <dp...@gm...> >>> Date: Sun, Jun 30, 2013 at 3:37 PM >>> Subject: Re: [Kaldi-users] issues with decoding (self loops) in tri2a >>> for 1-minute speech segments >>> To: kal...@li... >>> >>> >>> Can you describe the language model you use for the decoding phase? >>> I'd like to know in order to understand what scenarios this is most >>> likely to happen in. >>> What values did you use for "beam" and "lattice-beam"? Typically you >>> should be able to solve these problems by reducing "lattice-beam". >>> >>> >>> Dan >>> >>> >>> On Sun, Jun 30, 2013 at 3:26 PM, Mailing list used for User >>> Communication and Updates <kal...@li...> wrote: >>>> >>>> Hello, >>>> >>>> I'm trying to decode long (1 minute) speech segments (multiple sentences) >>>> having trained on short (<15 second) corpuses. For some reason I get >>>> random failures of the following(even longer logs below): >>>> >>>> WARNING (gmm-latgen-simple:Close():kaldi-io.cc:444) Pipe compute-cmvn-stats >>>> scp:data/test_childspeech/feats.scp ark:- | apply-cmvn --norm-vars=false >>>> ark:- scp:data/test_childspeech/feats.scp ark:- | add-deltas ark:- ark:- | >>>> had nonzero return status 36096 >>>> ERROR >>>> (gmm-latgen-simple:EpsilonClosure():fstext/determinize-lattice-pruned-inl.h:664) >>>> Lattice determinization aborted since looped more than 500000 times during >>>> epsilon closure. >>>> >>>> I've had it decode 135 of these, but sometimes it will fail after 5-10 with >>>> this error. >>>> >>>> My training model is the LDC9763 child's speech corpus, of which I use about >>>> 4K sentences. If I test on a subset (100 not int he training set) I get >>>> great results (using the tri2a method). >>>> >>>> The language model I use for testing is the target speakers' target sentence >>>> (45 speakers speaking the same 3 sentences). >>>> >>>> Next, I was going to try breaking up the decoding into smaller segments >>>> (those seem to work) as well as have a more accurate language model for the >>>> decoding phase. >>>> >>>> Any ideas? >>>> >>>> Thanks, >>>> >>>> Nathan >>>> >>>> >>>> >>>> >>>> LOG >>>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>>> Rebuilding repository. >>>> LOG >>>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>>> Rebuilding repository. >>>> LOG >>>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>>> Rebuilding repository. >>>> LOG >>>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>>> Rebuilding repository. >>>> LOG >>>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>>> Rebuilding repository. >>>> LOG >>>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>>> Rebuilding repository. >>>> LOG >>>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl. >>>> >>>> Nathan Dunn, Ph.D. >>>> Scientific Programer >>>> College of Arts and Science IT >>>> 541-221-2418 >>>> nd...@ca... >>>> >>>> >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> This SF.net email is sponsored by Windows: >>>> >>>> Build for Windows Store. >>>> >>>> http://p.sf.net/sfu/windows-dev2dev >>>> _______________________________________________ >>>> Kaldi-users mailing list >>>> Kal...@li... >>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users >>>> >> |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-07-01 05:40:54
|
That's what I thought. Thanks for the help. We've been very happy with the results. Will try to post back any recipes we get. Thanks, Nathan On Jun 30, 2013, at 10:34 PM, Daniel Povey wrote: > Lowering the beam will make it faster and somewhat less accurate, > that's the main effect. > Dan > > > On Mon, Jul 1, 2013 at 1:32 AM, Nathan Dunn <nd...@ca...> wrote: >> >> This seemed to do the trick. >> >> Would you also recommend lower the beam, as well. >> >> Thanks, >> >> Nathan >> >> >> On Jun 30, 2013, at 9:44 PM, Daniel Povey wrote: >> >> You could try gmm-latgen-faster which should be faster. >> You can set the lattice beam using e.g. --lattice-beam=5.0 >> That should stop this error. >> Dan >> >> >> On Mon, Jul 1, 2013 at 12:40 AM, Nathan Dunn <nd...@ca...> wrote: >> >> >> Dan, >> >> >> Thanks for quick reply. I am using an updated resource management script. >> The beam is 20 and an acoustic scale of 0.1. I'm not sure how I would >> specify the lattice-beam. I am in the stable branch. >> >> >> gmm-latgen-simple --beam=20.0 --acoustic-scale=0.1 >> --word-symbol-table=$lang/words.txt \ >> >> $srcdir/final.mdl $graphdir/HCLG.fst "$feats" "ark:|gzip -c > $dir/lat.gz" >> \ >> >> ark,t:$dir/test.tra ark,t:$dir/test.ali \ >> >> 2> $dir/decode.log || exit 1; >> >> >> I generated the language model using the CMU toolkit (not sure if that is >> best practices). For decoding its a little unusual. >> >> >> The language model is (currently) a set of the same 3 canonical passages. >> Each reader (100) reads the same 3 passages. We use these to build the >> language model, which is not ideal as it would be better to use the correct >> transcript (we are currently in the process of getting this), though it >> would surprise me if it failed because of this (and it works surprisingly >> well when it doesn't fail). >> >> >> So, after reading the script and taking your advice, I'm trying going to >> reduce the beam to 13 and an acoustic value of 0.07 (though I think it tries >> to fit this anyway). However, if gives me a similar failure. >> >> >> So, here are a few observations: >> >> 1 - it looks like I am getting this from poor quality audio (reader max's >> out input gain) >> >> 2 - should I use another method other than: gmm-latgen-simple . . . looked >> like there were quite a few other options >> >> 3 - are there other good parameters you would recommend? >> >> >> Thanks, >> >> >> Nathan >> >> >> >> On Jun 30, 2013, at 12:37 PM, Daniel Povey wrote: >> >> >> In case you are not on the list or did not get the reply. >> >> Please cc the list if you reply. >> >> >> >> ---------- Forwarded message ---------- >> >> From: Daniel Povey <dp...@gm...> >> >> Date: Sun, Jun 30, 2013 at 3:37 PM >> >> Subject: Re: [Kaldi-users] issues with decoding (self loops) in tri2a >> >> for 1-minute speech segments >> >> To: kal...@li... >> >> >> >> Can you describe the language model you use for the decoding phase? >> >> I'd like to know in order to understand what scenarios this is most >> >> likely to happen in. >> >> What values did you use for "beam" and "lattice-beam"? Typically you >> >> should be able to solve these problems by reducing "lattice-beam". >> >> >> >> Dan >> >> >> >> On Sun, Jun 30, 2013 at 3:26 PM, Mailing list used for User >> >> Communication and Updates <kal...@li...> wrote: >> >> >> Hello, >> >> >> I'm trying to decode long (1 minute) speech segments (multiple sentences) >> >> having trained on short (<15 second) corpuses. For some reason I get >> >> random failures of the following(even longer logs below): >> >> >> WARNING (gmm-latgen-simple:Close():kaldi-io.cc:444) Pipe compute-cmvn-stats >> >> scp:data/test_childspeech/feats.scp ark:- | apply-cmvn --norm-vars=false >> >> ark:- scp:data/test_childspeech/feats.scp ark:- | add-deltas ark:- ark:- | >> >> had nonzero return status 36096 >> >> ERROR >> >> (gmm-latgen-simple:EpsilonClosure():fstext/determinize-lattice-pruned-inl.h:664) >> >> Lattice determinization aborted since looped more than 500000 times during >> >> epsilon closure. >> >> >> I've had it decode 135 of these, but sometimes it will fail after 5-10 with >> >> this error. >> >> >> My training model is the LDC9763 child's speech corpus, of which I use about >> >> 4K sentences. If I test on a subset (100 not int he training set) I get >> >> great results (using the tri2a method). >> >> >> The language model I use for testing is the target speakers' target sentence >> >> (45 speakers speaking the same 3 sentences). >> >> >> Next, I was going to try breaking up the decoding into smaller segments >> >> (those seem to work) as well as have a more accurate language model for the >> >> decoding phase. >> >> >> Any ideas? >> >> >> Thanks, >> >> >> Nathan >> >> >> >> >> >> LOG >> >> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >> >> Rebuilding repository. >> >> LOG >> >> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >> >> Rebuilding repository. >> >> LOG >> >> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >> >> Rebuilding repository. >> >> LOG >> >> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >> >> Rebuilding repository. >> >> LOG >> >> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >> >> Rebuilding repository. >> >> LOG >> >> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >> >> Rebuilding repository. >> >> LOG >> >> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl. >> >> >> Nathan Dunn, Ph.D. >> >> Scientific Programer >> >> College of Arts and Science IT >> >> 541-221-2418 >> >> nd...@ca... >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> This SF.net email is sponsored by Windows: >> >> >> Build for Windows Store. >> >> >> http://p.sf.net/sfu/windows-dev2dev >> >> _______________________________________________ >> >> Kaldi-users mailing list >> >> Kal...@li... >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> >> >> |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-07-01 05:34:18
|
Lowering the beam will make it faster and somewhat less accurate, that's the main effect. Dan On Mon, Jul 1, 2013 at 1:32 AM, Nathan Dunn <nd...@ca...> wrote: > > This seemed to do the trick. > > Would you also recommend lower the beam, as well. > > Thanks, > > Nathan > > > On Jun 30, 2013, at 9:44 PM, Daniel Povey wrote: > > You could try gmm-latgen-faster which should be faster. > You can set the lattice beam using e.g. --lattice-beam=5.0 > That should stop this error. > Dan > > > On Mon, Jul 1, 2013 at 12:40 AM, Nathan Dunn <nd...@ca...> wrote: > > > Dan, > > > Thanks for quick reply. I am using an updated resource management script. > The beam is 20 and an acoustic scale of 0.1. I'm not sure how I would > specify the lattice-beam. I am in the stable branch. > > > gmm-latgen-simple --beam=20.0 --acoustic-scale=0.1 > --word-symbol-table=$lang/words.txt \ > > $srcdir/final.mdl $graphdir/HCLG.fst "$feats" "ark:|gzip -c > $dir/lat.gz" > \ > > ark,t:$dir/test.tra ark,t:$dir/test.ali \ > > 2> $dir/decode.log || exit 1; > > > I generated the language model using the CMU toolkit (not sure if that is > best practices). For decoding its a little unusual. > > > The language model is (currently) a set of the same 3 canonical passages. > Each reader (100) reads the same 3 passages. We use these to build the > language model, which is not ideal as it would be better to use the correct > transcript (we are currently in the process of getting this), though it > would surprise me if it failed because of this (and it works surprisingly > well when it doesn't fail). > > > So, after reading the script and taking your advice, I'm trying going to > reduce the beam to 13 and an acoustic value of 0.07 (though I think it tries > to fit this anyway). However, if gives me a similar failure. > > > So, here are a few observations: > > 1 - it looks like I am getting this from poor quality audio (reader max's > out input gain) > > 2 - should I use another method other than: gmm-latgen-simple . . . looked > like there were quite a few other options > > 3 - are there other good parameters you would recommend? > > > Thanks, > > > Nathan > > > > On Jun 30, 2013, at 12:37 PM, Daniel Povey wrote: > > > In case you are not on the list or did not get the reply. > > Please cc the list if you reply. > > > > ---------- Forwarded message ---------- > > From: Daniel Povey <dp...@gm...> > > Date: Sun, Jun 30, 2013 at 3:37 PM > > Subject: Re: [Kaldi-users] issues with decoding (self loops) in tri2a > > for 1-minute speech segments > > To: kal...@li... > > > > Can you describe the language model you use for the decoding phase? > > I'd like to know in order to understand what scenarios this is most > > likely to happen in. > > What values did you use for "beam" and "lattice-beam"? Typically you > > should be able to solve these problems by reducing "lattice-beam". > > > > Dan > > > > On Sun, Jun 30, 2013 at 3:26 PM, Mailing list used for User > > Communication and Updates <kal...@li...> wrote: > > > Hello, > > > I'm trying to decode long (1 minute) speech segments (multiple sentences) > > having trained on short (<15 second) corpuses. For some reason I get > > random failures of the following(even longer logs below): > > > WARNING (gmm-latgen-simple:Close():kaldi-io.cc:444) Pipe compute-cmvn-stats > > scp:data/test_childspeech/feats.scp ark:- | apply-cmvn --norm-vars=false > > ark:- scp:data/test_childspeech/feats.scp ark:- | add-deltas ark:- ark:- | > > had nonzero return status 36096 > > ERROR > > (gmm-latgen-simple:EpsilonClosure():fstext/determinize-lattice-pruned-inl.h:664) > > Lattice determinization aborted since looped more than 500000 times during > > epsilon closure. > > > I've had it decode 135 of these, but sometimes it will fail after 5-10 with > > this error. > > > My training model is the LDC9763 child's speech corpus, of which I use about > > 4K sentences. If I test on a subset (100 not int he training set) I get > > great results (using the tri2a method). > > > The language model I use for testing is the target speakers' target sentence > > (45 speakers speaking the same 3 sentences). > > > Next, I was going to try breaking up the decoding into smaller segments > > (those seem to work) as well as have a more accurate language model for the > > decoding phase. > > > Any ideas? > > > Thanks, > > > Nathan > > > > > > LOG > > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > > Rebuilding repository. > > LOG > > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > > Rebuilding repository. > > LOG > > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > > Rebuilding repository. > > LOG > > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > > Rebuilding repository. > > LOG > > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > > Rebuilding repository. > > LOG > > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > > Rebuilding repository. > > LOG > > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl. > > > Nathan Dunn, Ph.D. > > Scientific Programer > > College of Arts and Science IT > > 541-221-2418 > > nd...@ca... > > > > > > ------------------------------------------------------------------------------ > > This SF.net email is sponsored by Windows: > > > Build for Windows Store. > > > http://p.sf.net/sfu/windows-dev2dev > > _______________________________________________ > > Kaldi-users mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-07-01 04:44:08
|
You could try gmm-latgen-faster which should be faster. You can set the lattice beam using e.g. --lattice-beam=5.0 That should stop this error. Dan On Mon, Jul 1, 2013 at 12:40 AM, Nathan Dunn <nd...@ca...> wrote: > > Dan, > > Thanks for quick reply. I am using an updated resource management script. The beam is 20 and an acoustic scale of 0.1. I'm not sure how I would specify the lattice-beam. I am in the stable branch. > > gmm-latgen-simple --beam=20.0 --acoustic-scale=0.1 --word-symbol-table=$lang/words.txt \ > $srcdir/final.mdl $graphdir/HCLG.fst "$feats" "ark:|gzip -c > $dir/lat.gz" \ > ark,t:$dir/test.tra ark,t:$dir/test.ali \ > 2> $dir/decode.log || exit 1; > > I generated the language model using the CMU toolkit (not sure if that is best practices). For decoding its a little unusual. > > The language model is (currently) a set of the same 3 canonical passages. Each reader (100) reads the same 3 passages. We use these to build the language model, which is not ideal as it would be better to use the correct transcript (we are currently in the process of getting this), though it would surprise me if it failed because of this (and it works surprisingly well when it doesn't fail). > > So, after reading the script and taking your advice, I'm trying going to reduce the beam to 13 and an acoustic value of 0.07 (though I think it tries to fit this anyway). However, if gives me a similar failure. > > So, here are a few observations: > 1 - it looks like I am getting this from poor quality audio (reader max's out input gain) > 2 - should I use another method other than: gmm-latgen-simple . . . looked like there were quite a few other options > 3 - are there other good parameters you would recommend? > > Thanks, > > Nathan > > > On Jun 30, 2013, at 12:37 PM, Daniel Povey wrote: > >> In case you are not on the list or did not get the reply. >> Please cc the list if you reply. >> >> >> ---------- Forwarded message ---------- >> From: Daniel Povey <dp...@gm...> >> Date: Sun, Jun 30, 2013 at 3:37 PM >> Subject: Re: [Kaldi-users] issues with decoding (self loops) in tri2a >> for 1-minute speech segments >> To: kal...@li... >> >> >> Can you describe the language model you use for the decoding phase? >> I'd like to know in order to understand what scenarios this is most >> likely to happen in. >> What values did you use for "beam" and "lattice-beam"? Typically you >> should be able to solve these problems by reducing "lattice-beam". >> >> >> Dan >> >> >> On Sun, Jun 30, 2013 at 3:26 PM, Mailing list used for User >> Communication and Updates <kal...@li...> wrote: >>> >>> Hello, >>> >>> I'm trying to decode long (1 minute) speech segments (multiple sentences) >>> having trained on short (<15 second) corpuses. For some reason I get >>> random failures of the following(even longer logs below): >>> >>> WARNING (gmm-latgen-simple:Close():kaldi-io.cc:444) Pipe compute-cmvn-stats >>> scp:data/test_childspeech/feats.scp ark:- | apply-cmvn --norm-vars=false >>> ark:- scp:data/test_childspeech/feats.scp ark:- | add-deltas ark:- ark:- | >>> had nonzero return status 36096 >>> ERROR >>> (gmm-latgen-simple:EpsilonClosure():fstext/determinize-lattice-pruned-inl.h:664) >>> Lattice determinization aborted since looped more than 500000 times during >>> epsilon closure. >>> >>> I've had it decode 135 of these, but sometimes it will fail after 5-10 with >>> this error. >>> >>> My training model is the LDC9763 child's speech corpus, of which I use about >>> 4K sentences. If I test on a subset (100 not int he training set) I get >>> great results (using the tri2a method). >>> >>> The language model I use for testing is the target speakers' target sentence >>> (45 speakers speaking the same 3 sentences). >>> >>> Next, I was going to try breaking up the decoding into smaller segments >>> (those seem to work) as well as have a more accurate language model for the >>> decoding phase. >>> >>> Any ideas? >>> >>> Thanks, >>> >>> Nathan >>> >>> >>> >>> >>> LOG >>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>> Rebuilding repository. >>> LOG >>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>> Rebuilding repository. >>> LOG >>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>> Rebuilding repository. >>> LOG >>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>> Rebuilding repository. >>> LOG >>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>> Rebuilding repository. >>> LOG >>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) >>> Rebuilding repository. >>> LOG >>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl. >>> >>> Nathan Dunn, Ph.D. >>> Scientific Programer >>> College of Arts and Science IT >>> 541-221-2418 >>> nd...@ca... >>> >>> >>> >>> >>> ------------------------------------------------------------------------------ >>> This SF.net email is sponsored by Windows: >>> >>> Build for Windows Store. >>> >>> http://p.sf.net/sfu/windows-dev2dev >>> _______________________________________________ >>> Kaldi-users mailing list >>> Kal...@li... >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users >>> > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-07-01 04:41:03
|
Dan,
Thanks for quick reply. I am using an updated resource management script. The beam is 20 and an acoustic scale of 0.1. I'm not sure how I would specify the lattice-beam. I am in the stable branch.
gmm-latgen-simple --beam=20.0 --acoustic-scale=0.1 --word-symbol-table=$lang/words.txt \
$srcdir/final.mdl $graphdir/HCLG.fst "$feats" "ark:|gzip -c > $dir/lat.gz" \
ark,t:$dir/test.tra ark,t:$dir/test.ali \
2> $dir/decode.log || exit 1;
I generated the language model using the CMU toolkit (not sure if that is best practices). For decoding its a little unusual.
The language model is (currently) a set of the same 3 canonical passages. Each reader (100) reads the same 3 passages. We use these to build the language model, which is not ideal as it would be better to use the correct transcript (we are currently in the process of getting this), though it would surprise me if it failed because of this (and it works surprisingly well when it doesn't fail).
So, after reading the script and taking your advice, I'm trying going to reduce the beam to 13 and an acoustic value of 0.07 (though I think it tries to fit this anyway). However, if gives me a similar failure.
So, here are a few observations:
1 - it looks like I am getting this from poor quality audio (reader max's out input gain)
2 - should I use another method other than: gmm-latgen-simple . . . looked like there were quite a few other options
3 - are there other good parameters you would recommend?
Thanks,
Nathan
On Jun 30, 2013, at 12:37 PM, Daniel Povey wrote:
> In case you are not on the list or did not get the reply.
> Please cc the list if you reply.
>
>
> ---------- Forwarded message ----------
> From: Daniel Povey <dp...@gm...>
> Date: Sun, Jun 30, 2013 at 3:37 PM
> Subject: Re: [Kaldi-users] issues with decoding (self loops) in tri2a
> for 1-minute speech segments
> To: kal...@li...
>
>
> Can you describe the language model you use for the decoding phase?
> I'd like to know in order to understand what scenarios this is most
> likely to happen in.
> What values did you use for "beam" and "lattice-beam"? Typically you
> should be able to solve these problems by reducing "lattice-beam".
>
>
> Dan
>
>
> On Sun, Jun 30, 2013 at 3:26 PM, Mailing list used for User
> Communication and Updates <kal...@li...> wrote:
>>
>> Hello,
>>
>> I'm trying to decode long (1 minute) speech segments (multiple sentences)
>> having trained on short (<15 second) corpuses. For some reason I get
>> random failures of the following(even longer logs below):
>>
>> WARNING (gmm-latgen-simple:Close():kaldi-io.cc:444) Pipe compute-cmvn-stats
>> scp:data/test_childspeech/feats.scp ark:- | apply-cmvn --norm-vars=false
>> ark:- scp:data/test_childspeech/feats.scp ark:- | add-deltas ark:- ark:- |
>> had nonzero return status 36096
>> ERROR
>> (gmm-latgen-simple:EpsilonClosure():fstext/determinize-lattice-pruned-inl.h:664)
>> Lattice determinization aborted since looped more than 500000 times during
>> epsilon closure.
>>
>> I've had it decode 135 of these, but sometimes it will fail after 5-10 with
>> this error.
>>
>> My training model is the LDC9763 child's speech corpus, of which I use about
>> 4K sentences. If I test on a subset (100 not int he training set) I get
>> great results (using the tri2a method).
>>
>> The language model I use for testing is the target speakers' target sentence
>> (45 speakers speaking the same 3 sentences).
>>
>> Next, I was going to try breaking up the decoding into smaller segments
>> (those seem to work) as well as have a more accurate language model for the
>> decoding phase.
>>
>> Any ideas?
>>
>> Thanks,
>>
>> Nathan
>>
>>
>>
>>
>> LOG
>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285)
>> Rebuilding repository.
>> LOG
>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285)
>> Rebuilding repository.
>> LOG
>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285)
>> Rebuilding repository.
>> LOG
>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285)
>> Rebuilding repository.
>> LOG
>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285)
>> Rebuilding repository.
>> LOG
>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285)
>> Rebuilding repository.
>> LOG
>> (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.
>>
>> Nathan Dunn, Ph.D.
>> Scientific Programer
>> College of Arts and Science IT
>> 541-221-2418
>> nd...@ca...
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by Windows:
>>
>> Build for Windows Store.
>>
>> http://p.sf.net/sfu/windows-dev2dev
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>
|
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-30 19:37:11
|
Can you describe the language model you use for the decoding phase? I'd like to know in order to understand what scenarios this is most likely to happen in. What values did you use for "beam" and "lattice-beam"? Typically you should be able to solve these problems by reducing "lattice-beam". Dan On Sun, Jun 30, 2013 at 3:26 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > > Hello, > > I'm trying to decode long (1 minute) speech segments (multiple sentences) > having trained on short (<15 second) corpuses. For some reason I get > random failures of the following(even longer logs below): > > WARNING (gmm-latgen-simple:Close():kaldi-io.cc:444) Pipe compute-cmvn-stats > scp:data/test_childspeech/feats.scp ark:- | apply-cmvn --norm-vars=false > ark:- scp:data/test_childspeech/feats.scp ark:- | add-deltas ark:- ark:- | > had nonzero return status 36096 > ERROR > (gmm-latgen-simple:EpsilonClosure():fstext/determinize-lattice-pruned-inl.h:664) > Lattice determinization aborted since looped more than 500000 times during > epsilon closure. > > I've had it decode 135 of these, but sometimes it will fail after 5-10 with > this error. > > My training model is the LDC9763 child's speech corpus, of which I use about > 4K sentences. If I test on a subset (100 not int he training set) I get > great results (using the tri2a method). > > The language model I use for testing is the target speakers' target sentence > (45 speakers speaking the same 3 sentences). > > Next, I was going to try breaking up the decoding into smaller segments > (those seem to work) as well as have a more accurate language model for the > decoding phase. > > Any ideas? > > Thanks, > > Nathan > > > > > LOG > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > Rebuilding repository. > LOG > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > Rebuilding repository. > LOG > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > Rebuilding repository. > LOG > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > Rebuilding repository. > LOG > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > Rebuilding repository. > LOG > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) > Rebuilding repository. > LOG > (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl. > > Nathan Dunn, Ph.D. > Scientific Programer > College of Arts and Science IT > 541-221-2418 > nd...@ca... > > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-30 19:26:40
|
Hello, I'm trying to decode long (1 minute) speech segments (multiple sentences) having trained on short (<15 second) corpuses. For some reason I get random failures of the following(even longer logs below): WARNING (gmm-latgen-simple:Close():kaldi-io.cc:444) Pipe compute-cmvn-stats scp:data/test_childspeech/feats.scp ark:- | apply-cmvn --norm-vars=false ark:- scp:data/test_childspeech/feats.scp ark:- | add-deltas ark:- ark:- | had nonzero return status 36096 ERROR (gmm-latgen-simple:EpsilonClosure():fstext/determinize-lattice-pruned-inl.h:664) Lattice determinization aborted since looped more than 500000 times during epsilon closure. I've had it decode 135 of these, but sometimes it will fail after 5-10 with this error. My training model is the LDC9763 child's speech corpus, of which I use about 4K sentences. If I test on a subset (100 not int he training set) I get great results (using the tri2a method). The language model I use for testing is the target speakers' target sentence (45 speakers speaking the same 3 sentences). Next, I was going to try breaking up the decoding into smaller segments (those seem to work) as well as have a more accurate language model for the decoding phase. Any ideas? Thanks, Nathan LOG (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) Rebuilding repository. LOG (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) Rebuilding repository. LOG (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) Rebuilding repository. LOG (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) Rebuilding repository. LOG (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) Rebuilding repository. LOG (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl.h:285) Rebuilding repository. LOG (gmm-latgen-simple:RebuildRepository():fstext/determinize-lattice-pruned-inl. Nathan Dunn, Ph.D. Scientific Programer College of Arts and Science IT 541-221-2418 nd...@ca... |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-28 15:06:49
|
It's not the same as that. Each machine does SGD separately and, periodically, the parameters are averaged across machines. Dan On Fri, Jun 28, 2013 at 11:03 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Wow, nice. > Does the implementation similar to the Jeff Dean's paper Large Scale > Distributed Deep Networks > (http://www.cs.toronto.edu/~ranzato/publications/DistBeliefNIPS2012_withAppendix.pdf) > ? > Does Kaldi use Asynchronous SGD? > > Please give me a brief description. > > Thanks, > Lahiru > > > On Fri, Jun 28, 2013 at 10:28 PM, Mailing list used for User Communication > and Updates <kal...@li...> wrote: >> >> It's on multiple machines and also multiple threads per machine. >> Dan >> >> >> On Fri, Jun 28, 2013 at 2:05 AM, Mailing list used for User >> Communication and Updates <kal...@li...> wrote: >> > Thanks guys :-) >> > >> > Dan, is your setup for distributed training? Or is it only parallelize >> > with >> > in a single machine? >> > >> > Thanks, >> > Lahiru >> > >> > >> > >> > On Fri, Jun 28, 2013 at 5:29 AM, Mailing list used for User >> > Communication >> > and Updates <kal...@li...> wrote: >> >> >> >> In my setup there is RBM pre-training: >> >> http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf >> >> <http://www.cs.toronto.edu/%7Ehinton/absps/guideTR.pdf> >> >> followed by per-frame cross entropy training and sMBR training: >> >> http://www.danielpovey.com/files/2013_interspeech_dnn.pdf >> >> >> >> >> >> Dne 27.6.2013 13:21, Mailing list used for User Communication and >> >> Updates napsal(a): >> >> > There are basically two setups there: Karel's setup, generally called >> >> > run_dnn.sh or run_nnet.sh, which is for GPUs, and my setup, called >> >> > run_nnet_cpu.sh, which is for CPUs in parallel. Karel's setup may >> >> > have an ICASSP paper, Karel can tell you. Mine is mostly >> >> > unpublished. >> >> > >> >> > Dan >> >> > >> >> > >> >> > On Thu, Jun 27, 2013 at 5:31 AM, Mailing list used for User >> >> > Communication and Updates <kal...@li...> wrote: >> >> >> Hi All, >> >> >> >> >> >> I am in the process of running the wsj/s5 recipe. Now I am about the >> >> >> run DNN >> >> >> experiments and specifically interested in the DNN training. I am >> >> >> planning >> >> >> to look into the DNN code for more understanding. Since there are >> >> >> many >> >> >> DNN >> >> >> variants, could anyone tell me the papers Kalid DNN implementation >> >> >> represents? >> >> >> >> >> >> Thanks, >> >> >> Lahiru >> >> >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> >> This SF.net email is sponsored by Windows: >> >> >> >> >> >> Build for Windows Store. >> >> >> >> >> >> http://p.sf.net/sfu/windows-dev2dev >> >> >> _______________________________________________ >> >> >> Kaldi-users mailing list >> >> >> Kal...@li... >> >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> >> >> >> > >> >> > >> >> > ------------------------------------------------------------------------------ >> >> > This SF.net email is sponsored by Windows: >> >> > >> >> > Build for Windows Store. >> >> > >> >> > http://p.sf.net/sfu/windows-dev2dev >> >> > _______________________________________________ >> >> > Kaldi-users mailing list >> >> > Kal...@li... >> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> >> >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> This SF.net email is sponsored by Windows: >> >> >> >> Build for Windows Store. >> >> >> >> http://p.sf.net/sfu/windows-dev2dev >> >> _______________________________________________ >> >> Kaldi-users mailing list >> >> Kal...@li... >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> > >> > >> > >> > >> > ------------------------------------------------------------------------------ >> > This SF.net email is sponsored by Windows: >> > >> > Build for Windows Store. >> > >> > http://p.sf.net/sfu/windows-dev2dev >> > _______________________________________________ >> > Kaldi-users mailing list >> > Kal...@li... >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >> > >> >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-28 15:04:06
|
Wow, nice. Does the implementation similar to the Jeff Dean's paper Large Scale Distributed Deep Networks ( http://www.cs.toronto.edu/~ranzato/publications/DistBeliefNIPS2012_withAppendix.pdf) ? Does Kaldi use Asynchronous SGD? Please give me a brief description. Thanks, Lahiru On Fri, Jun 28, 2013 at 10:28 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > It's on multiple machines and also multiple threads per machine. > Dan > > > On Fri, Jun 28, 2013 at 2:05 AM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: > > Thanks guys :-) > > > > Dan, is your setup for distributed training? Or is it only parallelize > with > > in a single machine? > > > > Thanks, > > Lahiru > > > > > > > > On Fri, Jun 28, 2013 at 5:29 AM, Mailing list used for User Communication > > and Updates <kal...@li...> wrote: > >> > >> In my setup there is RBM pre-training: > >> http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf > >> <http://www.cs.toronto.edu/%7Ehinton/absps/guideTR.pdf> > >> followed by per-frame cross entropy training and sMBR training: > >> http://www.danielpovey.com/files/2013_interspeech_dnn.pdf > >> > >> > >> Dne 27.6.2013 13:21, Mailing list used for User Communication and > >> Updates napsal(a): > >> > There are basically two setups there: Karel's setup, generally called > >> > run_dnn.sh or run_nnet.sh, which is for GPUs, and my setup, called > >> > run_nnet_cpu.sh, which is for CPUs in parallel. Karel's setup may > >> > have an ICASSP paper, Karel can tell you. Mine is mostly unpublished. > >> > > >> > Dan > >> > > >> > > >> > On Thu, Jun 27, 2013 at 5:31 AM, Mailing list used for User > >> > Communication and Updates <kal...@li...> wrote: > >> >> Hi All, > >> >> > >> >> I am in the process of running the wsj/s5 recipe. Now I am about the > >> >> run DNN > >> >> experiments and specifically interested in the DNN training. I am > >> >> planning > >> >> to look into the DNN code for more understanding. Since there are > many > >> >> DNN > >> >> variants, could anyone tell me the papers Kalid DNN implementation > >> >> represents? > >> >> > >> >> Thanks, > >> >> Lahiru > >> >> > >> >> > >> >> > ------------------------------------------------------------------------------ > >> >> This SF.net email is sponsored by Windows: > >> >> > >> >> Build for Windows Store. > >> >> > >> >> http://p.sf.net/sfu/windows-dev2dev > >> >> _______________________________________________ > >> >> Kaldi-users mailing list > >> >> Kal...@li... > >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > >> >> > >> > > >> > > ------------------------------------------------------------------------------ > >> > This SF.net email is sponsored by Windows: > >> > > >> > Build for Windows Store. > >> > > >> > http://p.sf.net/sfu/windows-dev2dev > >> > _______________________________________________ > >> > Kaldi-users mailing list > >> > Kal...@li... > >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users > >> > >> > >> > >> > ------------------------------------------------------------------------------ > >> This SF.net email is sponsored by Windows: > >> > >> Build for Windows Store. > >> > >> http://p.sf.net/sfu/windows-dev2dev > >> _______________________________________________ > >> Kaldi-users mailing list > >> Kal...@li... > >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > > > > > ------------------------------------------------------------------------------ > > This SF.net email is sponsored by Windows: > > > > Build for Windows Store. > > > > http://p.sf.net/sfu/windows-dev2dev > > _______________________________________________ > > Kaldi-users mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-28 14:28:48
|
It's on multiple machines and also multiple threads per machine. Dan On Fri, Jun 28, 2013 at 2:05 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Thanks guys :-) > > Dan, is your setup for distributed training? Or is it only parallelize with > in a single machine? > > Thanks, > Lahiru > > > > On Fri, Jun 28, 2013 at 5:29 AM, Mailing list used for User Communication > and Updates <kal...@li...> wrote: >> >> In my setup there is RBM pre-training: >> http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf >> <http://www.cs.toronto.edu/%7Ehinton/absps/guideTR.pdf> >> followed by per-frame cross entropy training and sMBR training: >> http://www.danielpovey.com/files/2013_interspeech_dnn.pdf >> >> >> Dne 27.6.2013 13:21, Mailing list used for User Communication and >> Updates napsal(a): >> > There are basically two setups there: Karel's setup, generally called >> > run_dnn.sh or run_nnet.sh, which is for GPUs, and my setup, called >> > run_nnet_cpu.sh, which is for CPUs in parallel. Karel's setup may >> > have an ICASSP paper, Karel can tell you. Mine is mostly unpublished. >> > >> > Dan >> > >> > >> > On Thu, Jun 27, 2013 at 5:31 AM, Mailing list used for User >> > Communication and Updates <kal...@li...> wrote: >> >> Hi All, >> >> >> >> I am in the process of running the wsj/s5 recipe. Now I am about the >> >> run DNN >> >> experiments and specifically interested in the DNN training. I am >> >> planning >> >> to look into the DNN code for more understanding. Since there are many >> >> DNN >> >> variants, could anyone tell me the papers Kalid DNN implementation >> >> represents? >> >> >> >> Thanks, >> >> Lahiru >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> This SF.net email is sponsored by Windows: >> >> >> >> Build for Windows Store. >> >> >> >> http://p.sf.net/sfu/windows-dev2dev >> >> _______________________________________________ >> >> Kaldi-users mailing list >> >> Kal...@li... >> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> >> > >> > ------------------------------------------------------------------------------ >> > This SF.net email is sponsored by Windows: >> > >> > Build for Windows Store. >> > >> > http://p.sf.net/sfu/windows-dev2dev >> > _______________________________________________ >> > Kaldi-users mailing list >> > Kal...@li... >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-28 06:05:10
|
Thanks guys :-) Dan, is your setup for distributed training? Or is it only parallelize with in a single machine? Thanks, Lahiru On Fri, Jun 28, 2013 at 5:29 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > In my setup there is RBM pre-training: > http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf > <http://www.cs.toronto.edu/%7Ehinton/absps/guideTR.pdf> > followed by per-frame cross entropy training and sMBR training: > http://www.danielpovey.com/files/2013_interspeech_dnn.pdf > > > Dne 27.6.2013 13:21, Mailing list used for User Communication and > Updates napsal(a): > > There are basically two setups there: Karel's setup, generally called > > run_dnn.sh or run_nnet.sh, which is for GPUs, and my setup, called > > run_nnet_cpu.sh, which is for CPUs in parallel. Karel's setup may > > have an ICASSP paper, Karel can tell you. Mine is mostly unpublished. > > > > Dan > > > > > > On Thu, Jun 27, 2013 at 5:31 AM, Mailing list used for User > > Communication and Updates <kal...@li...> wrote: > >> Hi All, > >> > >> I am in the process of running the wsj/s5 recipe. Now I am about the > run DNN > >> experiments and specifically interested in the DNN training. I am > planning > >> to look into the DNN code for more understanding. Since there are many > DNN > >> variants, could anyone tell me the papers Kalid DNN implementation > >> represents? > >> > >> Thanks, > >> Lahiru > >> > >> > ------------------------------------------------------------------------------ > >> This SF.net email is sponsored by Windows: > >> > >> Build for Windows Store. > >> > >> http://p.sf.net/sfu/windows-dev2dev > >> _______________________________________________ > >> Kaldi-users mailing list > >> Kal...@li... > >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > >> > > > ------------------------------------------------------------------------------ > > This SF.net email is sponsored by Windows: > > > > Build for Windows Store. > > > > http://p.sf.net/sfu/windows-dev2dev > > _______________________________________________ > > Kaldi-users mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-27 21:30:03
|
In my setup there is RBM pre-training: http://www.cs.toronto.edu/~hinton/absps/guideTR.pdf <http://www.cs.toronto.edu/%7Ehinton/absps/guideTR.pdf> followed by per-frame cross entropy training and sMBR training: http://www.danielpovey.com/files/2013_interspeech_dnn.pdf Dne 27.6.2013 13:21, Mailing list used for User Communication and Updates napsal(a): > There are basically two setups there: Karel's setup, generally called > run_dnn.sh or run_nnet.sh, which is for GPUs, and my setup, called > run_nnet_cpu.sh, which is for CPUs in parallel. Karel's setup may > have an ICASSP paper, Karel can tell you. Mine is mostly unpublished. > > Dan > > > On Thu, Jun 27, 2013 at 5:31 AM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: >> Hi All, >> >> I am in the process of running the wsj/s5 recipe. Now I am about the run DNN >> experiments and specifically interested in the DNN training. I am planning >> to look into the DNN code for more understanding. Since there are many DNN >> variants, could anyone tell me the papers Kalid DNN implementation >> represents? >> >> Thanks, >> Lahiru >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-27 17:21:31
|
There are basically two setups there: Karel's setup, generally called run_dnn.sh or run_nnet.sh, which is for GPUs, and my setup, called run_nnet_cpu.sh, which is for CPUs in parallel. Karel's setup may have an ICASSP paper, Karel can tell you. Mine is mostly unpublished. Dan On Thu, Jun 27, 2013 at 5:31 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi All, > > I am in the process of running the wsj/s5 recipe. Now I am about the run DNN > experiments and specifically interested in the DNN training. I am planning > to look into the DNN code for more understanding. Since there are many DNN > variants, could anyone tell me the papers Kalid DNN implementation > represents? > > Thanks, > Lahiru > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-27 14:13:40
|
Thanks for letting us know. (Paul Dixon, I imagine?) Dan On Thu, Jun 27, 2013 at 7:44 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi Giles, > > This is great, I just tried the new version and it is much faster. > I compared arpa2fst to ngramread from opengrm and arpa2fst is nearly two > times faster. > (Both were compiled with -O2 -g flags) > > Thanks > > Paul > > > > On 19 June 2013 06:00, Mailing list used for User Communication and Updates > <kal...@li...> wrote: >> >> I just committed an improved version of arpa2fst. >> It is twice as fast, and memory use has been reduced by 20% to 50%. >> >> So now for a typical 4-gram LM: >> >> on-disk arpa file: 3.8 GB >> arpa2fst memory usage: 20 GB >> in-memory size of G fst alone: 8.9 GB >> on disk size of G fst: 3.7 GB >> >> - Gilles >> >> >> >> ------------------------------------------------------------------------------ >> This SF.net email is sponsored by Windows: >> >> Build for Windows Store. >> >> http://p.sf.net/sfu/windows-dev2dev >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-27 11:44:44
|
Hi Giles, This is great, I just tried the new version and it is much faster. I compared arpa2fst to ngramread from opengrm and arpa2fst is nearly two times faster. (Both were compiled with -O2 -g flags) Thanks Paul On 19 June 2013 06:00, Mailing list used for User Communication and Updates <kal...@li...> wrote: > I just committed an improved version of arpa2fst. > It is twice as fast, and memory use has been reduced by 20% to 50%. > > So now for a typical 4-gram LM: > > on-disk arpa file: 3.8 GB > arpa2fst memory usage: 20 GB > in-memory size of G fst alone: 8.9 GB > on disk size of G fst: 3.7 GB > > - Gilles > > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-27 09:32:05
|
Hi All, I am in the process of running the wsj/s5 recipe. Now I am about the run DNN experiments and specifically interested in the DNN training. I am planning to look into the DNN code for more understanding. Since there are many DNN variants, could anyone tell me the papers Kalid DNN implementation represents? Thanks, Lahiru |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-26 18:03:07
|
I already responded, perhaps your email is not going to your account from sourceforge? On Wed, Jun 26, 2013 at 2:01 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi all, > > The HMM topology in standard Kaldi recipe doesn't seem to have state > skipping, eg, hmm state 0 and 1 doesn't go to state 3 directly. Would this > introduce a limitation that a phone must be pronounced for at least 3 > frames (30ms), eg, takes 3 frames to transition out? > > The reason for asking is that we have seen some poor decoding accuracy for > very fast speeches. In the fast speech segments , phones were pronounced > definitely less than 30ms. This results in very high phone errors. > Separate gmm-align experiments in the same segments also point to this as > well. The smallest phone alignment window from gmm-align is 30ms. > > We probably will experiment with introducing skipping in HMM topology. > Before we start, any heads-ups or particular reasons that this may not be a > good idea? Or, am I missing something entirely? > > -- > Thanks > > Ben Jiang > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-26 18:01:51
|
Hi all, The HMM topology in standard Kaldi recipe doesn't seem to have state skipping, eg, hmm state 0 and 1 doesn't go to state 3 directly. Would this introduce a limitation that a phone must be pronounced for at least 3 frames (30ms), eg, takes 3 frames to transition out? The reason for asking is that we have seen some poor decoding accuracy for very fast speeches. In the fast speech segments , phones were pronounced definitely less than 30ms. This results in very high phone errors. Separate gmm-align experiments in the same segments also point to this as well. The smallest phone alignment window from gmm-align is 30ms. We probably will experiment with introducing skipping in HMM topology. Before we start, any heads-ups or particular reasons that this may not be a good idea? Or, am I missing something entirely? -- Thanks Ben Jiang |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-26 17:09:00
|
You can easily introduce skipping, just edit the topo.proto file or whatever it is called. If a phone is completely skippable (i.e. there is path that involves skipping all the states), it may cause problems though. Dan On Wed, Jun 26, 2013 at 11:37 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi all, > > The standard Kaldi HMM topology doesn't seem to have state skipping, eg, hmm > state 0 and 1 doesn't go to state 3 directly. Would this introduce a > limitation that a phone must be pronounced for at least 3 frames (30ms)? > > The reason for asking is that we have seen some poor decoding accuracy for > very fast speeches. Our analysis shows rather high phone error. Some > phones in the fast speech segments were pronounced definitely less than > 30ms. gmm-align seems to point this as well. The smallest phone alignment > window from gmm-align is 30ms. > > We probably will experiment with introducing skipping in HMM topology. > Before we start, any heads-ups? Potential pointers/ideas? Or, am I missing > something entirely? > > -- > Thanks > Ben Jiang > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by Windows: > > Build for Windows Store. > > http://p.sf.net/sfu/windows-dev2dev > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-26 16:38:07
|
Hi all, The standard Kaldi HMM topology doesn't seem to have state skipping, eg, hmm state 0 and 1 doesn't go to state 3 directly. Would this introduce a limitation that a phone must be pronounced for at least 3 frames (30ms)? The reason for asking is that we have seen some poor decoding accuracy for very fast speeches. Our analysis shows rather high phone error. Some phones in the fast speech segments were pronounced definitely less than 30ms. gmm-align seems to point this as well. The smallest phone alignment window from gmm-align is 30ms. We probably will experiment with introducing skipping in HMM topology. Before we start, any heads-ups? Potential pointers/ideas? Or, am I missing something entirely? -- Thanks Ben Jiang |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-20 19:46:14
|
Everyone, I moved some stuff around in the latest commit-- to compile it will help do to make depend make clean make # or make -j 8 Dan ---------- Forwarded message ---------- From: Ho Yin Chan <ric...@gm...> Date: Thu, Jun 20, 2013 at 3:42 PM Subject: Kaldi trunk To: Daniel Povey <dp...@gm...> In src/sgmm2bin, latest update in main trunk fail to compile [root@ sgmm2bin]# make g++ -rdynamic sgmm2-est.o ../lat/kaldi-lat.a ../decoder/kaldi-decoder.a ../feat/kaldi-feature.a ../sgmm2/kaldi-sgmm.a ../transform/kaldi-transform.a ../gmm/kaldi-gmm.a ../hmm/kaldi-hmm.a ../tree/kaldi-tree.a ../matrix/kaldi-matrix.a ../thread/kaldi-thread.a ../util/kaldi-util.a ../base/kaldi-base.a /home/ricky/softwares/kaldi-code/tools/openfst/lib/libfst.a -ldl /usr/local/atlas/lib/liblapack.a /usr/local/atlas/lib/libcblas.a /usr/local/atlas/lib/libatlas.a /usr/local/atlas/lib/libf77blas.a -lm -lpthread -o sgmm2-est ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::Sgmm2LikelihoodCache::NextFrame()': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:35: multiple definition of `kaldi::Sgmm2LikelihoodCache::NextFrame()' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:35: first defined here ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::Pdf2Group(int) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:192: multiple definition of `kaldi::AmSgmm2::Pdf2Group(int) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:192: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::Pdf2Group(int) const' changed from 96 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 102 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::InitializeCovars()': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1230: multiple definition of `kaldi::AmSgmm2::InitializeCovars()' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1230: first defined here ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::Sgmm2GauPost::Read(std::basic_istream<char, std::char_traits<char> >&, bool)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1454: multiple definition of `kaldi::Sgmm2GauPost::Read(std::basic_istream<char, std::char_traits<char> >&, bool)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1454: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::Sgmm2GauPost::Read(std::basic_istream<char, std::char_traits<char> >&, bool)' changed from 619 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 633 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::Sgmm2GauPost::Write(std::basic_ostream<char, std::char_traits<char> >&, bool) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1436: multiple definition of `kaldi::Sgmm2GauPost::Write(std::basic_ostream<char, std::char_traits<char> >&, bool) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1436: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::Sgmm2GauPost::Write(std::basic_ostream<char, std::char_traits<char> >&, bool) const' changed from 400 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 414 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::GaussianSelection(kaldi::Sgmm2GselectConfig const&, kaldi::VectorBase<float> const&, std::vector<int, std::allocator<int> >*) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1390: multiple definition of `kaldi::AmSgmm2::GaussianSelection(kaldi::Sgmm2GselectConfig const&, kaldi::VectorBase<float> const&, std::vector<int, std::allocator<int> >*) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1390: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::GaussianSelection(kaldi::Sgmm2GselectConfig const&, kaldi::VectorBase<float> const&, std::vector<int, std::allocator<int> >*) const' changed from 1633 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1657 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputePerSpkDerivedVars(kaldi::Sgmm2PerSpkDerivedVars*) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1351: multiple definition of `kaldi::AmSgmm2::ComputePerSpkDerivedVars(kaldi::Sgmm2PerSpkDerivedVars*) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1351: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComputePerSpkDerivedVars(kaldi::Sgmm2PerSpkDerivedVars*) const' changed from 1147 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1165 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputePdfMappings()': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:68: multiple definition of `kaldi::AmSgmm2::ComputePdfMappings()' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:68: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComputePdfMappings()' changed from 630 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 642 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::InitializeVecsAndSubstateWeights(float)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1189: multiple definition of `kaldi::AmSgmm2::InitializeVecsAndSubstateWeights(float)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1189: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::InitializeVecsAndSubstateWeights(float)' changed from 1312 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1324 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::CopyGlobalsInitVecs(kaldi::AmSgmm2 const&, std::vector<int, std::allocator<int> > const&, float)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1167: multiple definition of `kaldi::AmSgmm2::CopyGlobalsInitVecs(kaldi::AmSgmm2 const&, std::vector<int, std::allocator<int> > const&, float)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1167: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::CopyGlobalsInitVecs(kaldi::AmSgmm2 const&, std::vector<int, std::allocator<int> > const&, float)' changed from 450 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 456 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::CopyFromSgmm2(kaldi::AmSgmm2 const&, bool, bool)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:418: multiple definition of `kaldi::AmSgmm2::CopyFromSgmm2(kaldi::AmSgmm2 const&, bool, bool)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:418: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::CopyFromSgmm2(kaldi::AmSgmm2 const&, bool, bool)' changed from 691 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 703 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::InitializeNu(int, kaldi::Matrix<float> const&, bool)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1145: multiple definition of `kaldi::AmSgmm2::InitializeNu(int, kaldi::Matrix<float> const&, bool)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1145: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::InitializeNu(int, kaldi::Matrix<float> const&, bool)' changed from 541 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 559 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::IncreaseSpkSpaceDim(int, kaldi::Matrix<float> const&, bool)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:750: multiple definition of `kaldi::AmSgmm2::IncreaseSpkSpaceDim(int, kaldi::Matrix<float> const&, bool)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:750: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::IncreaseSpkSpaceDim(int, kaldi::Matrix<float> const&, bool)' changed from 1883 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1935 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::IncreasePhoneSpaceDim(int, kaldi::Matrix<float> const&)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:701: multiple definition of `kaldi::AmSgmm2::IncreasePhoneSpaceDim(int, kaldi::Matrix<float> const&)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:701: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::IncreasePhoneSpaceDim(int, kaldi::Matrix<float> const&)' changed from 2149 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 2201 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::InitializeMw(int, kaldi::Matrix<float> const&)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1121: multiple definition of `kaldi::AmSgmm2::InitializeMw(int, kaldi::Matrix<float> const&)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1121: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::InitializeMw(int, kaldi::Matrix<float> const&)' changed from 734 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 752 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputeWeights()': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:797: multiple definition of `kaldi::AmSgmm2::ComputeWeights()' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:797: first defined here ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputeGammaI(kaldi::Vector<float> const&, kaldi::Vector<float>*) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:47: multiple definition of `kaldi::AmSgmm2::ComputeGammaI(kaldi::Vector<float> const&, kaldi::Vector<float>*) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:47: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComputeGammaI(kaldi::Vector<float> const&, kaldi::Vector<float>*) const' changed from 626 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 632 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputeHsmFromModel(std::vector<kaldi::SpMatrix<float>, std::allocator<kaldi::SpMatrix<float> > > const&, kaldi::Vector<float> const&, kaldi::SpMatrix<float>*, float) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:1246: multiple definition of `kaldi::AmSgmm2::ComputeHsmFromModel(std::vector<kaldi::SpMatrix<float>, std::allocator<kaldi::SpMatrix<float> > > const&, kaldi::Vector<float> const&, kaldi::SpMatrix<float>*, float) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:1246: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComputeHsmFromModel(std::vector<kaldi::SpMatrix<float>, std::allocator<kaldi::SpMatrix<float> > > const&, kaldi::Vector<float> const&, kaldi::SpMatrix<float>*, float) const' changed from 996 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1022 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputeFmllrPreXform(kaldi::Vector<float> const&, kaldi::Matrix<float>*, kaldi::Matrix<float>*, kaldi::Vector<float>*) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:963: multiple definition of `kaldi::AmSgmm2::ComputeFmllrPreXform(kaldi::Vector<float> const&, kaldi::Matrix<float>*, kaldi::Matrix<float>*, kaldi::Vector<float>*) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:963: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComputeFmllrPreXform(kaldi::Vector<float> const&, kaldi::Matrix<float>*, kaldi::Matrix<float>*, kaldi::Vector<float>*) const' changed from 3159 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 3181 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::GetDjms(int, int, kaldi::Sgmm2PerSpkDerivedVars*) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:944: multiple definition of `kaldi::AmSgmm2::GetDjms(int, int, kaldi::Sgmm2PerSpkDerivedVars*) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:944: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::GetDjms(int, int, kaldi::Sgmm2PerSpkDerivedVars*) const' changed from 312 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 318 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputePerFrameVars(kaldi::VectorBase<float> const&, std::vector<int, std::allocator<int> > const&, kaldi::Sgmm2PerSpkDerivedVars const&, kaldi::Sgmm2PerFrameDerivedVars*) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:446: multiple definition of `kaldi::AmSgmm2::ComputePerFrameVars(kaldi::VectorBase<float> const&, std::vector<int, std::allocator<int> > const&, kaldi::Sgmm2PerSpkDerivedVars const&, kaldi::Sgmm2PerFrameDerivedVars*) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:446: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComputePerFrameVars(kaldi::VectorBase<float> const&, std::vector<int, std::allocator<int> > const&, kaldi::Sgmm2PerSpkDerivedVars const&, kaldi::Sgmm2PerFrameDerivedVars*) const' changed from 1293 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1299 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputeNormalizersInternal(int, int, int*, double*)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:870: multiple definition of `kaldi::AmSgmm2::ComputeNormalizersInternal(int, int, int*, double*)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:870: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComputeNormalizersInternal(int, int, int*, double*)' changed from 2608 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 2636 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputeNormalizers()': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:852: multiple definition of `kaldi::AmSgmm2::ComputeNormalizers()' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:852: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComputeNormalizers()' changed from 741 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 766 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComputeDerivedVars()': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:811: multiple definition of `kaldi::AmSgmm2::ComputeDerivedVars()' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:811: first defined here ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::InitializeFromFullGmm(kaldi::FullGmm const&, std::vector<int, std::allocator<int> > const&, int, int, bool, float)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:382: multiple definition of `kaldi::AmSgmm2::InitializeFromFullGmm(kaldi::FullGmm const&, std::vector<int, std::allocator<int> > const&, int, int, bool, float)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:382: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::InitializeFromFullGmm(kaldi::FullGmm const&, std::vector<int, std::allocator<int> > const&, int, int, bool, float)' changed from 1273 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1305 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::SplitSubstatesInGroup(kaldi::Vector<float> const&, kaldi::Sgmm2SplitSubstatesConfig const&, kaldi::SpMatrix<float> const&, int, int)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:604: multiple definition of `kaldi::AmSgmm2::SplitSubstatesInGroup(kaldi::Vector<float> const&, kaldi::Sgmm2SplitSubstatesConfig const&, kaldi::SpMatrix<float> const&, int, int)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:604: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::SplitSubstatesInGroup(kaldi::Vector<float> const&, kaldi::Sgmm2SplitSubstatesConfig const&, kaldi::SpMatrix<float> const&, int, int)' changed from 2034 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 2042 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::SplitSubstates(kaldi::Vector<float> const&, kaldi::Sgmm2SplitSubstatesConfig const&)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:659: multiple definition of `kaldi::AmSgmm2::SplitSubstates(kaldi::Vector<float> const&, kaldi::Sgmm2SplitSubstatesConfig const&)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:659: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::SplitSubstates(kaldi::Vector<float> const&, kaldi::Sgmm2SplitSubstatesConfig const&)' changed from 1359 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1387 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::ComponentPosteriors(kaldi::Sgmm2PerFrameDerivedVars const&, int, kaldi::Sgmm2PerSpkDerivedVars*, kaldi::Matrix<float>*) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:578: multiple definition of `kaldi::AmSgmm2::ComponentPosteriors(kaldi::Sgmm2PerFrameDerivedVars const&, int, kaldi::Sgmm2PerSpkDerivedVars*, kaldi::Matrix<float>*) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:578: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::ComponentPosteriors(kaldi::Sgmm2PerFrameDerivedVars const&, int, kaldi::Sgmm2PerSpkDerivedVars*, kaldi::Matrix<float>*) const' changed from 362 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 374 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::LogLikelihood(kaldi::Sgmm2PerFrameDerivedVars const&, int, kaldi::Sgmm2LikelihoodCache*, kaldi::Sgmm2PerSpkDerivedVars*, float) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:522: multiple definition of `kaldi::AmSgmm2::LogLikelihood(kaldi::Sgmm2PerFrameDerivedVars const&, int, kaldi::Sgmm2LikelihoodCache*, kaldi::Sgmm2PerSpkDerivedVars*, float) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:522: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::LogLikelihood(kaldi::Sgmm2PerFrameDerivedVars const&, int, kaldi::Sgmm2LikelihoodCache*, kaldi::Sgmm2PerSpkDerivedVars*, float) const' changed from 718 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 730 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::Check(bool)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:272: multiple definition of `kaldi::AmSgmm2::Check(bool)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:272: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::Check(bool)' changed from 3148 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 3322 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::Write(std::basic_ostream<char, std::char_traits<char> >&, bool, unsigned short) const': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:201: multiple definition of `kaldi::AmSgmm2::Write(std::basic_ostream<char, std::char_traits<char> >&, bool, unsigned short) const' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:201: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::Write(std::basic_ostream<char, std::char_traits<char> >&, bool, unsigned short) const' changed from 1536 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 1588 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) ../sgmm2/kaldi-sgmm.a(am-sgmm2.o): In function `kaldi::AmSgmm2::Read(std::basic_istream<char, std::char_traits<char> >&, bool)': /media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm2.cc:85: multiple definition of `kaldi::AmSgmm2::Read(std::basic_istream<char, std::char_traits<char> >&, bool)' ../sgmm2/kaldi-sgmm.a(am-sgmm.o):/media/disk-1/exist_user/ricky/softwares/kaldi-code/src/sgmm2/am-sgmm.cc:85: first defined here /usr/bin/ld: Warning: size of symbol `kaldi::AmSgmm2::Read(std::basic_istream<char, std::char_traits<char> >&, bool)' changed from 2986 in ../sgmm2/kaldi-sgmm.a(am-sgmm.o) to 3048 in ../sgmm2/kaldi-sgmm.a(am-sgmm2.o) collect2: ld returned 1 exit status make: *** [sgmm2-est] Error 1 |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-18 21:00:49
|
I just committed an improved version of arpa2fst. It is twice as fast, and memory use has been reduced by 20% to 50%. So now for a typical 4-gram LM: on-disk arpa file: 3.8 GB arpa2fst memory usage: 20 GB in-memory size of G fst alone: 8.9 GB on disk size of G fst: 3.7 GB - Gilles |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-06-18 14:47:28
|
I just committed a fix to all the denlat-generating scripts so they check
that HCLG.fst is newer than the model we're using, before skipping its
creation. Thanks for pointing out the relevant code.
I have not tested this fix but it's pretty simple:
echo "Compiling decoding graph in $dir/dengraph"
-if [ -s $dir/dengraph/HCLG.fst ]; then
+if [ -s $dir/dengraph/HCLG.fst ] && [ $dir/dengraph/HCLG.fst -nt
$srcdir/final.mdl ]; then
echo "Graph $dir/dengraph/HCLG.fst already exists: skipping graph
creation."
Dan
On Tue, Jun 18, 2013 at 3:03 AM, Mailing list used for User Communication
and Updates <kal...@li...> wrote:
> Thank you for your prompt and accurate replies. The script was indeed
> finding an incompatible HCLG.fst from a previous run and would not update
> it:
>
> if [ -s $dir/dengraph/HCLG.fst ]; then
> echo Not creating denominator graph $dir/dengraph/HCLG.fst since it
> already exists.
> else
> scripts/mkgraph.sh $dir/lang $alidir $dir/dengraph || exit 1;
> fi
>
> I removed the old fst and so it was rebuilt … Things are working fine now.
>
> nassos
>
> On Jun 17, 2013, at 11:26 PM, Mailing list used for User Communication and
> Updates wrote:
>
> Is not this a mismatch between feature and model dimensions?
> Petr
>
>
>
> 2013/6/17 Mailing list used for User Communication and Updates <
> kal...@li...>
>
>> It looks to me like your HCLG.fst may be out of date with respect to your
>> model-- perhaps you changed your model and the HCLG.fst did not get rebuilt?
>> Dan
>>
>>
>>
>> On Mon, Jun 17, 2013 at 1:51 PM, Mailing list used for User Communication
>> and Updates <kal...@li...> wrote:
>>
>>> Hi all,
>>>
>>> I'm trying to run the s3 recipe for WSJ and I'm running into a problem I
>>> was wondering whether you could shed any light upon:
>>>
>>> The script run.sh works fine up to steps/align_lda_mllt.sh. However,
>>> when I am running:
>>>
>>> steps/make_denlats_lda_etc.sh --num-jobs 4 --cmd "$train_cmd" \
>>> data/train_si84 data/lang exp/tri2b_ali_si84 exp/tri2b_denlats_si84
>>>
>>> I get a KALDI_ASSERT error. I updated to the most recent version of the
>>> trunk and retried but with no effect.
>>>
>>> More specifically, I get the following output in one of the log files
>>> (I'm running the recipe on a cluster and I'm submitting to a queue using 20
>>> jobs):
>>>
>>> =====================================
>>> >> cat wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>>
>>> Running on ro
>>> Started at Tue Jun 18 05:27:00 EEST 2013
>>> gmm-latgen-faster --beam=13.0 --lattice-beam=7.0 --acoustic-scale=0.1
>>> --max-mem=20000000 --max-active=5000
>>> --word-symbol-table=data/lang/words.txt exp/tri2b_ali_si84/final.mdl
>>> exp/tri2b_denlats_si84/dengraph/HCLG.fst 'ark:apply-cmvn --norm-vars=false
>>> --utt2spk=ark:data/train_si84/split20/24/utt2spk
>>> ark:exp/tri2b_ali_si84/24.cmvn scp:data/train_si84/split20/24/feats.scp
>>> ark:- | splice-feats ark:- ark:- | transform-feats
>>> exp/tri2b_ali_si84/final.mat ark:- ark:- |' 'ark:|gzip -c
>>> >exp/tri2b_denlats_si84/lat.24.gz'
>>> splice-feats ark:- ark:-
>>> apply-cmvn --norm-vars=false
>>> --utt2spk=ark:data/train_si84/split20/24/utt2spk
>>> ark:exp/tri2b_ali_si84/24.cmvn scp:data/train_si84/split20/24/feats.scp
>>> ark:-
>>> transform-feats exp/tri2b_ali_si84/final.mat ark:- ark:-
>>> KALDI_ASSERT: at
>>> gmm-latgen-faster:TransitionIdToPdf:hmm/transition-model.h:309, failed:
>>> static_cast<size_t>(trans_id) < id2state_.size()
>>> Stack trace is:
>>> kaldi::KaldiGetStackTrace()
>>> kaldi::KaldiAssertFailure_(char const*, char const*, int, char const*)
>>> kaldi::TransitionModel::TransitionIdToPdf(int) const
>>> kaldi::DecodableAmDiagGmmScaled::LogLikelihood(int, int)
>>> kaldi::LatticeFasterDecoder::ProcessEmitting(kaldi::DecodableInterface*,
>>> int)
>>> kaldi::LatticeFasterDecoder::Decode(kaldi::DecodableInterface*)
>>> kaldi::DecodeUtteranceLatticeFaster(kaldi::LatticeFasterDecoder&,
>>> kaldi::DecodableInterface&, fst::SymbolTable const*, std::string, double,
>>> bool, bool, kaldi::TableWriter<kaldi::BasicVectorHolder<int> >*,
>>> kaldi::TableWriter<kaldi::BasicVectorHolder<int> >*,
>>> kaldi::TableWriter<kaldi::CompactLatticeHolder>*,
>>> kaldi::TableWriter<kaldi::LatticeHolder>*, double*)
>>> gmm-latgen-faster(main+0xc3b) [0x58dad6]
>>> /lib64/libc.so.6(__libc_start_main+0xe6) [0x2ba2f7d9cc16]
>>> gmm-latgen-faster() [0x58cd11]
>>> /rmt/programs/gridengine_new/default/spool/ro/job_scripts/10778: line 6:
>>> 26822 Aborted (core dumped) ( gmm-latgen-faster --beam=13.0
>>> --lattice-beam=7.0 --acoustic-scale=0.1 --max-mem=20000000
>>> --max-active=5000 --word-symbol-table=data/lang/words.txt
>>> exp/tri2b_ali_si84/final.mdl exp/tri2b_denlats_si84/dengraph/HCLG.fst
>>> "ark:apply-cmvn --norm-vars=false
>>> --utt2spk=ark:data/train_si84/split20/24/utt2spk
>>> ark:exp/tri2b_ali_si84/24.cmvn scp:data/train_si84/split20/24/feats.scp
>>> ark:- | splice-feats ark:- ark:- | transform-feats
>>> exp/tri2b_ali_si84/final.mat ark:- ark:- |" "ark:|gzip -c
>>> >exp/tri2b_denlats_si84/lat.24.gz" ) 2>>
>>> /rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>> >>
>>> /rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>>
>>> =====================================
>>>
>>> I've started looking into the code in further detail but I guess
>>> debugging in this way will take a while since I have very little experience
>>> with kaldi. So, any ideas or suggestions will be greatly appreciated.
>>>
>>> Thank you,
>>> nassos
>>>
>>>
>>>
>>>
>>> PS: The decode_den.24.sh script:
>>>
>>> =====================================
>>>
>>> #!/bin/bash
>>> cd /rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3
>>> . path.sh
>>> echo Running on `hostname`
>>> >/rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>> echo Started at `date`
>>> >>/rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>> ( gmm-latgen-faster --beam=13.0 --lattice-beam=7.0 --acoustic-scale=0.1
>>> --max-mem=20000000 --max-active=5000
>>> --word-symbol-table=data/lang/words.txt exp/tri2b_ali_si84/final.mdl
>>> exp/tri2b_denlats_si84/dengraph/HCLG.fst "ark:apply-cmvn --norm-vars=false
>>> --utt2spk=ark:data/train_si84/split20/24/utt2spk
>>> ark:exp/tri2b_ali_si84/24.cmvn scp:data/train_si84/split20/24/feats.scp
>>> ark:- | splice-feats ark:- ark:- | transform-feats
>>> exp/tri2b_ali_si84/final.mat ark:- ark:- |" "ark:|gzip -c
>>> >exp/tri2b_denlats_si84/lat.24.gz" )
>>> 2>>/rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>> >>/rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>> ret=$?
>>> echo
>>> >>/rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>> echo Finished at `date`
>>> >>/rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>> exit $ret
>>> ## submitted with:
>>> # qsub -S /bin/bash -sync y -j y -o
>>> /rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/decode_den.24.log
>>> -l mem_free=700M
>>> /rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/q/
>>> decode_den.24.sh>>/rmt/work/audio_asr/kaldi/kaldi-trunk/egs/wsj/s3/exp/tri2b_denlats_si84/q/queue.log
>>> 2>&1
>>> =====================================
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> This SF.net email is sponsored by Windows:
>>>
>>> Build for Windows Store.
>>>
>>> http://p.sf.net/sfu/windows-dev2dev
>>> _______________________________________________
>>> Kaldi-users mailing list
>>> Kal...@li...
>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> This SF.net email is sponsored by Windows:
>>
>> Build for Windows Store.
>>
>> http://p.sf.net/sfu/windows-dev2dev
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>
>>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
>
> http://p.sf.net/sfu/windows-dev2dev_______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>
>
>
>
> ------------------------------------------------------------------------------
> This SF.net email is sponsored by Windows:
>
> Build for Windows Store.
>
> http://p.sf.net/sfu/windows-dev2dev
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>
>
|
1 message has been excluded from this view by a project administrator.
