Menu
▾
▴
kaldi-users — Kaldi Users
You can subscribe to this list here.
| 2011 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(2) |
Aug
(2) |
Sep
(1) |
Oct
(1) |
Nov
|
Dec
|
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2012 |
Jan
|
Feb
|
Mar
(8) |
Apr
(4) |
May
(2) |
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2013 |
Jan
|
Feb
(2) |
Mar
(2) |
Apr
(7) |
May
(31) |
Jun
(40) |
Jul
(65) |
Aug
(37) |
Sep
(12) |
Oct
(57) |
Nov
(15) |
Dec
(35) |
| 2014 |
Jan
(3) |
Feb
(30) |
Mar
(57) |
Apr
(26) |
May
(49) |
Jun
(26) |
Jul
(63) |
Aug
(33) |
Sep
(20) |
Oct
(153) |
Nov
(62) |
Dec
(20) |
| 2015 |
Jan
(6) |
Feb
(21) |
Mar
(42) |
Apr
(33) |
May
(76) |
Jun
(102) |
Jul
(39) |
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-09-20 20:37:55
|
Hi.
That really was a partial message. I pressed shift+enter occasionally
and then realized how to solve my problem.
But another one have emerged.
I’m trying to train a dnn on my own small base (1 speaker, about
10hrs splitted on 7 words long utterances less than 10s duration
each) using egs/swbd/s5b/local/run_dnn.sh with appropriate alterations
(no feature-transform, paths).
I run this
$cuda_cmd $dir/_pretrain_dbn.log \
steps/pretrain_dbn.sh --hid_dim 2048 --train_utts 15000 --cmvn_utts
1000 $t $dir || exit 1
<set proper paths>
and this
$cuda_cmd $dir/_train_nnet.log \
steps/train_nnet.sh --dbn $dbn --hid-layers 0 --learn-rate 0.008 \
$t $cv $lang $ali $ali_cv $dir || exit 1;
Pre-training is ok now, but MLP training falls.
In prerun.log there are a lot of messages like this
WARNING (nnet-train-xent-hardlab-frmshuff:main():nnet-train-xent-hardlab-frmshuf
f.cc:148) Alignment has wrong length, ali 258 vs. feats 334, utt 101-11
and finally
KALDI_ASSERT: at
nnet-train-xent-hardlab-frmshuff:CloseInternal:util/kaldi-table-inl.h:1546,
failed: holder_ == NULL
Stack trace is:
kaldi::KaldiGetStackTrace()
kaldi::KaldiAssertFailure_(char const*, char const*, int, char const*)
kaldi::RandomAccessTableReaderArchiveImplBase<kaldi::BasicVectorHolder<int>
>::CloseInternal()
.In _train_nnet.;log (last stage) :
# RUNNING THE NN-TRAINING SCHEDULER
steps/train_nnet_scheduler.sh --feature-transform
exp/tri3b2_pretrain-dbn73_dnn/tr_splice5-1_cmvn-g.nnet --learn-rate
0.008 --seed 777 exp/tri3b2_pretrain-dbn73_dnn/nnet_6.dbn_dnn.init
ark:copy-feats scp:exp/tri3b2_pretrain-dbn73_dnn/train.scp ark:- |
ark:copy-feats scp:exp/tri3b2_pretrain-dbn73_dnn/cv.scp ark:- |
ark:ali-to-pdf exp/tri3b2_ali/final.mdl "ark:gunzip -c
exp/tri3b2_ali/ali.*.gz exp/tri3b2_ali_cvseg/ali.*.gz |" ark:- |
exp/tri3b2_pretrain-dbn73_dnn
steps/train_nnet_scheduler.sh: line 78: 5525 Aborted
(core dumped) $train_tool --cross-validate=true
--bunchsize=$bunch_size --cachesize=$cache_size --verbose=$verbose
${feature_transform:+ --feature-transform=$feature_transform}
${use_gpu_id:+ --use-gpu-id=$use_gpu_id} $mlp_best "$feats_cv"
"$labels" 2> $dir/log/prerun.log
It’s not a list sort problem because I can train simple triphone
models on the same alignment and decode the cv set.
I’m using default feature settings, so I suppose it should be plain
mfcc with 5 frames contexts.
Could you tell where to look to make this work?
I run ubuntu 12.10 64-bits and my video card is GTX 580. with 1.5G RAM
Regards,
Valentin
|
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-09-20 00:43:37
|
Seems to be a partial message. On Thu, Sep 19, 2013 at 5:14 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi, > > I,m trying to train a dnn on my own small base (about 10hrs, 1 speaker) > > ------------------------------------------------------------------------------ > LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! > 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint > 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes > Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. > http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-09-19 21:14:19
|
Hi, I,m trying to train a dnn on my own small base (about 10hrs, 1 speaker) |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-09-04 14:17:40
|
That's an unusual situation in that it doesn't print out any error
messages. Try to run the command line (the line after the '#') on
your terminal after sourcing path.sh (". path.sh"), and see what
happens.
Dan
On Wed, Sep 4, 2013 at 9:06 AM, Mailing list used for User
Communication and Updates <kal...@li...> wrote:
> HI,
>
> In the wsj/s5 setup, following command failed.
>
> steps/decode_fmllr.sh --nj 10 --cmd "$decode_cmd" exp/tri4b/graph_tgpr
> data/test_dev93 exp/tri4b/decode_tgpr_dev93
>
> The content of a log file is given below. Please advice. What is the reason
> for status 135 ?
>
> Thanks.
>
>
> # gmm-latgen-faster --max-arcs=-1 --max-active=2000 --beam=10.0
> --lattice-beam=6.0 --acoustic-scale=0.083333 --allow-partial=true
> --word-symbol-table=exp/tri4b/graph_tgpr/words.txt exp/tri4b/final.alimdl
> exp/tri4b/graph_tgpr/HCLG.fst "ark,s,cs:apply-cmvn --norm-vars=false
> --utt2spk=ark:data/test_dev93/split10/1/utt2spk
> scp:data/test_dev93/split10/1/cmvn.scp
> scp:data/test_dev93/split10/1/feats.scp ark:- | splice-feats
> --left-context=3 --right-context=3 ark:- ark:- | transform-feats
> exp/tri4b/final.mat ark:- ark:- |" "ark:|gzip -c >
> exp/tri4b/decode_tgpr_dev93.si/lat.1.gz"
> gmm-latgen-faster --max-arcs=-1 --max-active=2000 --beam=10.0
> --lattice-beam=6.0 --acoustic-scale=0.083333 --allow-partial=true
> --word-symbol-table=exp/tri4b/graph_tgpr/words.txt exp/tri4b/final.alimdl
> exp/tri4b/graph_tgpr/HCLG.fst 'ark,s,cs:apply-cmvn --norm-vars=false
> --utt2spk=ark:data/test_dev93/split10/1/utt2spk
> scp:data/test_dev93/split10/1/cmvn.scp
> scp:data/test_dev93/split10/1/feats.scp ark:- | splice-feats
> --left-context=3 --right-context=3 ark:- ark:- | transform-feats
> exp/tri4b/final.mat ark:- ark:- |' 'ark:|gzip -c >
> exp/tri4b/decode_tgpr_dev93.si/lat.1.gz'
> splice-feats --left-context=3 --right-context=3 ark:- ark:-
> apply-cmvn --norm-vars=false --utt2spk=ark:data/test_dev93/split10/1/utt2spk
> scp:data/test_dev93/split10/1/cmvn.scp
> scp:data/test_dev93/split10/1/feats.scp ark:-
> transform-feats exp/tri4b/final.mat ark:- ark:-
> # Finished at Wed Sep 4 20:15:34 SGT 2013 with status 135
>
>
> ------------------------------------------------------------------------------
> Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
> Discover the easy way to master current and previous Microsoft technologies
> and advance your career. Get an incredible 1,500+ hours of step-by-step
> tutorial videos with LearnDevNow. Subscribe today and save!
> http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk
> _______________________________________________
> Kaldi-users mailing list
> Kal...@li...
> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>
|
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-09-04 13:06:53
|
HI, In the wsj/s5 setup, following command failed. steps/decode_fmllr.sh --nj 10 --cmd "$decode_cmd" exp/tri4b/graph_tgpr data/test_dev93 exp/tri4b/decode_tgpr_dev93 The content of a log file is given below. Please advice. What is the reason for status 135 ? Thanks. # gmm-latgen-faster --max-arcs=-1 --max-active=2000 --beam=10.0 --lattice-beam=6.0 --acoustic-scale=0.083333 --allow-partial=true --word-symbol-table=exp/tri4b/graph_tgpr/words.txt exp/tri4b/final.alimdl exp/tri4b/graph_tgpr/HCLG.fst "ark,s,cs:apply-cmvn --norm-vars=false --utt2spk=ark:data/test_dev93/split10/1/utt2spk scp:data/test_dev93/split10/1/cmvn.scp scp:data/test_dev93/split10/1/feats.scp ark:- | splice-feats --left-context=3 --right-context=3 ark:- ark:- | transform-feats exp/tri4b/final.mat ark:- ark:- |" "ark:|gzip -c > exp/tri4b/ decode_tgpr_dev93.si/lat.1.gz" gmm-latgen-faster --max-arcs=-1 --max-active=2000 --beam=10.0 --lattice-beam=6.0 --acoustic-scale=0.083333 --allow-partial=true --word-symbol-table=exp/tri4b/graph_tgpr/words.txt exp/tri4b/final.alimdl exp/tri4b/graph_tgpr/HCLG.fst 'ark,s,cs:apply-cmvn --norm-vars=false --utt2spk=ark:data/test_dev93/split10/1/utt2spk scp:data/test_dev93/split10/1/cmvn.scp scp:data/test_dev93/split10/1/feats.scp ark:- | splice-feats --left-context=3 --right-context=3 ark:- ark:- | transform-feats exp/tri4b/final.mat ark:- ark:- |' 'ark:|gzip -c > exp/tri4b/ decode_tgpr_dev93.si/lat.1.gz' splice-feats --left-context=3 --right-context=3 ark:- ark:- apply-cmvn --norm-vars=false --utt2spk=ark:data/test_dev93/split10/1/utt2spk scp:data/test_dev93/split10/1/cmvn.scp scp:data/test_dev93/split10/1/feats.scp ark:- transform-feats exp/tri4b/final.mat ark:- ark:- # Finished at Wed Sep 4 20:15:34 SGT 2013 with status 135 |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-28 03:04:27
|
That's good to know. I didn't find any in any literature or scripts neither. Thanks, Dan. Eric On Tue, Aug 27, 2013 at 3:57 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > No, this is not possible. In fact, apart from standard fMLLR on the > features you feed to the DNN, I am not aware of any methods in the > literature that are particularly effective for this. > Dan > > > On Tue, Aug 27, 2013 at 5:01 PM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: > > Hi all, > > > > I have found this in any of the scripts yet... Is there any support for > > supervised acoustic adaptation of the DNN model? Is it possible to adapt > the > > DNN models with a few minutes of accurate text and audio, much like MLLR > for > > the GMM model, or even MAP? > > > > > > Thanks, > > Eric > > > > "Confidential & Privileged: This email message is for the sole use of the > > intended recipient(s) and may contain confidential and privileged > > information. Any unauthorized review, use, disclosure or distribution is > > prohibited. if you are not the intended recipient, please contact the > sender > > by reply email and destroy all copies of the original message.” > > > ------------------------------------------------------------------------------ > > Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! > > Discover the easy way to master current and previous Microsoft > technologies > > and advance your career. Get an incredible 1,500+ hours of step-by-step > > tutorial videos with LearnDevNow. Subscribe today and save! > > > http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk > > _______________________________________________ > > Kaldi-users mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > ------------------------------------------------------------------------------ > Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! > Discover the easy way to master current and previous Microsoft technologies > and advance your career. Get an incredible 1,500+ hours of step-by-step > tutorial videos with LearnDevNow. Subscribe today and save! > http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > -- "Confidential & Privileged: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. if you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.” |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-27 19:57:15
|
No, this is not possible. In fact, apart from standard fMLLR on the features you feed to the DNN, I am not aware of any methods in the literature that are particularly effective for this. Dan On Tue, Aug 27, 2013 at 5:01 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi all, > > I have found this in any of the scripts yet... Is there any support for > supervised acoustic adaptation of the DNN model? Is it possible to adapt the > DNN models with a few minutes of accurate text and audio, much like MLLR for > the GMM model, or even MAP? > > > Thanks, > Eric > > "Confidential & Privileged: This email message is for the sole use of the > intended recipient(s) and may contain confidential and privileged > information. Any unauthorized review, use, disclosure or distribution is > prohibited. if you are not the intended recipient, please contact the sender > by reply email and destroy all copies of the original message.” > ------------------------------------------------------------------------------ > Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! > Discover the easy way to master current and previous Microsoft technologies > and advance your career. Get an incredible 1,500+ hours of step-by-step > tutorial videos with LearnDevNow. Subscribe today and save! > http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-27 16:09:35
|
Hi all, I have found this in any of the scripts yet... Is there any support for supervised acoustic adaptation of the DNN model? Is it possible to adapt the DNN models with a few minutes of accurate text and audio, much like MLLR for the GMM model, or even MAP? Thanks, Eric -- "Confidential & Privileged: This email message is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. if you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message." |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-27 14:21:58
|
Right... Well for the blank it's because I use the output of a RNN trained with CTC for decoding, so it's a kind of workaround. Thanks so much for spotting this #0 problem, that solved the issue. Cheers Téo ________________________________________ From: Mailing list used for User Communication and Updates [kal...@li...] Sent: Tuesday, August 27, 2013 3:48 PM To: kal...@li... Subject: Re: [Kaldi-users] Problem with align-mapped Both of those lexicons look kind of weird. Firstly you have these <BLANK> things everywhere, but apart from that, the path from that word back to the start state in L.png requires the disambig symbol #0 on the output side, which does not seem right. The symbol should always be accepted optionally but should never be made mandatory. Dan On Tue, Aug 27, 2013 at 3:20 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi again, > > So, what I've always done is use a lexicon which contains disambiguation symbols, and use the --read-disambig-syms option and had no problem. But not anymore with my new FST which has a somewhat different structure.. > A smaller experiment showed that they were probably the reason why it didn't work. > Attached are L.png: lexicon FST w/ disambig, L2.png: without > The graph is empty when disambig symbols are there, but everything's fine when I remove them. I should check why this particular structure make the whole thing crash.. > > Cheers, > Téo > > > > > ________________________________________ > From: Mailing list used for User Communication and Updates [kal...@li...] > Sent: Tuesday, August 27, 2013 11:21 AM > To: kal...@li... > Subject: Re: [Kaldi-users] Problem with align-mapped > > I just committed a change to compile-train-graphs so that it notices > when graphs are empty and does not output them, and prints warnings. > The underlying problem here is probably something like a lexicon > mismatch or a lexicon that lacks some pronunciations. > > Dan > > > On Tue, Aug 27, 2013 at 9:57 AM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: >> Hi! thanks! >> I actually did something similar just after sending the email yesterday : >> >>> compile-train-graphs --transition-scale=1.0 --self-loop-scale=0.1 --read-disambig-syms=disambig_chars.list tree best.mdl L.fst ark:test.tra ark,t:- >> p03-096_6 >> >> LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 >> >> -> the graph is indeed empty... I don't get why right know, but I guess the next step would be to try on a smaller yet realistic problem. I let you know when I find the origin of the problem what it was and how I solved it. >> >> Cheers, >> Téo >> >> >> >> ________________________________________ >> From: Mailing list used for User Communication and Updates [kal...@li...] >> Sent: Monday, August 26, 2013 7:26 PM >> To: kal...@li... >> Subject: Re: [Kaldi-users] Problem with align-mapped >> >> Check that compile-train-graphs is checking for nonempty graphs >> (fst->NumStates() != 0, or similar. >> Dan >> >> >> On Mon, Aug 26, 2013 at 3:24 PM, Mailing list used for User >> Communication and Updates <kal...@li...> wrote: >>> Hi! >>> >>> I have a problem with align-mapped. It says >>> >>> WARNING (align-mapped:main():align-mapped.cc:124) Empty decoding graph for p03-096_6 >>> >>> But if I use exactly the same arguments for compile-train-graphs, I get no problem : >>> >>> LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 >>> >>> Maybe it is a check that is done in align-mapped but not in compile-train-graphs. >>> >>> Anyway, any idea about what happens? >>> >>> I use the option '--read-disambig-syms' so the disambig symbols in my lex.fst shouldn't be an issue.. Any all words in the transcript are in the vocabulary.. >>> >>> Cheers, >>> >>> Téo >>> ------------------------------------------------------------------------------ >>> Introducing Performance Central, a new site from SourceForge and >>> AppDynamics. Performance Central is your source for news, insights, >>> analysis and resources for efficient Application Performance Management. >>> Visit us today! >>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Kaldi-users mailing list >>> Kal...@li... >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > ------------------------------------------------------------------------------ Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk _______________________________________________ Kaldi-users mailing list Kal...@li... https://lists.sourceforge.net/lists/listinfo/kaldi-users |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-27 13:48:41
|
Both of those lexicons look kind of weird. Firstly you have these <BLANK> things everywhere, but apart from that, the path from that word back to the start state in L.png requires the disambig symbol #0 on the output side, which does not seem right. The symbol should always be accepted optionally but should never be made mandatory. Dan On Tue, Aug 27, 2013 at 3:20 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi again, > > So, what I've always done is use a lexicon which contains disambiguation symbols, and use the --read-disambig-syms option and had no problem. But not anymore with my new FST which has a somewhat different structure.. > A smaller experiment showed that they were probably the reason why it didn't work. > Attached are L.png: lexicon FST w/ disambig, L2.png: without > The graph is empty when disambig symbols are there, but everything's fine when I remove them. I should check why this particular structure make the whole thing crash.. > > Cheers, > Téo > > > > > ________________________________________ > From: Mailing list used for User Communication and Updates [kal...@li...] > Sent: Tuesday, August 27, 2013 11:21 AM > To: kal...@li... > Subject: Re: [Kaldi-users] Problem with align-mapped > > I just committed a change to compile-train-graphs so that it notices > when graphs are empty and does not output them, and prints warnings. > The underlying problem here is probably something like a lexicon > mismatch or a lexicon that lacks some pronunciations. > > Dan > > > On Tue, Aug 27, 2013 at 9:57 AM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: >> Hi! thanks! >> I actually did something similar just after sending the email yesterday : >> >>> compile-train-graphs --transition-scale=1.0 --self-loop-scale=0.1 --read-disambig-syms=disambig_chars.list tree best.mdl L.fst ark:test.tra ark,t:- >> p03-096_6 >> >> LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 >> >> -> the graph is indeed empty... I don't get why right know, but I guess the next step would be to try on a smaller yet realistic problem. I let you know when I find the origin of the problem what it was and how I solved it. >> >> Cheers, >> Téo >> >> >> >> ________________________________________ >> From: Mailing list used for User Communication and Updates [kal...@li...] >> Sent: Monday, August 26, 2013 7:26 PM >> To: kal...@li... >> Subject: Re: [Kaldi-users] Problem with align-mapped >> >> Check that compile-train-graphs is checking for nonempty graphs >> (fst->NumStates() != 0, or similar. >> Dan >> >> >> On Mon, Aug 26, 2013 at 3:24 PM, Mailing list used for User >> Communication and Updates <kal...@li...> wrote: >>> Hi! >>> >>> I have a problem with align-mapped. It says >>> >>> WARNING (align-mapped:main():align-mapped.cc:124) Empty decoding graph for p03-096_6 >>> >>> But if I use exactly the same arguments for compile-train-graphs, I get no problem : >>> >>> LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 >>> >>> Maybe it is a check that is done in align-mapped but not in compile-train-graphs. >>> >>> Anyway, any idea about what happens? >>> >>> I use the option '--read-disambig-syms' so the disambig symbols in my lex.fst shouldn't be an issue.. Any all words in the transcript are in the vocabulary.. >>> >>> Cheers, >>> >>> Téo >>> ------------------------------------------------------------------------------ >>> Introducing Performance Central, a new site from SourceForge and >>> AppDynamics. Performance Central is your source for news, insights, >>> analysis and resources for efficient Application Performance Management. >>> Visit us today! >>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Kaldi-users mailing list >>> Kal...@li... >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-27 13:21:09
|
Hi again, So, what I've always done is use a lexicon which contains disambiguation symbols, and use the --read-disambig-syms option and had no problem. But not anymore with my new FST which has a somewhat different structure.. A smaller experiment showed that they were probably the reason why it didn't work. Attached are L.png: lexicon FST w/ disambig, L2.png: without The graph is empty when disambig symbols are there, but everything's fine when I remove them. I should check why this particular structure make the whole thing crash.. Cheers, Téo ________________________________________ From: Mailing list used for User Communication and Updates [kal...@li...] Sent: Tuesday, August 27, 2013 11:21 AM To: kal...@li... Subject: Re: [Kaldi-users] Problem with align-mapped I just committed a change to compile-train-graphs so that it notices when graphs are empty and does not output them, and prints warnings. The underlying problem here is probably something like a lexicon mismatch or a lexicon that lacks some pronunciations. Dan On Tue, Aug 27, 2013 at 9:57 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi! thanks! > I actually did something similar just after sending the email yesterday : > >> compile-train-graphs --transition-scale=1.0 --self-loop-scale=0.1 --read-disambig-syms=disambig_chars.list tree best.mdl L.fst ark:test.tra ark,t:- > p03-096_6 > > LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 > > -> the graph is indeed empty... I don't get why right know, but I guess the next step would be to try on a smaller yet realistic problem. I let you know when I find the origin of the problem what it was and how I solved it. > > Cheers, > Téo > > > > ________________________________________ > From: Mailing list used for User Communication and Updates [kal...@li...] > Sent: Monday, August 26, 2013 7:26 PM > To: kal...@li... > Subject: Re: [Kaldi-users] Problem with align-mapped > > Check that compile-train-graphs is checking for nonempty graphs > (fst->NumStates() != 0, or similar. > Dan > > > On Mon, Aug 26, 2013 at 3:24 PM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: >> Hi! >> >> I have a problem with align-mapped. It says >> >> WARNING (align-mapped:main():align-mapped.cc:124) Empty decoding graph for p03-096_6 >> >> But if I use exactly the same arguments for compile-train-graphs, I get no problem : >> >> LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 >> >> Maybe it is a check that is done in align-mapped but not in compile-train-graphs. >> >> Anyway, any idea about what happens? >> >> I use the option '--read-disambig-syms' so the disambig symbols in my lex.fst shouldn't be an issue.. Any all words in the transcript are in the vocabulary.. >> >> Cheers, >> >> Téo >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users ------------------------------------------------------------------------------ Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk _______________________________________________ Kaldi-users mailing list Kal...@li... https://lists.sourceforge.net/lists/listinfo/kaldi-users |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-27 09:21:48
|
I just committed a change to compile-train-graphs so that it notices when graphs are empty and does not output them, and prints warnings. The underlying problem here is probably something like a lexicon mismatch or a lexicon that lacks some pronunciations. Dan On Tue, Aug 27, 2013 at 9:57 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi! thanks! > I actually did something similar just after sending the email yesterday : > >> compile-train-graphs --transition-scale=1.0 --self-loop-scale=0.1 --read-disambig-syms=disambig_chars.list tree best.mdl L.fst ark:test.tra ark,t:- > p03-096_6 > > LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 > > -> the graph is indeed empty... I don't get why right know, but I guess the next step would be to try on a smaller yet realistic problem. I let you know when I find the origin of the problem what it was and how I solved it. > > Cheers, > Téo > > > > ________________________________________ > From: Mailing list used for User Communication and Updates [kal...@li...] > Sent: Monday, August 26, 2013 7:26 PM > To: kal...@li... > Subject: Re: [Kaldi-users] Problem with align-mapped > > Check that compile-train-graphs is checking for nonempty graphs > (fst->NumStates() != 0, or similar. > Dan > > > On Mon, Aug 26, 2013 at 3:24 PM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: >> Hi! >> >> I have a problem with align-mapped. It says >> >> WARNING (align-mapped:main():align-mapped.cc:124) Empty decoding graph for p03-096_6 >> >> But if I use exactly the same arguments for compile-train-graphs, I get no problem : >> >> LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 >> >> Maybe it is a check that is done in align-mapped but not in compile-train-graphs. >> >> Anyway, any idea about what happens? >> >> I use the option '--read-disambig-syms' so the disambig symbols in my lex.fst shouldn't be an issue.. Any all words in the transcript are in the vocabulary.. >> >> Cheers, >> >> Téo >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-27 07:57:42
|
Hi! thanks! I actually did something similar just after sending the email yesterday : > compile-train-graphs --transition-scale=1.0 --self-loop-scale=0.1 --read-disambig-syms=disambig_chars.list tree best.mdl L.fst ark:test.tra ark,t:- p03-096_6 LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 -> the graph is indeed empty... I don't get why right know, but I guess the next step would be to try on a smaller yet realistic problem. I let you know when I find the origin of the problem what it was and how I solved it. Cheers, Téo ________________________________________ From: Mailing list used for User Communication and Updates [kal...@li...] Sent: Monday, August 26, 2013 7:26 PM To: kal...@li... Subject: Re: [Kaldi-users] Problem with align-mapped Check that compile-train-graphs is checking for nonempty graphs (fst->NumStates() != 0, or similar. Dan On Mon, Aug 26, 2013 at 3:24 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi! > > I have a problem with align-mapped. It says > > WARNING (align-mapped:main():align-mapped.cc:124) Empty decoding graph for p03-096_6 > > But if I use exactly the same arguments for compile-train-graphs, I get no problem : > > LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 > > Maybe it is a check that is done in align-mapped but not in compile-train-graphs. > > Anyway, any idea about what happens? > > I use the option '--read-disambig-syms' so the disambig symbols in my lex.fst shouldn't be an issue.. Any all words in the transcript are in the vocabulary.. > > Cheers, > > Téo > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users ------------------------------------------------------------------------------ Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk _______________________________________________ Kaldi-users mailing list Kal...@li... https://lists.sourceforge.net/lists/listinfo/kaldi-users |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-26 17:26:32
|
Check that compile-train-graphs is checking for nonempty graphs (fst->NumStates() != 0, or similar. Dan On Mon, Aug 26, 2013 at 3:24 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi! > > I have a problem with align-mapped. It says > > WARNING (align-mapped:main():align-mapped.cc:124) Empty decoding graph for p03-096_6 > > But if I use exactly the same arguments for compile-train-graphs, I get no problem : > > LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 > > Maybe it is a check that is done in align-mapped but not in compile-train-graphs. > > Anyway, any idea about what happens? > > I use the option '--read-disambig-syms' so the disambig symbols in my lex.fst shouldn't be an issue.. Any all words in the transcript are in the vocabulary.. > > Cheers, > > Téo > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-26 13:37:57
|
Hi! I have a problem with align-mapped. It says WARNING (align-mapped:main():align-mapped.cc:124) Empty decoding graph for p03-096_6 But if I use exactly the same arguments for compile-train-graphs, I get no problem : LOG (compile-train-graphs:main():compile-train-graphs.cc:138) compile-train-graphs: succeeded for 1 graphs, failed for 0 Maybe it is a check that is done in align-mapped but not in compile-train-graphs. Anyway, any idea about what happens? I use the option '--read-disambig-syms' so the disambig symbols in my lex.fst shouldn't be an issue.. Any all words in the transcript are in the vocabulary.. Cheers, Téo |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-23 04:39:28
|
I think it is not the case, because h_rt and s_rt both are set to infinity. Thanks, Lahiru On Thu, Aug 22, 2013 at 10:45 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > Hi, did you specify you SGE queue and queue parameters (especially the > expected time to run) correctly? Some SGE grids have really short time the > task is allowed to run (and is killed after exhausting this limit)... In > that case you have to explicitly request more "walltime" when submitting > the tasks to SGE. > y. > > > On Thu, Aug 22, 2013 at 10:22 AM, Mailing list used for User Communication > and Updates <kal...@li...> wrote: > >> The training fails while "Generating training examples on disk" giving >> the following message: >> >> *queue.pl: Error, unfinished job no longer exists, log is in >> exp/nnet5c1/log/get_egs.*.log >> Possible reasons: a) Exceeded time limit? -> Use more jobs! b) >> Shutdown/Frozen machine? -> Run again!* >> >> I checked all the get_egs.*.log files and only the jobs submitted to one >> server are unfinished. >> And I can access that server using unix commands So it is not frozen. >> >> Is there a time limit in this stage of training? >> >> Any ideas? >> >> Anyway I am going to run again. >> >> Thanks, >> Lahiru >> >> >> On Wed, Aug 21, 2013 at 5:08 PM, Lahiru Samarakoon <lah...@gm...>wrote: >> >>> Thanks a lot. >>> >>> >>> On Wed, Aug 21, 2013 at 5:07 PM, Mailing list used for User >>> Communication and Updates <kal...@li...> wrote: >>> >>>> No, you don't have to submit the script to sge, the script doesn't do >>>> much apart from submit jobs to SGE itself, and it's more convenient to >>>> just run it from a command line. (but use nohup or run it inside >>>> screen to stop it getting interrupted if you log out). >>>> >>>> Dan >>>> >>>> >>>> On Wed, Aug 21, 2013 at 11:04 AM, Mailing list used for User >>>> Communication and Updates <kal...@li...> wrote: >>>> > When running the dnn training on cpu, is it necessary to submit the >>>> script >>>> > to SGE explicitly? >>>> > Or just running the script will take care of the job because the >>>> script uses >>>> > the queue.pl? >>>> > >>>> > Thanks, >>>> > Lahiru >>>> > >>>> > >>>> > >>>> > On Tue, Aug 20, 2013 at 3:28 PM, Mailing list used for User >>>> Communication >>>> > and Updates <kal...@li...> wrote: >>>> >> >>>> >> num_jobs_nnet should be the same as the #machine (i.e. 4), but you >>>> may >>>> >> want to decrease the learning rate a bit (e.g. by a factor of 2) if >>>> >> you reducde the #machines from 16 to 4. >>>> >> Dan >>>> >> >>>> >> >>>> >> On Tue, Aug 20, 2013 at 9:21 AM, Mailing list used for User >>>> >> Communication and Updates <kal...@li...> wrote: >>>> >> > Hi All, >>>> >> > >>>> >> > I am planning to run the wsj/s5 set up for DNN training on 4 >>>> machines >>>> >> > and >>>> >> > each has 16 cores. Could anyone give me some pointers about how to >>>> >> > change >>>> >> > important parameters like num_jobs_nnet in the script for my setup? >>>> >> > >>>> >> > Thank you, >>>> >> > >>>> >> > Best Regards, >>>> >> > Lahiru >>>> >> > >>>> >> > >>>> >> > >>>> ------------------------------------------------------------------------------ >>>> >> > Introducing Performance Central, a new site from SourceForge and >>>> >> > AppDynamics. Performance Central is your source for news, insights, >>>> >> > analysis and resources for efficient Application Performance >>>> Management. >>>> >> > Visit us today! >>>> >> > >>>> >> > >>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>>> >> > _______________________________________________ >>>> >> > Kaldi-users mailing list >>>> >> > Kal...@li... >>>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >>>> >> > >>>> >> >>>> >> >>>> >> >>>> ------------------------------------------------------------------------------ >>>> >> Introducing Performance Central, a new site from SourceForge and >>>> >> AppDynamics. Performance Central is your source for news, insights, >>>> >> analysis and resources for efficient Application Performance >>>> Management. >>>> >> Visit us today! >>>> >> >>>> >> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>>> >> _______________________________________________ >>>> >> Kaldi-users mailing list >>>> >> Kal...@li... >>>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >>>> > >>>> > >>>> > >>>> > >>>> ------------------------------------------------------------------------------ >>>> > Introducing Performance Central, a new site from SourceForge and >>>> > AppDynamics. Performance Central is your source for news, insights, >>>> > analysis and resources for efficient Application Performance >>>> Management. >>>> > Visit us today! >>>> > >>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>>> > _______________________________________________ >>>> > Kaldi-users mailing list >>>> > Kal...@li... >>>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >>>> > >>>> >>>> >>>> ------------------------------------------------------------------------------ >>>> Introducing Performance Central, a new site from SourceForge and >>>> AppDynamics. Performance Central is your source for news, insights, >>>> analysis and resources for efficient Application Performance Management. >>>> Visit us today! >>>> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> Kaldi-users mailing list >>>> Kal...@li... >>>> https://lists.sourceforge.net/lists/listinfo/kaldi-users >>>> >>> >>> >> >> >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-22 14:45:56
|
Hi, did you specify you SGE queue and queue parameters (especially the expected time to run) correctly? Some SGE grids have really short time the task is allowed to run (and is killed after exhausting this limit)... In that case you have to explicitly request more "walltime" when submitting the tasks to SGE. y. On Thu, Aug 22, 2013 at 10:22 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > The training fails while "Generating training examples on disk" giving > the following message: > > *queue.pl: Error, unfinished job no longer exists, log is in > exp/nnet5c1/log/get_egs.*.log > Possible reasons: a) Exceeded time limit? -> Use more jobs! b) > Shutdown/Frozen machine? -> Run again!* > > I checked all the get_egs.*.log files and only the jobs submitted to one > server are unfinished. > And I can access that server using unix commands So it is not frozen. > > Is there a time limit in this stage of training? > > Any ideas? > > Anyway I am going to run again. > > Thanks, > Lahiru > > > On Wed, Aug 21, 2013 at 5:08 PM, Lahiru Samarakoon <lah...@gm...>wrote: > >> Thanks a lot. >> >> >> On Wed, Aug 21, 2013 at 5:07 PM, Mailing list used for User Communication >> and Updates <kal...@li...> wrote: >> >>> No, you don't have to submit the script to sge, the script doesn't do >>> much apart from submit jobs to SGE itself, and it's more convenient to >>> just run it from a command line. (but use nohup or run it inside >>> screen to stop it getting interrupted if you log out). >>> >>> Dan >>> >>> >>> On Wed, Aug 21, 2013 at 11:04 AM, Mailing list used for User >>> Communication and Updates <kal...@li...> wrote: >>> > When running the dnn training on cpu, is it necessary to submit the >>> script >>> > to SGE explicitly? >>> > Or just running the script will take care of the job because the >>> script uses >>> > the queue.pl? >>> > >>> > Thanks, >>> > Lahiru >>> > >>> > >>> > >>> > On Tue, Aug 20, 2013 at 3:28 PM, Mailing list used for User >>> Communication >>> > and Updates <kal...@li...> wrote: >>> >> >>> >> num_jobs_nnet should be the same as the #machine (i.e. 4), but you may >>> >> want to decrease the learning rate a bit (e.g. by a factor of 2) if >>> >> you reducde the #machines from 16 to 4. >>> >> Dan >>> >> >>> >> >>> >> On Tue, Aug 20, 2013 at 9:21 AM, Mailing list used for User >>> >> Communication and Updates <kal...@li...> wrote: >>> >> > Hi All, >>> >> > >>> >> > I am planning to run the wsj/s5 set up for DNN training on 4 >>> machines >>> >> > and >>> >> > each has 16 cores. Could anyone give me some pointers about how to >>> >> > change >>> >> > important parameters like num_jobs_nnet in the script for my setup? >>> >> > >>> >> > Thank you, >>> >> > >>> >> > Best Regards, >>> >> > Lahiru >>> >> > >>> >> > >>> >> > >>> ------------------------------------------------------------------------------ >>> >> > Introducing Performance Central, a new site from SourceForge and >>> >> > AppDynamics. Performance Central is your source for news, insights, >>> >> > analysis and resources for efficient Application Performance >>> Management. >>> >> > Visit us today! >>> >> > >>> >> > >>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> >> > _______________________________________________ >>> >> > Kaldi-users mailing list >>> >> > Kal...@li... >>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >>> >> > >>> >> >>> >> >>> >> >>> ------------------------------------------------------------------------------ >>> >> Introducing Performance Central, a new site from SourceForge and >>> >> AppDynamics. Performance Central is your source for news, insights, >>> >> analysis and resources for efficient Application Performance >>> Management. >>> >> Visit us today! >>> >> >>> >> >>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> >> _______________________________________________ >>> >> Kaldi-users mailing list >>> >> Kal...@li... >>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >>> > >>> > >>> > >>> > >>> ------------------------------------------------------------------------------ >>> > Introducing Performance Central, a new site from SourceForge and >>> > AppDynamics. Performance Central is your source for news, insights, >>> > analysis and resources for efficient Application Performance >>> Management. >>> > Visit us today! >>> > >>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> > _______________________________________________ >>> > Kaldi-users mailing list >>> > Kal...@li... >>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >>> > >>> >>> >>> ------------------------------------------------------------------------------ >>> Introducing Performance Central, a new site from SourceForge and >>> AppDynamics. Performance Central is your source for news, insights, >>> analysis and resources for efficient Application Performance Management. >>> Visit us today! >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Kaldi-users mailing list >>> Kal...@li... >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users >>> >> >> > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-22 14:32:24
|
Likely that machine is or was having NFS problems, e.g. at that time it could not see the directory concerned, or maybe there were other problems on that machine. Dan On Thu, Aug 22, 2013 at 4:22 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > The training fails while "Generating training examples on disk" giving the > following message: > > queue.pl: Error, unfinished job no longer exists, log is in > exp/nnet5c1/log/get_egs.*.log > Possible reasons: a) Exceeded time limit? -> Use more jobs! b) > Shutdown/Frozen machine? -> Run again! > > I checked all the get_egs.*.log files and only the jobs submitted to one > server are unfinished. > And I can access that server using unix commands So it is not frozen. > > Is there a time limit in this stage of training? > > Any ideas? > > Anyway I am going to run again. > > Thanks, > Lahiru > > > On Wed, Aug 21, 2013 at 5:08 PM, Lahiru Samarakoon <lah...@gm...> > wrote: >> >> Thanks a lot. >> >> >> On Wed, Aug 21, 2013 at 5:07 PM, Mailing list used for User Communication >> and Updates <kal...@li...> wrote: >>> >>> No, you don't have to submit the script to sge, the script doesn't do >>> much apart from submit jobs to SGE itself, and it's more convenient to >>> just run it from a command line. (but use nohup or run it inside >>> screen to stop it getting interrupted if you log out). >>> >>> Dan >>> >>> >>> On Wed, Aug 21, 2013 at 11:04 AM, Mailing list used for User >>> Communication and Updates <kal...@li...> wrote: >>> > When running the dnn training on cpu, is it necessary to submit the >>> > script >>> > to SGE explicitly? >>> > Or just running the script will take care of the job because the script >>> > uses >>> > the queue.pl? >>> > >>> > Thanks, >>> > Lahiru >>> > >>> > >>> > >>> > On Tue, Aug 20, 2013 at 3:28 PM, Mailing list used for User >>> > Communication >>> > and Updates <kal...@li...> wrote: >>> >> >>> >> num_jobs_nnet should be the same as the #machine (i.e. 4), but you may >>> >> want to decrease the learning rate a bit (e.g. by a factor of 2) if >>> >> you reducde the #machines from 16 to 4. >>> >> Dan >>> >> >>> >> >>> >> On Tue, Aug 20, 2013 at 9:21 AM, Mailing list used for User >>> >> Communication and Updates <kal...@li...> wrote: >>> >> > Hi All, >>> >> > >>> >> > I am planning to run the wsj/s5 set up for DNN training on 4 >>> >> > machines >>> >> > and >>> >> > each has 16 cores. Could anyone give me some pointers about how to >>> >> > change >>> >> > important parameters like num_jobs_nnet in the script for my setup? >>> >> > >>> >> > Thank you, >>> >> > >>> >> > Best Regards, >>> >> > Lahiru >>> >> > >>> >> > >>> >> > >>> >> > ------------------------------------------------------------------------------ >>> >> > Introducing Performance Central, a new site from SourceForge and >>> >> > AppDynamics. Performance Central is your source for news, insights, >>> >> > analysis and resources for efficient Application Performance >>> >> > Management. >>> >> > Visit us today! >>> >> > >>> >> > >>> >> > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> >> > _______________________________________________ >>> >> > Kaldi-users mailing list >>> >> > Kal...@li... >>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >>> >> > >>> >> >>> >> >>> >> >>> >> ------------------------------------------------------------------------------ >>> >> Introducing Performance Central, a new site from SourceForge and >>> >> AppDynamics. Performance Central is your source for news, insights, >>> >> analysis and resources for efficient Application Performance >>> >> Management. >>> >> Visit us today! >>> >> >>> >> >>> >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> >> _______________________________________________ >>> >> Kaldi-users mailing list >>> >> Kal...@li... >>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users >>> > >>> > >>> > >>> > >>> > ------------------------------------------------------------------------------ >>> > Introducing Performance Central, a new site from SourceForge and >>> > AppDynamics. Performance Central is your source for news, insights, >>> > analysis and resources for efficient Application Performance >>> > Management. >>> > Visit us today! >>> > >>> > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> > _______________________________________________ >>> > Kaldi-users mailing list >>> > Kal...@li... >>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >>> > >>> >>> >>> ------------------------------------------------------------------------------ >>> Introducing Performance Central, a new site from SourceForge and >>> AppDynamics. Performance Central is your source for news, insights, >>> analysis and resources for efficient Application Performance Management. >>> Visit us today! >>> >>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Kaldi-users mailing list >>> Kal...@li... >>> https://lists.sourceforge.net/lists/listinfo/kaldi-users >> >> > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-22 14:22:38
|
The training fails while "Generating training examples on disk" giving the
following message:
*queue.pl: Error, unfinished job no longer exists, log is in
exp/nnet5c1/log/get_egs.*.log
Possible reasons: a) Exceeded time limit? -> Use more jobs! b)
Shutdown/Frozen machine? -> Run again!*
I checked all the get_egs.*.log files and only the jobs submitted to one
server are unfinished.
And I can access that server using unix commands So it is not frozen.
Is there a time limit in this stage of training?
Any ideas?
Anyway I am going to run again.
Thanks,
Lahiru
On Wed, Aug 21, 2013 at 5:08 PM, Lahiru Samarakoon <lah...@gm...>wrote:
> Thanks a lot.
>
>
> On Wed, Aug 21, 2013 at 5:07 PM, Mailing list used for User Communication
> and Updates <kal...@li...> wrote:
>
>> No, you don't have to submit the script to sge, the script doesn't do
>> much apart from submit jobs to SGE itself, and it's more convenient to
>> just run it from a command line. (but use nohup or run it inside
>> screen to stop it getting interrupted if you log out).
>>
>> Dan
>>
>>
>> On Wed, Aug 21, 2013 at 11:04 AM, Mailing list used for User
>> Communication and Updates <kal...@li...> wrote:
>> > When running the dnn training on cpu, is it necessary to submit the
>> script
>> > to SGE explicitly?
>> > Or just running the script will take care of the job because the script
>> uses
>> > the queue.pl?
>> >
>> > Thanks,
>> > Lahiru
>> >
>> >
>> >
>> > On Tue, Aug 20, 2013 at 3:28 PM, Mailing list used for User
>> Communication
>> > and Updates <kal...@li...> wrote:
>> >>
>> >> num_jobs_nnet should be the same as the #machine (i.e. 4), but you may
>> >> want to decrease the learning rate a bit (e.g. by a factor of 2) if
>> >> you reducde the #machines from 16 to 4.
>> >> Dan
>> >>
>> >>
>> >> On Tue, Aug 20, 2013 at 9:21 AM, Mailing list used for User
>> >> Communication and Updates <kal...@li...> wrote:
>> >> > Hi All,
>> >> >
>> >> > I am planning to run the wsj/s5 set up for DNN training on 4 machines
>> >> > and
>> >> > each has 16 cores. Could anyone give me some pointers about how to
>> >> > change
>> >> > important parameters like num_jobs_nnet in the script for my setup?
>> >> >
>> >> > Thank you,
>> >> >
>> >> > Best Regards,
>> >> > Lahiru
>> >> >
>> >> >
>> >> >
>> ------------------------------------------------------------------------------
>> >> > Introducing Performance Central, a new site from SourceForge and
>> >> > AppDynamics. Performance Central is your source for news, insights,
>> >> > analysis and resources for efficient Application Performance
>> Management.
>> >> > Visit us today!
>> >> >
>> >> >
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>> >> > _______________________________________________
>> >> > Kaldi-users mailing list
>> >> > Kal...@li...
>> >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >> >
>> >>
>> >>
>> >>
>> ------------------------------------------------------------------------------
>> >> Introducing Performance Central, a new site from SourceForge and
>> >> AppDynamics. Performance Central is your source for news, insights,
>> >> analysis and resources for efficient Application Performance
>> Management.
>> >> Visit us today!
>> >>
>> >>
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>> >> _______________________________________________
>> >> Kaldi-users mailing list
>> >> Kal...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Introducing Performance Central, a new site from SourceForge and
>> > AppDynamics. Performance Central is your source for news, insights,
>> > analysis and resources for efficient Application Performance Management.
>> > Visit us today!
>> >
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>> > _______________________________________________
>> > Kaldi-users mailing list
>> > Kal...@li...
>> > https://lists.sourceforge.net/lists/listinfo/kaldi-users
>> >
>>
>>
>> ------------------------------------------------------------------------------
>> Introducing Performance Central, a new site from SourceForge and
>> AppDynamics. Performance Central is your source for news, insights,
>> analysis and resources for efficient Application Performance Management.
>> Visit us today!
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk
>> _______________________________________________
>> Kaldi-users mailing list
>> Kal...@li...
>> https://lists.sourceforge.net/lists/listinfo/kaldi-users
>>
>
>
|
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-21 09:08:29
|
Thanks a lot. On Wed, Aug 21, 2013 at 5:07 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > No, you don't have to submit the script to sge, the script doesn't do > much apart from submit jobs to SGE itself, and it's more convenient to > just run it from a command line. (but use nohup or run it inside > screen to stop it getting interrupted if you log out). > > Dan > > > On Wed, Aug 21, 2013 at 11:04 AM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: > > When running the dnn training on cpu, is it necessary to submit the > script > > to SGE explicitly? > > Or just running the script will take care of the job because the script > uses > > the queue.pl? > > > > Thanks, > > Lahiru > > > > > > > > On Tue, Aug 20, 2013 at 3:28 PM, Mailing list used for User Communication > > and Updates <kal...@li...> wrote: > >> > >> num_jobs_nnet should be the same as the #machine (i.e. 4), but you may > >> want to decrease the learning rate a bit (e.g. by a factor of 2) if > >> you reducde the #machines from 16 to 4. > >> Dan > >> > >> > >> On Tue, Aug 20, 2013 at 9:21 AM, Mailing list used for User > >> Communication and Updates <kal...@li...> wrote: > >> > Hi All, > >> > > >> > I am planning to run the wsj/s5 set up for DNN training on 4 machines > >> > and > >> > each has 16 cores. Could anyone give me some pointers about how to > >> > change > >> > important parameters like num_jobs_nnet in the script for my setup? > >> > > >> > Thank you, > >> > > >> > Best Regards, > >> > Lahiru > >> > > >> > > >> > > ------------------------------------------------------------------------------ > >> > Introducing Performance Central, a new site from SourceForge and > >> > AppDynamics. Performance Central is your source for news, insights, > >> > analysis and resources for efficient Application Performance > Management. > >> > Visit us today! > >> > > >> > > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > >> > _______________________________________________ > >> > Kaldi-users mailing list > >> > Kal...@li... > >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users > >> > > >> > >> > >> > ------------------------------------------------------------------------------ > >> Introducing Performance Central, a new site from SourceForge and > >> AppDynamics. Performance Central is your source for news, insights, > >> analysis and resources for efficient Application Performance Management. > >> Visit us today! > >> > >> > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > >> _______________________________________________ > >> Kaldi-users mailing list > >> Kal...@li... > >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > > > > > ------------------------------------------------------------------------------ > > Introducing Performance Central, a new site from SourceForge and > > AppDynamics. Performance Central is your source for news, insights, > > analysis and resources for efficient Application Performance Management. > > Visit us today! > > > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > > _______________________________________________ > > Kaldi-users mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-21 09:07:29
|
No, you don't have to submit the script to sge, the script doesn't do much apart from submit jobs to SGE itself, and it's more convenient to just run it from a command line. (but use nohup or run it inside screen to stop it getting interrupted if you log out). Dan On Wed, Aug 21, 2013 at 11:04 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > When running the dnn training on cpu, is it necessary to submit the script > to SGE explicitly? > Or just running the script will take care of the job because the script uses > the queue.pl? > > Thanks, > Lahiru > > > > On Tue, Aug 20, 2013 at 3:28 PM, Mailing list used for User Communication > and Updates <kal...@li...> wrote: >> >> num_jobs_nnet should be the same as the #machine (i.e. 4), but you may >> want to decrease the learning rate a bit (e.g. by a factor of 2) if >> you reducde the #machines from 16 to 4. >> Dan >> >> >> On Tue, Aug 20, 2013 at 9:21 AM, Mailing list used for User >> Communication and Updates <kal...@li...> wrote: >> > Hi All, >> > >> > I am planning to run the wsj/s5 set up for DNN training on 4 machines >> > and >> > each has 16 cores. Could anyone give me some pointers about how to >> > change >> > important parameters like num_jobs_nnet in the script for my setup? >> > >> > Thank you, >> > >> > Best Regards, >> > Lahiru >> > >> > >> > ------------------------------------------------------------------------------ >> > Introducing Performance Central, a new site from SourceForge and >> > AppDynamics. Performance Central is your source for news, insights, >> > analysis and resources for efficient Application Performance Management. >> > Visit us today! >> > >> > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> > _______________________________________________ >> > Kaldi-users mailing list >> > Kal...@li... >> > https://lists.sourceforge.net/lists/listinfo/kaldi-users >> > >> >> >> ------------------------------------------------------------------------------ >> Introducing Performance Central, a new site from SourceForge and >> AppDynamics. Performance Central is your source for news, insights, >> analysis and resources for efficient Application Performance Management. >> Visit us today! >> >> http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk >> _______________________________________________ >> Kaldi-users mailing list >> Kal...@li... >> https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-21 09:05:01
|
When running the dnn training on cpu, is it necessary to submit the script to SGE explicitly? Or just running the script will take care of the job because the script uses the queue.pl? Thanks, Lahiru On Tue, Aug 20, 2013 at 3:28 PM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > num_jobs_nnet should be the same as the #machine (i.e. 4), but you may > want to decrease the learning rate a bit (e.g. by a factor of 2) if > you reducde the #machines from 16 to 4. > Dan > > > On Tue, Aug 20, 2013 at 9:21 AM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: > > Hi All, > > > > I am planning to run the wsj/s5 set up for DNN training on 4 machines and > > each has 16 cores. Could anyone give me some pointers about how to change > > important parameters like num_jobs_nnet in the script for my setup? > > > > Thank you, > > > > Best Regards, > > Lahiru > > > > > ------------------------------------------------------------------------------ > > Introducing Performance Central, a new site from SourceForge and > > AppDynamics. Performance Central is your source for news, insights, > > analysis and resources for efficient Application Performance Management. > > Visit us today! > > > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > > _______________________________________________ > > Kaldi-users mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-20 08:32:44
|
Yes, it's fixed now. Shiyin 2013/8/20 Mailing list used for User Communication and Updates < kal...@li...> > I just changed the code to explicitly instantiate that template, > although from looking at the code, for sure it should have > instantiated it as it was called in another thing that was explicitly > instantiated (i.e. it's a compiler bug). Let me know if my commit > fixes the problem. > Dan > > > On Tue, Aug 20, 2013 at 10:07 AM, Mailing list used for User > Communication and Updates <kal...@li...> wrote: > > hi, > > > > I'm trying to compile kaldi with intel compiler and option '-O3'. > > I got an Error when compiling the test code in dir 'trunk/src/matrix' as > > follows. > > This error does not happen when compile with 'gcc' or 'icpc -O2'. > > > > It seems 'icpc -O3' optimized out the interface of the instantiated > > template function > > void SpMatrix<Real>::Qr(MatrixBase<Real> *Q); > > > > Maybe SpMatrix::Qr()have to be explicitly instantiated in qr.cc in this > > case. > > > > Best, > > Shiyin > > > > > > > > /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix $ make test > > icpc -rdynamic -L/data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib > > -Wl,-R/data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib > > matrix-lib-test.o kaldi-matrix.a ../base/kaldi-base.a > > /data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib/libfst.a -ldl > > -Wl,--start-group > > /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_intel_lp64.a > > /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_sequential.a > > /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_core.a > > -Wl,--end-group -L/opt/intel/lib/intel64 -Wl,-Bstatic -liomp5 > -Wl,-Bdynamic > > -lm -lpthread -o matrix-lib-test > > matrix-lib-test.o: In function `void > > kaldi::UnitTestTridiagonalizeAndQr<double>()': > > /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix/matrix-lib-test.cc:1373: > > undefined reference to > > `kaldi::SpMatrix<double>::Qr(kaldi::MatrixBase<double>*)' > > matrix-lib-test.o: In function `void > > kaldi::UnitTestTridiagonalizeAndQr<float>()': > > /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix/matrix-lib-test.cc:1373: > > undefined reference to > > `kaldi::SpMatrix<float>::Qr(kaldi::MatrixBase<float>*)' > > make: *** [matrix-lib-test] Error 1 > > > > > > > > > > > ------------------------------------------------------------------------------ > > Introducing Performance Central, a new site from SourceForge and > > AppDynamics. Performance Central is your source for news, insights, > > analysis and resources for efficient Application Performance Management. > > Visit us today! > > > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > > _______________________________________________ > > Kaldi-users mailing list > > Kal...@li... > > https://lists.sourceforge.net/lists/listinfo/kaldi-users > > > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-20 08:16:19
|
I just changed the code to explicitly instantiate that template, although from looking at the code, for sure it should have instantiated it as it was called in another thing that was explicitly instantiated (i.e. it's a compiler bug). Let me know if my commit fixes the problem. Dan On Tue, Aug 20, 2013 at 10:07 AM, Mailing list used for User Communication and Updates <kal...@li...> wrote: > hi, > > I'm trying to compile kaldi with intel compiler and option '-O3'. > I got an Error when compiling the test code in dir 'trunk/src/matrix' as > follows. > This error does not happen when compile with 'gcc' or 'icpc -O2'. > > It seems 'icpc -O3' optimized out the interface of the instantiated > template function > void SpMatrix<Real>::Qr(MatrixBase<Real> *Q); > > Maybe SpMatrix::Qr()have to be explicitly instantiated in qr.cc in this > case. > > Best, > Shiyin > > > > /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix $ make test > icpc -rdynamic -L/data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib > -Wl,-R/data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib > matrix-lib-test.o kaldi-matrix.a ../base/kaldi-base.a > /data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib/libfst.a -ldl > -Wl,--start-group > /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_intel_lp64.a > /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_sequential.a > /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_core.a > -Wl,--end-group -L/opt/intel/lib/intel64 -Wl,-Bstatic -liomp5 -Wl,-Bdynamic > -lm -lpthread -o matrix-lib-test > matrix-lib-test.o: In function `void > kaldi::UnitTestTridiagonalizeAndQr<double>()': > /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix/matrix-lib-test.cc:1373: > undefined reference to > `kaldi::SpMatrix<double>::Qr(kaldi::MatrixBase<double>*)' > matrix-lib-test.o: In function `void > kaldi::UnitTestTridiagonalizeAndQr<float>()': > /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix/matrix-lib-test.cc:1373: > undefined reference to > `kaldi::SpMatrix<float>::Qr(kaldi::MatrixBase<float>*)' > make: *** [matrix-lib-test] Error 1 > > > > > ------------------------------------------------------------------------------ > Introducing Performance Central, a new site from SourceForge and > AppDynamics. Performance Central is your source for news, insights, > analysis and resources for efficient Application Performance Management. > Visit us today! > http://pubads.g.doubleclick.net/gampad/clk?id=48897511&iu=/4140/ostg.clktrk > _______________________________________________ > Kaldi-users mailing list > Kal...@li... > https://lists.sourceforge.net/lists/listinfo/kaldi-users > |
|
From: Mailing l. u. f. U. C. a. U. <kal...@li...> - 2013-08-20 08:07:39
|
hi, I'm trying to compile kaldi with intel compiler and option '-O3'. I got an Error when compiling the test code in dir 'trunk/src/matrix' as follows. This error does not happen when compile with 'gcc' or 'icpc -O2'. It seems 'icpc -O3' optimized out the interface of the instantiated template function void SpMatrix<Real>::Qr(MatrixBase<Real> *Q); Maybe SpMatrix::Qr()have to be explicitly instantiated in qr.cc in this case. Best, Shiyin /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix $ make test icpc -rdynamic -L/data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib -Wl,-R/data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib matrix-lib-test.o kaldi-matrix.a ../base/kaldi-base.a /data/sykang/work/asr/kaldi-cpu/trunk/tools/openfst/lib/libfst.a -ldl -Wl,--start-group /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_intel_lp64.a /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_sequential.a /opt/intel/composer_xe_2013.1.117/mkl/lib/intel64/libmkl_core.a -Wl,--end-group -L/opt/intel/lib/intel64 -Wl,-Bstatic -liomp5 -Wl,-Bdynamic -lm -lpthread -o matrix-lib-test matrix-lib-test.o: In function `void kaldi::UnitTestTridiagonalizeAndQr<double>()': /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix/matrix-lib-test.cc:1373: undefined reference to `kaldi::SpMatrix<double>::Qr(kaldi::MatrixBase<double>*)' matrix-lib-test.o: In function `void kaldi::UnitTestTridiagonalizeAndQr<float>()': /data/sykang/work/asr/kaldi-cpu/trunk/src/matrix/matrix-lib-test.cc:1373: undefined reference to `kaldi::SpMatrix<float>::Qr(kaldi::MatrixBase<float>*)' make: *** [matrix-lib-test] Error 1 |
1 message has been excluded from this view by a project administrator.
