sphinx training problem

[Sam Pasupalak]

I am doing sphinx training following the required tutorial. I have created all
the set of required files needed for training, but I get the following error
when I run the RunAll.pl script. It does not give me a clue of what and how to
debug. Can someone please help?

Phase 1: Cleaning up directories:
accumulator...logs...qmanager...models...
Phase 2: Flat initialize
Phase 3: Forward-Backward
Baum welch starting for 1 Gaussian(s), iteration: 1 (1 of 1)
0%
Only 0 parts of 1 of Baum Welch were successfully completed
Parts 1 failed to run!
Training failed in iteration 1
Something failed:
(/home/spasupal/Desktop/tutorial/myam/scripts_pl/20.ci_hmm/slave_convg.pl)

[Sam Pasupalak]

My log file has the following relevant error message:

bw: hash.c:254: hash_enter: Assertion `key != ((void *)0)' failed.

[Nickolay V. Shmyrev]

Your dictionary or your filler dictionary has empty line.

Please learn to provide versions of the software you are using when you report
about problems. That helps to identify them better and provide the solution
faster. If you think that your dictionary is good, learn to provide the files
you are using. You can upload them to any public file sharing resource and
give here a link. Both things save not just our time, but YOUR time as well.

[Sam Pasupalak]

Thanks for the help, now it stops at 60%. I am using the Sphinx 3 tarball
provided in the robust group tutorial : http://www.speech.cs.cmu.edu/sphinx/t
utorial.html#sphinx3.

I am getting the following error message now and it doesn't give me a clue
about where the error is:

./scripts_pl/20.ci_hmm/slave_convg.pl
MODULE: 20 Training Context Independent models
Phase 1: Cleaning up directories:
accumulator...logs...qmanager...models...
Phase 2: Flat initialize
Phase 3: Forward-Backward
Baum welch starting for 1 Gaussian(s), iteration: 1 (1 of 1)
0% 10% 20% 30% 40% 50% 60%
WARNING: This step had 0 ERROR messages and 2 WARNING messages. Please check
the log file for details.
Only 0 parts of 1 of Baum Welch were successfully completed
Parts 1 failed to run!
Training failed in iteration 1

I just know its in this script but there is no debugger to figure it out. I
checked spaces in every file now and its fine. Thanks.

[Sam Pasupalak]

This is my log file and I don't know if it helps to post here :

INFO: main.c(196): Compiled on Jul 26 2010 at 11:05:00
/home/spasupal/Desktop/tutorial/myam/bin/bw \
-moddeffn /home/spasupal/Desktop/tutorial/myam/model_architecture/myam.ci.mdef \
-ts2cbfn .cont. \
-mixwfn /home/spasupal/Desktop/tutorial/myam/model_parameters/myam.ci_cont_flatinitial/mixture_weights \
-mwfloor 1e-08 \
-tmatfn /home/spasupal/Desktop/tutorial/myam/model_parameters/myam.ci_cont_flatinitial/transition_matrices \
-meanfn /home/spasupal/Desktop/tutorial/myam/model_parameters/myam.ci_cont_flatinitial/means \
-varfn /home/spasupal/Desktop/tutorial/myam/model_parameters/myam.ci_cont_flatinitial/variances \
-ltsoov no \
-dictfn /home/spasupal/Desktop/tutorial/myam/etc/myam.dic \
-fdictfn /home/spasupal/Desktop/tutorial/myam/etc/myam.filler \
-ctlfn /home/spasupal/Desktop/tutorial/myam/etc/myam_train.fileids \
-part 1 \
-npart 1 \
-cepdir /home/spasupal/Desktop/tutorial/myam/feat \
-cepext mfc \
-lsnfn /home/spasupal/Desktop/tutorial/myam/etc/myam_train.transcription \
-accumdir /home/spasupal/Desktop/tutorial/myam/bwaccumdir/myam_buff_1 \
-varfloor 0.0001 \
-topn 1 \
-abeam 1e-90 \
-bbeam 1e-10 \
-agc none \
-cmn current \
-varnorm no \
-meanreest yes \
-varreest yes -2passvar no \
-tmatreest yes \
-fullvar no \
-diagfull no \
-feat 1s_c_d_dd \
-ceplen 13 \
-timing no

-help no no
-example no no
-hmmdir
-moddeffn /home/spasupal/Desktop/tutorial/myam/model_architecture/myam.ci.mdef
-tmatfn /home/spasupal/Desktop/tutorial/myam/model_parameters/myam.ci_cont_flatinitial/transition_matrices
-mixwfn /home/spasupal/Desktop/tutorial/myam/model_parameters/myam.ci_cont_flatinitial/mixture_weights
-meanfn /home/spasupal/Desktop/tutorial/myam/model_parameters/myam.ci_cont_flatinitial/means
-varfn /home/spasupal/Desktop/tutorial/myam/model_parameters/myam.ci_cont_flatinitial/variances
-fullvar no no
-diagfull no no
-mwfloor 0.00001 1.000000e-08
-tpfloor 0.0001 1.000000e-04
-varfloor 0.00001 1.000000e-04
-topn 4 1
-dictfn /home/spasupal/Desktop/tutorial/myam/etc/myam.dic
-fdictfn /home/spasupal/Desktop/tutorial/myam/etc/myam.filler
-ltsoov no no
-ctlfn /home/spasupal/Desktop/tutorial/myam/etc/myam_train.fileids
-nskip
-runlen -1 -1
-part 1
-npart 1
-cepext mfc mfc
-cepdir /home/spasupal/Desktop/tutorial/myam/feat
-phsegext phseg phseg
-phsegdir
-outphsegdir
-sentdir
-sentext sent sent
-lsnfn /home/spasupal/Desktop/tutorial/myam/etc/myam_train.transcription
-accumdir /home/spasupal/Desktop/tutorial/myam/bwaccumdir/myam_buff_1
-ceplen 13 13
-cepwin 0 0
-agc max none
-cmn current current
-varnorm no no
-silcomp none none
-sildel no no
-siltag SIL SIL
-abeam 1e-100 1.000000e-90
-bbeam 1e-100 1.000000e-10
-varreest yes yes
-meanreest yes yes
-mixwreest yes yes
-tmatreest yes yes
-mllrmat
-cb2mllrfn .1cls. .1cls.
-ts2cbfn .cont.
-feat 1s_c_d_dd 1s_c_d_dd
-svspec
-ldafn
-ldadim 29 29
-ldaaccum no no
-timing yes no
-viterbi no no
-2passvar no no
-sildelfn
-spthresh 0.0 0.000000e+00
-maxuttlen 0 0
-ckptintv
-outputfullpath no no
-fullsuffixmatch no no
-pdumpdir
INFO: main.c(255): Reading
/home/spasupal/Desktop/tutorial/myam/model_architecture/myam.ci.mdef
INFO: model_def_io.c(587): Model definition info:
INFO: model_def_io.c(588): 33 total models defined (33 base, 0 tri)
INFO: model_def_io.c(589): 132 total states
INFO: model_def_io.c(590): 99 total tied states
INFO: model_def_io.c(591): 99 total tied CI states
INFO: model_def_io.c(592): 33 total tied transition matrices
INFO: model_def_io.c(593): 4 max state/model
INFO: model_def_io.c(594): 4 min state/model
INFO: s3mixw_io.c(116): Read /home/spasupal/Desktop/tutorial/myam/model_parame
ters/myam.ci_cont_flatinitial/mixture_weights
INFO: s3tmat_io.c(115): Read /home/spasupal/Desktop/tutorial/myam/model_parame
ters/myam.ci_cont_flatinitial/transition_matrices
INFO: mod_inv.c(297): inserting tprob floor 1.000000e-04 and renormalizing
INFO: s3gau_io.c(166): Read /home/spasupal/Desktop/tutorial/myam/model_paramet
ers/myam.ci_cont_flatinitial/means
INFO: s3gau_io.c(166): Read /home/spasupal/Desktop/tutorial/myam/model_paramet
ers/myam.ci_cont_flatinitial/variances
INFO: gauden.c(181): 99 total mgau
INFO: gauden.c(155): 1 feature streams (|0|=39 )
INFO: gauden.c(192): 1 total densities
INFO: gauden.c(98): min_var=1.000000e-04
INFO: gauden.c(170): compute 1 densities/frame
INFO: main.c(363): Will reestimate mixing weights.
INFO: main.c(365): Will reestimate means.
INFO: main.c(367): Will reestimate variances.
INFO: main.c(369): WIll NOT optionally delete silence in Baum Welch or
Viterbi.
INFO: main.c(377): Will reestimate transition matrices
INFO: main.c(390): Reading main lexicon:
/home/spasupal/Desktop/tutorial/myam/etc/myam.dic
INFO: lexicon.c(233): 63 entries added from
/home/spasupal/Desktop/tutorial/myam/etc/myam.dic
INFO: main.c(402): Reading filler lexicon:
/home/spasupal/Desktop/tutorial/myam/etc/myam.filler
INFO: lexicon.c(233): 3 entries added from
/home/spasupal/Desktop/tutorial/myam/etc/myam.filler
INFO: main.c(423): Silence Tag SIL
INFO: corpus.c(1343): Will process all remaining utts starting at 0
INFO: main.c(622): Reestimation: Baum-Welch
column defns
<seq>
<id>
<n_frame_in>
<n_frame_del>
<n_state_shmm>
<avg_states_alpha>
<avg_states_beta>
<avg_states_reest>
<avg_posterior_prune>
<frame_log_lik>
<utt_log_lik>
... timing info ...
utt> 0 File01 538 0INFO: cvt2triphone.c(199): no multiphones defined, no
conversion done
16 15 11 21 5.042202e-12 7.447113e+00 4.006547e+03
utt> 1 File02 655 0 24 23 15 30 1.129555e-11 8.846831e+00 5.794674e+03
utt> 2 File03 554 0 16 15 11 21 5.084486e-12 2.129505e+01 1.179746e+04
utt> 3 File04 831 0 48 46 24 49 2.527441e-11 1.287454e+01 1.069874e+04
utt> 4 File05 970 0 36 35 20 40 1.940533e-11 2.023713e+01 1.963002e+04
utt> 5 File06 970 0 48 47 24 49 2.584438e-11 1.442929e+01 1.399641e+04
utt> 6 File07 970 0 48 47 24 49 2.584438e-11 1.538105e+01 1.491962e+04
utt> 7 File08 1247 0 48 47 24 49 2.560824e-11 1.472863e+01 1.836660e+04
utt> 8 File09 1108 0 32 31 19 37 1.703171e-11 1.502056e+01 1.664279e+04
utt> 9 File10 1385 0 36 35 20 40 1.940012e-11 2.082443e+01 2.884183e+04
utt> 10 File11 1247 0 52 51 26 52 2.739898e-11 1.981804e+01 2.471310e+04
utt> 11 File12 1247 0WARNING: "mk_phone_list.c", line 173: Unable to lookup
(File12) in the lexicon
WARNING: "next_utt_states.c", line 82: Unable to produce CI phones for utt
Sun Aug 8 00:59:00 2010</utt_log_lik></frame_log_lik></avg_posterior_prune></avg_states_reest></avg_states_beta></avg_states_alpha></n_state_shmm></n_frame_del></n_frame_in></id></seq>

[Nickolay V. Shmyrev]

utt> 11 File12 1247 0WARNING: "mk_phone_list.c", line 173: Unable to lookup
(File12) in the lexicon

It fails to find pronunciation for the word "File12". It looks for a
pronunciation for the word "File12" because your transcription file has wrong
format on the line containing this word.

[Nickolay V. Shmyrev]

And most of your problems raise from using obsolete tutorial and obsolete
software. Use this

http://cmusphinx.sourceforge.net/wiki/tutorialam

and

http://cmusphinx.sourceforge.net/wiki/download/

[Sam Pasupalak]

Thanks Nickolay, It worked :)