Menu ▾ ▴

LDA/MLLT training question

Help
Berker Batur
2010-11-22
2012-09-22

  • Berker Batur

    Berker Batur - 2010-11-22

    Hi,
    I have been trying to train with LDA/MLLT but the following error was occured.

    FATAL_ERROR: "main.c", line 98: Failed to read LDA matrix

    Here is the log file in Phase 4: LDA transform estimation :

    INFO: main.c(196): Compiled on Nov 22 2010 at 10:04:12
    /home/berker/sphinx/an4/bin/bw \
    -moddeffn /home/berker/sphinx/an4/model_architecture/an4.ci.mdef \
    -ts2cbfn .cont. \
    -mixwfn /home/berker/sphinx/an4/model_parameters/an4.ci_lda/mixture_weights \
    -mwfloor 1e-05 \
    -tpfloor 1e-05 \
    -tmatfn /home/berker/sphinx/an4/model_parameters/an4.ci_lda/transition_matrices \
    -meanfn /home/berker/sphinx/an4/model_parameters/an4.ci_lda/means \
    -varfn /home/berker/sphinx/an4/model_parameters/an4.ci_lda/variances \
    -ltsoov no \
    -dictfn /home/berker/sphinx/an4/etc/an4.dic \
    -fdictfn /home/berker/sphinx/an4/etc/an4.filler \
    -ctlfn /home/berker/sphinx/an4/etc/an4_train.fileids \
    -part 1 \
    -npart 1 \
    -cepdir /home/berker/sphinx/an4/feat \
    -cepext mfc \
    -lsnfn /home/berker/sphinx/an4/etc/an4_train.transcription \
    -accumdir /home/berker/sphinx/an4/bwaccumdir/an4_buff_1 \
    -varfloor 0.0001 \
    -topn 8 \
    -abeam 1e-90 \
    -bbeam 1e-10 \
    -agc none \
    -cmn current \
    -varnorm no \
    -meanreest yes \
    -varreest yes -2passvar yes \
    -fullvar yes \
    -diagfull yes \
    -feat 1s_c_d_dd \
    -ceplen 13 \
    -cepwin 0 \
    -timing no

    -help no no
    -example no no
    -hmmdir
    -moddeffn /home/berker/sphinx/an4/model_architecture/an4.ci.mdef
    -tmatfn /home/berker/sphinx/an4/model_parameters/an4.ci_lda/transition_matrices
    -mixwfn /home/berker/sphinx/an4/model_parameters/an4.ci_lda/mixture_weights
    -meanfn /home/berker/sphinx/an4/model_parameters/an4.ci_lda/means
    -varfn /home/berker/sphinx/an4/model_parameters/an4.ci_lda/variances
    -fullvar no yes
    -diagfull no yes
    -mwfloor 0.00001 1.000000e-05
    -tpfloor 0.0001 1.000000e-05
    -varfloor 0.00001 1.000000e-04
    -topn 4 8
    -dictfn /home/berker/sphinx/an4/etc/an4.dic
    -fdictfn /home/berker/sphinx/an4/etc/an4.filler
    -ltsoov no no
    -ctlfn /home/berker/sphinx/an4/etc/an4_train.fileids
    -nskip
    -runlen -1 -1
    -part 1
    -npart 1
    -cepext mfc mfc
    -cepdir /home/berker/sphinx/an4/feat
    -phsegext phseg phseg
    -phsegdir
    -outphsegdir
    -sentdir
    -sentext sent sent
    -lsnfn /home/berker/sphinx/an4/etc/an4_train.transcription
    -accumdir /home/berker/sphinx/an4/bwaccumdir/an4_buff_1
    -ceplen 13 13
    -cepwin 0 0
    -agc max none
    -cmn current current
    -varnorm no no
    -silcomp none none
    -sildel no no
    -siltag SIL SIL
    -abeam 1e-100 1.000000e-90
    -bbeam 1e-100 1.000000e-10
    -varreest yes yes
    -meanreest yes yes
    -mixwreest yes yes
    -tmatreest yes yes
    -mllrmat
    -cb2mllrfn .1cls. .1cls.
    -ts2cbfn .cont.
    -feat 1s_c_d_dd 1s_c_d_dd
    -svspec
    -ldafn
    -ldadim 29 29
    -ldaaccum no no
    -timing yes no
    -viterbi no no
    -2passvar no yes
    -sildelfn
    -spthresh 0.0 0.000000e+00
    -maxuttlen 0 0
    -ckptintv
    -outputfullpath no no
    -fullsuffixmatch no no
    -pdumpdir
    INFO: main.c(255): Reading
    /home/berker/sphinx/an4/model_architecture/an4.ci.mdef
    INFO: model_def_io.c(587): Model definition info:
    INFO: model_def_io.c(588): 40 total models defined (40 base, 0 tri)
    INFO: model_def_io.c(589): 160 total states
    INFO: model_def_io.c(590): 120 total tied states
    INFO: model_def_io.c(591): 120 total tied CI states
    INFO: model_def_io.c(592): 40 total tied transition matrices
    INFO: model_def_io.c(593): 4 max state/model
    INFO: model_def_io.c(594): 4 min state/model
    INFO: s3mixw_io.c(116): Read
    /home/berker/sphinx/an4/model_parameters/an4.ci_lda/mixture_weights
    INFO: s3tmat_io.c(115): Read
    /home/berker/sphinx/an4/model_parameters/an4.ci_lda/transition_matrices
    INFO: mod_inv.c(297): inserting tprob floor 1.000000e-05 and renormalizing
    INFO: s3gau_io.c(166): Read
    /home/berker/sphinx/an4/model_parameters/an4.ci_lda/means
    ERROR: "s3gau_full_io.c", line 129: Failed to read full covariance file
    /home/berker/sphinx/an4/model_parameters/an4.ci_lda/variances (expected 182520
    values, got 4680)
    INFO: s3gau_io.c(166): Read
    /home/berker/sphinx/an4/model_parameters/an4.ci_lda/means
    INFO: s3gau_io.c(166): Read
    /home/berker/sphinx/an4/model_parameters/an4.ci_lda/variances
    INFO: gauden.c(181): 120 total mgau
    INFO: gauden.c(155): 1 feature streams (|0|=39 )
    INFO: gauden.c(192): 1 total densities
    INFO: gauden.c(98): min_var=1.000000e-04
    WARNING: "mod_inv.c", line 257: n_top 8 > n_density 1. n_top <- 1
    INFO: gauden.c(170): compute 1 densities/frame
    INFO: main.c(363): Will reestimate mixing weights.
    INFO: main.c(365): Will reestimate means.
    INFO: main.c(367): Will reestimate variances.
    INFO: main.c(369): WIll NOT optionally delete silence in Baum Welch or
    Viterbi.
    INFO: main.c(377): Will reestimate transition matrices
    INFO: main.c(390): Reading main lexicon: /home/berker/sphinx/an4/etc/an4.dic
    INFO: lexicon.c(233): 133656 entries added from
    /home/berker/sphinx/an4/etc/an4.dic
    INFO: main.c(402): Reading filler lexicon:
    /home/berker/sphinx/an4/etc/an4.filler
    INFO: lexicon.c(233): 3 entries added from
    /home/berker/sphinx/an4/etc/an4.filler
    INFO: main.c(423): Silence Tag SIL
    INFO: corpus.c(1343): Will process all remaining utts starting at 0
    INFO: main.c(622): Reestimation: Baum-Welch
    column defns
    <seq>
    <id>
    <n_frame_in>
    <n_frame_del>
    <n_state_shmm>
    <avg_states_alpha>
    <avg_states_beta>
    <avg_states_reest>
    <avg_posterior_prune>
    <frame_log_lik>
    <utt_log_lik>
    ... timing info ...
    utt> 0 00 586 0INFO: cvt2triphone.c(199): no multiphones defined, no
    conversion done
    172 14 1 3 2.256470e-12 5.673046e+00 3.324405e+03
    utt> 1 01 536 0 144 15 1 2 2.441693e-12 6.551326e+00 3.511511e+03
    utt> 2 02 449 0 120 15 2 3 3.761356e-12 8.665690e+00 3.890895e+03
    utt> 3 03 511 0 140 16 1 3 3.123241e-12 8.079638e+00 4.128695e+03
    utt> 4 04 536 0 184 18 1 2 2.144310e-12 4.341352e+00 2.326965e+03
    utt> 5 05 624 0 192 18 1 3 2.692306e-12 4.964245e+00 3.097689e+03
    utt> 6 06 661 0 188 17 1 2 2.184585e-12 5.940131e+00 3.926427e+03
    utt> 7 07 636 0 184 18 1 2 3.254788e-12 5.605101e+00 3.564844e+03
    utt> 8 08 411 0 92 15 1 3 2.299324e-12 8.576485e+00 3.524935e+03
    utt> 9 09 649 0 168 17 1 2 1.310831e-12 6.122083e+00 3.973232e+03
    utt> 10 10 499 0 184 23 1 3 2.560583e-12 1.877188e-01 9.367167e+01
    utt> 11 12 304 0 136 19 1 3 4.470153e-12 -6.363837e+00 -1.934606e+03
    overall> berker-desktop 6402 (-0) 5.221597e+00 3.342866e+04 </utt_log_lik></frame_log_lik></avg_posterior_prune></avg_states_reest></avg_states_beta></avg_states_alpha></n_state_shmm></n_frame_del></n_frame_in></id></seq>

    INFO: s3mixw_io.c(232): Wrote
    /home/berker/sphinx/an4/bwaccumdir/an4_buff_1/mixw_counts
    INFO: s3tmat_io.c(174): Wrote
    /home/berker/sphinx/an4/bwaccumdir/an4_buff_1/tmat_counts
    INFO: s3gau_full_io.c(497): Wrote
    /home/berker/sphinx/an4/bwaccumdir/an4_buff_1/gauden_counts with means with
    full vars (2pass)
    INFO: main.c(1035): Counts saved to
    /home/berker/sphinx/an4/bwaccumdir/an4_buff_1
    Mon Nov 22 10:13:15 2010

    Only this log file has an error message in Train LDA transformation.
    I installed python modules scipy and numpy.
    I am using SphinxTrain tarball in robust group tutorial.
    And also I am following this page : http://cmusphinx.sourceforge.net/wiki/lda
    mllt
    for LDA/MLLT training.
    I also searched this topic : http://sourceforge.net/projects/cmusphinx/forums
    /forum/5471/topic/3787435
    but no could not manage to solve the error.

     

  • Nickolay V. Shmyrev

    Nickolay V. Shmyrev - 2010-11-22

    I am using SphinxTrain tarball in robust group tutorial.

    Sorry, we can't provide you support for outdated versions. Use newer one. See

    http://cmusphinx.sourceforge.net/wiki/download

    for more details

    FATAL_ERROR: "main.c", line 98: Failed to read LDA matrix

    The matrix is missing because it was not created. Most common issue that leads
    to missing matrix is incorrect python setup. Most likely you wasn't accurate
    enough in configuration.

     

  • Berker Batur

    Berker Batur - 2010-11-22

    Thanx,

    I found the reason of why it couldn't create the matrix.
    Altough I executed python setup.py build command in SphinxTrain/python
    directory,
    in the output of training it was written that, sphinx looked
    an4/python/sphinx/lda.py instead of sphinxtrain/python/sphinx/lda.py
    I copied python directory in Sphinx Train to an4 directory where I called
    training scripts, than matrix successfully created.

     

Log in to post a comment.