I'm using bw for adaptation of a monophone model, and I get errors whenever i use the "-outphsegdir" parameter.
I do not get these same errors when I don't use "-outphsegdir"
The files generated by bw are identical, regardless of whether I output the phsegs.
However, I'm concerned about the presence of the errors, and the fact that some of the phseg outputs contain the "(null)" token. Is this in itself an issue, and could the mixw errors indicate a problem with my model? (both the unadapted and adapted model work ok when decoding)
An example phseg file (phrase is "this is the best study ever"):
SFrm EFrm SegAScr Phone
0 1 0 (null)
2 3 -1402445 (null)
4 5 -5859855 S
6 7 -8610662 IH
8 23 -38319833 Z
24 29 -10311756 (null)
30 41 -1605764 AX
42 44 -7710875 (null)
45 58 -18776771 EH
59 65 -3246114 (null)
66 67 -4414090 (null)
68 81 -52909845 S
82 83 -10132351 (null)
84 89 -10177282 (null)
90 91 -2909178 (null)
92 110 -9040795 IY
111 123 -4868306 EH
124 129 -3076250 V
130 151 -11893777 ER
[Switch][Default][Value]
-help no no
-example no no
-hmmdir
-moddeffn /homes/stupakov/research/models/am/timit/mdef
-tmatfn /homes/stupakov/research/models/am/timit/transition_matrices
-mixwfn /homes/stupakov/research/models/am/timit/mixture_weights
-meanfn /homes/stupakov/research/models/am/timit/means
-varfn /homes/stupakov/research/models/am/timit/variances
-fullvar no no
-diagfull no no
-mwfloor 0.00001 1.000000e-05
-tpfloor 0.0001 1.000000e-04
-varfloor 0.00001 1.000000e-05
-topn 4 4
-dictfn /homes/stupakov/research/dict/cmudict.0.7a_nolex_timit39
-fdictfn /homes/stupakov/research/am_train/adaptation/timit_dict/timit.nosil.filler.dict
-ltsoov no no
-ctlfn /homes/stupakov/research/am_train/adaptation/cosine/lists/ctl.lst
-nskip
-runlen -1 -1
-part
-npart
-cepext mfc mfcc
-cepdir /
-phsegext phseg phseg
-phsegdir
-outphsegdir /homes/stupakov/research/corpus/audio/cosine_train_adapt_segs//phseg
-sentdir
-sentext sent sent
-lsnfn /homes/stupakov/research/am_train/adaptation/cosine/cosine_train_no_OOV.trans
-accumdir /homes/stupakov/research/am_train/adaptation/cosine/cosine_train_no_OOV/accumdir
-ceplen 13 39
-cepwin 0 0
-agc max none
-cmn current none
-varnorm no no
-silcomp none none
-sildel no no
-siltag SIL SIL
-abeam 1e-100 1.000000e-100
-bbeam 1e-100 1.000000e-100
-varreest yes no
-meanreest yes yes
-mixwreest yes no
-tmatreest yes yes
-mllrmat
-cb2mllrfn .1cls. .1cls.
-ts2cbfn .cont.
-feat 1s_c_d_dd 1s_c
-ldafn
-ldadim 29 29
-ldaaccum no no
-timing yes yes
-viterbi no no
-2passvar no no
-sildelfn
-spthresh 0.0 0.000000e+00
-maxuttlen 0 0
-ckptintv
-outputfullpath no no
-fullsuffixmatch no no
-pdumpdir
INFO: main.c(253): Reading /homes/stupakov/research/models/am/timit/mdef
INFO: model_def_io.c(587): Model definition info:
INFO: model_def_io.c(588): 50 total models defined (50 base, 0 tri)
INFO: model_def_io.c(589): 200 total states
INFO: model_def_io.c(590): 150 total tied states
INFO: model_def_io.c(591): 150 total tied CI states
INFO: model_def_io.c(592): 50 total tied transition matrices
INFO: model_def_io.c(593): 4 max state/model
INFO: model_def_io.c(594): 4 min state/model
INFO: s3mixw_io.c(116): Read /homes/stupakov/research/models/am/timit/mixture_weights [150x1x128 array]
INFO: s3tmat_io.c(115): Read /homes/stupakov/research/models/am/timit/transition_matrices [50x3x4 array]
INFO: mod_inv.c(297): inserting tprob floor 1.000000e-04 and renormalizing
INFO: s3gau_io.c(166): Read /homes/stupakov/research/models/am/timit/means [150x1x128 array]
INFO: s3gau_io.c(166): Read /homes/stupakov/research/models/am/timit/variances [150x1x128 array]
INFO: gauden.c(181): 150 total mgau
INFO: gauden.c(155): 1 feature streams (|0|=39 )
INFO: gauden.c(192): 128 total densities
INFO: gauden.c(98): min_var=1.000000e-05
INFO: gauden.c(170): compute 4 densities/frame
INFO: main.c(361): Will NOT reestimate mixing weights.
INFO: main.c(363): Will reestimate means.
INFO: main.c(365): Will NOT reestimate variances.
INFO: main.c(367): WIll NOT optionally delete silence in Baum Welch or Viterbi.
INFO: main.c(375): Will reestimate transition matrices
INFO: main.c(388): Reading main lexicon: /homes/stupakov/research/dict/cmudict.0.7a_nolex_timit39
INFO: lexicon.c(233): 133764 entries added from /homes/stupakov/research/dict/cmudict.0.7a_nolex_timit39
INFO: main.c(400): Reading filler lexicon: /homes/stupakov/research/am_train/adaptation/timit_dict/timit.nosil.filler.dict
INFO: lexicon.c(233): 3 entries added from /homes/stupakov/research/am_train/adaptation/timit_dict/timit.nosil.filler.dict
INFO: main.c(421): Silence Tag SIL
INFO: corpus.c(1343): Will process all remaining utts starting at 0
INFO: main.c(620): Reestimation: Baum-Welch
INFO: main.c(625): Generating profiling information consumes significant CPU resources.
INFO: main.c(626): If you are not interested in profiling, use -timing no
column defns
<seq>
<id>
<n_frame_in>
<n_frame_del>
<n_state_shmm>
<avg_states_alpha>
<avg_states_beta>
<avg_states_reest>
<avg_posterior_prune>
<frame_log_lik>
<utt_log_lik>
... timing info ...
utt> 0 0 54 0INFO: cvt2triphone.c(199): no multiphones defined, no conversion done
20 13 ERROR: "viterbi.c", line 131: mixw 82 not found
9 12 1.087832e-102 -7.057864e+01 -3.811247e+03 utt 0.104x 1.095e upd 0.102x 1.051e fwd 0.039x 1.108e bwd 0.061x 1.009e gau 0.157x 1.236e rsts 0.007x 1.419e rstf 0.004x 0.893e rstu 0.002x 0.935e
utt> 1 1 152 0 76 41 ERROR: "viterbi.c", line 131: mixw 25 not found
ERROR: "viterbi.c", line 131: mixw 91 not found
ERROR: "viterbi.c", line 131: mixw 106 not found
ERROR: "viterbi.c", line 131: mixw 106 not found
ERROR: "viterbi.c", line 131: mixw 37 not found
ERROR: "viterbi.c", line 131: mixw 115 not found
ERROR: "viterbi.c", line 131: mixw 94 not found
ERROR: "viterbi.c", line 131: mixw 22 not found
ERROR: "viterbi.c", line 131: mixw 94 not found
14 16 2.681824e-102 -7.125934e+01 -1.083142e+04 utt 0.191x 1.079e upd 0.190x 1.070e fwd 0.097x 1.073e bwd 0.091x 1.073e gau 0.402x 1.219e rsts 0.018x 1.039e rstf 0.008x 0.526e rstu 0.001x 1.065e
utt> 2 2 180 0 92 32 ERROR: "viterbi.c", line 131: mixw 109 not found
ERROR: "viterbi.c", line 131: mixw 49 not found
ERROR: "viterbi.c", line 131: mixw 40 not found
ERROR: "viterbi.c", line 131: mixw 25 not found
ERROR: "viterbi.c", line 131: mixw 118 not found
ERROR: "viterbi.c", line 131: mixw 133 not found
ERROR: "viterbi.c", line 131: mixw 34 not found
ERROR: "viterbi.c", line 131: mixw 115 not found
ERROR: "viterbi.c", line 131: mixw 40 not found
ERROR: "viterbi.c", line 131: mixw 49 not found
ERROR: "viterbi.c", line 131: mixw 40 not found
14 11 4.830422e-102 -7.708644e+01 -1.387556e+04 utt 0.184x 1.089e upd 0.183x 1.081e fwd 0.086x 1.050e bwd 0.097x 1.106e gau 0.491x 1.033e rsts 0.017x 1.306e rstf 0.004x 0.946e rstu 0.001x 1.105e
utt> 3 3 20 0 12 10 ERROR: "viterbi.c", line 131: mixw 34 not found
6 13 0.000000e+00 -7.318526e+01 -1.463705e+03 utt 0.085x 1.297e upd 0.085x 1.082e fwd 0.035x 1.180e bwd 0.045x 1.069e gau 0.140x 0.896e rsts 0.015x 0.494e rstf -0.000x 0.000e rstu 0.005x 0.425e
utt> 4 4 22 0 12 10 ERROR: "viterbi.c", line 131: mixw 130 not found
ERROR: "viterbi.c", line 131: mixw 34 not found
.
.
.
etc..
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi,
I'm using bw for adaptation of a monophone model, and I get errors whenever i use the "-outphsegdir" parameter.
I do not get these same errors when I don't use "-outphsegdir"
The files generated by bw are identical, regardless of whether I output the phsegs.
However, I'm concerned about the presence of the errors, and the fact that some of the phseg outputs contain the "(null)" token. Is this in itself an issue, and could the mixw errors indicate a problem with my model? (both the unadapted and adapted model work ok when decoding)
An example phseg file (phrase is "this is the best study ever"):
SFrm EFrm SegAScr Phone
0 1 0 (null)
2 3 -1402445 (null)
4 5 -5859855 S
6 7 -8610662 IH
8 23 -38319833 Z
24 29 -10311756 (null)
30 41 -1605764 AX
42 44 -7710875 (null)
45 58 -18776771 EH
59 65 -3246114 (null)
66 67 -4414090 (null)
68 81 -52909845 S
82 83 -10132351 (null)
84 89 -10177282 (null)
90 91 -2909178 (null)
92 110 -9040795 IY
111 123 -4868306 EH
124 129 -3076250 V
130 151 -11893777 ER
The error log is below:
INFO: main.c(196): Compiled on Oct 3 2007 at 01:16:10
/homes/stupakov/research/SphinxTrain/bin.i686-pc-linux-gnu//bw \ -moddeffn /homes/stupakov/research/models/am/timit/mdef \ -mixwfn /homes/stupakov/research/models/am/timit/mixture_weights \ -meanfn /homes/stupakov/research/models/am/timit/means \ -varfn /homes/stupakov/research/models/am/timit/variances \ -tmatfn /homes/stupakov/research/models/am/timit/transition_matrices \ -dictfn /homes/stupakov/research/dict/cmudict.0.7a_nolex_timit39 \ -fdictfn /homes/stupakov/research/am_train/adaptation/timit_dict/timit.nosil.filler.dict \ -cepdir / \ -cepext mfcc \ -lsnfn /homes/stupakov/research/am_train/adaptation/cosine/cosine_train_no_OOV.trans \ -meanreest yes \ -varreest no \ -mixwreest no -2passvar no \ -feat 1s_c \ -ceplen 39 \ -ctlfn /homes/stupakov/research/am_train/adaptation/cosine/lists/ctl.lst \ -accumdir /homes/stupakov/research/am_train/adaptation/cosine/cosine_train_no_OOV/accumdir \ -agc none \ -cmn none \ -ts2cbfn .cont. \ -outphsegdir /homes/stupakov/research/corpus/audio/cosine_train_adapt_segs//phseg
[Switch] [Default] [Value]
-help no no
-example no no
-hmmdir
-moddeffn /homes/stupakov/research/models/am/timit/mdef
-tmatfn /homes/stupakov/research/models/am/timit/transition_matrices
-mixwfn /homes/stupakov/research/models/am/timit/mixture_weights
-meanfn /homes/stupakov/research/models/am/timit/means
-varfn /homes/stupakov/research/models/am/timit/variances
-fullvar no no
-diagfull no no
-mwfloor 0.00001 1.000000e-05
-tpfloor 0.0001 1.000000e-04
-varfloor 0.00001 1.000000e-05
-topn 4 4
-dictfn /homes/stupakov/research/dict/cmudict.0.7a_nolex_timit39
-fdictfn /homes/stupakov/research/am_train/adaptation/timit_dict/timit.nosil.filler.dict
-ltsoov no no
-ctlfn /homes/stupakov/research/am_train/adaptation/cosine/lists/ctl.lst
-nskip
-runlen -1 -1
-part
-npart
-cepext mfc mfcc
-cepdir /
-phsegext phseg phseg
-phsegdir
-outphsegdir /homes/stupakov/research/corpus/audio/cosine_train_adapt_segs//phseg
-sentdir
-sentext sent sent
-lsnfn /homes/stupakov/research/am_train/adaptation/cosine/cosine_train_no_OOV.trans
-accumdir /homes/stupakov/research/am_train/adaptation/cosine/cosine_train_no_OOV/accumdir
-ceplen 13 39
-cepwin 0 0
-agc max none
-cmn current none
-varnorm no no
-silcomp none none
-sildel no no
-siltag SIL SIL
-abeam 1e-100 1.000000e-100
-bbeam 1e-100 1.000000e-100
-varreest yes no
-meanreest yes yes
-mixwreest yes no
-tmatreest yes yes
-mllrmat
-cb2mllrfn .1cls. .1cls.
-ts2cbfn .cont.
-feat 1s_c_d_dd 1s_c
-ldafn
-ldadim 29 29
-ldaaccum no no
-timing yes yes
-viterbi no no
-2passvar no no
-sildelfn
-spthresh 0.0 0.000000e+00
-maxuttlen 0 0
-ckptintv
-outputfullpath no no
-fullsuffixmatch no no
-pdumpdir
INFO: main.c(253): Reading /homes/stupakov/research/models/am/timit/mdef
INFO: model_def_io.c(587): Model definition info:
INFO: model_def_io.c(588): 50 total models defined (50 base, 0 tri)
INFO: model_def_io.c(589): 200 total states
INFO: model_def_io.c(590): 150 total tied states
INFO: model_def_io.c(591): 150 total tied CI states
INFO: model_def_io.c(592): 50 total tied transition matrices
INFO: model_def_io.c(593): 4 max state/model
INFO: model_def_io.c(594): 4 min state/model
INFO: s3mixw_io.c(116): Read /homes/stupakov/research/models/am/timit/mixture_weights [150x1x128 array]
INFO: s3tmat_io.c(115): Read /homes/stupakov/research/models/am/timit/transition_matrices [50x3x4 array]
INFO: mod_inv.c(297): inserting tprob floor 1.000000e-04 and renormalizing
INFO: s3gau_io.c(166): Read /homes/stupakov/research/models/am/timit/means [150x1x128 array]
INFO: s3gau_io.c(166): Read /homes/stupakov/research/models/am/timit/variances [150x1x128 array]
INFO: gauden.c(181): 150 total mgau
INFO: gauden.c(155): 1 feature streams (|0|=39 )
INFO: gauden.c(192): 128 total densities
INFO: gauden.c(98): min_var=1.000000e-05
INFO: gauden.c(170): compute 4 densities/frame
INFO: main.c(361): Will NOT reestimate mixing weights.
INFO: main.c(363): Will reestimate means.
INFO: main.c(365): Will NOT reestimate variances.
INFO: main.c(367): WIll NOT optionally delete silence in Baum Welch or Viterbi.
INFO: main.c(375): Will reestimate transition matrices
INFO: main.c(388): Reading main lexicon: /homes/stupakov/research/dict/cmudict.0.7a_nolex_timit39
INFO: lexicon.c(233): 133764 entries added from /homes/stupakov/research/dict/cmudict.0.7a_nolex_timit39
INFO: main.c(400): Reading filler lexicon: /homes/stupakov/research/am_train/adaptation/timit_dict/timit.nosil.filler.dict
INFO: lexicon.c(233): 3 entries added from /homes/stupakov/research/am_train/adaptation/timit_dict/timit.nosil.filler.dict
INFO: main.c(421): Silence Tag SIL
INFO: corpus.c(1343): Will process all remaining utts starting at 0
INFO: main.c(620): Reestimation: Baum-Welch
INFO: main.c(625): Generating profiling information consumes significant CPU resources.
INFO: main.c(626): If you are not interested in profiling, use -timing no
column defns
<seq>
<id>
<n_frame_in>
<n_frame_del>
<n_state_shmm>
<avg_states_alpha>
<avg_states_beta>
<avg_states_reest>
<avg_posterior_prune>
<frame_log_lik>
<utt_log_lik>
... timing info ...
utt> 0 0 54 0INFO: cvt2triphone.c(199): no multiphones defined, no conversion done
20 13 ERROR: "viterbi.c", line 131: mixw 82 not found
9 12 1.087832e-102 -7.057864e+01 -3.811247e+03 utt 0.104x 1.095e upd 0.102x 1.051e fwd 0.039x 1.108e bwd 0.061x 1.009e gau 0.157x 1.236e rsts 0.007x 1.419e rstf 0.004x 0.893e rstu 0.002x 0.935e
utt> 1 1 152 0 76 41 ERROR: "viterbi.c", line 131: mixw 25 not found
ERROR: "viterbi.c", line 131: mixw 91 not found
ERROR: "viterbi.c", line 131: mixw 106 not found
ERROR: "viterbi.c", line 131: mixw 106 not found
ERROR: "viterbi.c", line 131: mixw 37 not found
ERROR: "viterbi.c", line 131: mixw 115 not found
ERROR: "viterbi.c", line 131: mixw 94 not found
ERROR: "viterbi.c", line 131: mixw 22 not found
ERROR: "viterbi.c", line 131: mixw 94 not found
14 16 2.681824e-102 -7.125934e+01 -1.083142e+04 utt 0.191x 1.079e upd 0.190x 1.070e fwd 0.097x 1.073e bwd 0.091x 1.073e gau 0.402x 1.219e rsts 0.018x 1.039e rstf 0.008x 0.526e rstu 0.001x 1.065e
utt> 2 2 180 0 92 32 ERROR: "viterbi.c", line 131: mixw 109 not found
ERROR: "viterbi.c", line 131: mixw 49 not found
ERROR: "viterbi.c", line 131: mixw 40 not found
ERROR: "viterbi.c", line 131: mixw 25 not found
ERROR: "viterbi.c", line 131: mixw 118 not found
ERROR: "viterbi.c", line 131: mixw 133 not found
ERROR: "viterbi.c", line 131: mixw 34 not found
ERROR: "viterbi.c", line 131: mixw 115 not found
ERROR: "viterbi.c", line 131: mixw 40 not found
ERROR: "viterbi.c", line 131: mixw 49 not found
ERROR: "viterbi.c", line 131: mixw 40 not found
14 11 4.830422e-102 -7.708644e+01 -1.387556e+04 utt 0.184x 1.089e upd 0.183x 1.081e fwd 0.086x 1.050e bwd 0.097x 1.106e gau 0.491x 1.033e rsts 0.017x 1.306e rstf 0.004x 0.946e rstu 0.001x 1.105e
utt> 3 3 20 0 12 10 ERROR: "viterbi.c", line 131: mixw 34 not found
6 13 0.000000e+00 -7.318526e+01 -1.463705e+03 utt 0.085x 1.297e upd 0.085x 1.082e fwd 0.035x 1.180e bwd 0.045x 1.069e gau 0.140x 0.896e rsts 0.015x 0.494e rstf -0.000x 0.000e rstu 0.005x 0.425e
utt> 4 4 22 0 12 10 ERROR: "viterbi.c", line 131: mixw 130 not found
ERROR: "viterbi.c", line 131: mixw 34 not found
.
.
.
etc..