I'm training cont. model with SphinxTrain, and stucked at CI training step. The symptom is:
The first iteration is going well. However in the 2nd iteration, there are a large number of "final state not reached" errors shown in the log. Almost 80% of utterances were ignored due to this error.
Evandro suggested me to decrease the abeam and bbeam. I reduce those values from 1e-90 and 1e-40 to both 1e-180. This doesn't seem help. I still have the same amount of errors.
Before I start debugging the bw program, I post here to see if anyone else has observed this before.
BTW,
I am using mfcc created by wave2feat. The same mfcc files have been used for training semi. model successfully. Also, the verification step doesn't show any error except that there is a filler word never used in the utterances.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am currently trying to do some speaker adaption with the wsj_8khz model like it is described in http://www-2.cs.cmu.edu/~archan/documentation/chapter9.ps
When i am using the bw 4 of 16 sentences were ignored because of "final state not reached".
Is this a normal behavior or did i use the false parameters?
My parameters for wave2feat are:
wave2feat -c ctrfile -nfilt 31 -lowerf 200 -upperf 3500 -srate 8000 -raw yes [...]
bw:
bw
-moddeffn etc/WSJ_8gau_13dCep_8kHz_31mel_200Hz_3500Hz.4000.mdef
-mixwfn cd_continuous_8gau/mixture_weights
-meanfn cd_continuous_8gau/means
-varfn cd_continuous_8gau/variances
-tmatfn cd_continuous_8gau/transition_matrices
-dictfn dict/cmudict.0.6d
-fdictfn dict/fillerdict
-cepdir mfc -cepext mfc -lsnfn transcript
-feat 1s_c_d_dd -ctlfn ctrfile -accumdir means
-ts2cbfn .cont.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Double checked. Doesn't seem the case of mismatch.
Actually both .fileids and .transcription fiiles are copied from an existing successful project.
Will take a closer look.
Here are some other observations:
1. I use a portion of training data to train a digit only model. Everything looks fine. The differences between the digit-only data and whole data set are:
7hrs vs. 80 hrs.
28 CI phones vs. 50 CI phones.
In the bw log files that contains error messages, the messages look like:
ERROR: "baum_welch.c", line 300: 0/ad_17_p14 ignored
I'm training cont. model with SphinxTrain, and stucked at CI training step. The symptom is:
The first iteration is going well. However in the 2nd iteration, there are a large number of "final state not reached" errors shown in the log. Almost 80% of utterances were ignored due to this error.
Evandro suggested me to decrease the abeam and bbeam. I reduce those values from 1e-90 and 1e-40 to both 1e-180. This doesn't seem help. I still have the same amount of errors.
Before I start debugging the bw program, I post here to see if anyone else has observed this before.
BTW,
I am using mfcc created by wave2feat. The same mfcc files have been used for training semi. model successfully. Also, the verification step doesn't show any error except that there is a filler word never used in the utterances.
I am currently trying to do some speaker adaption with the wsj_8khz model like it is described in
http://www-2.cs.cmu.edu/~archan/documentation/chapter9.ps
When i am using the bw 4 of 16 sentences were ignored because of "final state not reached".
Is this a normal behavior or did i use the false parameters?
My parameters for wave2feat are:
wave2feat -c ctrfile -nfilt 31 -lowerf 200 -upperf 3500 -srate 8000 -raw yes [...]
bw:
bw
-moddeffn etc/WSJ_8gau_13dCep_8kHz_31mel_200Hz_3500Hz.4000.mdef
-mixwfn cd_continuous_8gau/mixture_weights
-meanfn cd_continuous_8gau/means
-varfn cd_continuous_8gau/variances
-tmatfn cd_continuous_8gau/transition_matrices
-dictfn dict/cmudict.0.6d
-fdictfn dict/fillerdict
-cepdir mfc -cepext mfc -lsnfn transcript
-feat 1s_c_d_dd -ctlfn ctrfile -accumdir means
-ts2cbfn .cont.
Check the transcription.
Likely, it is an off-by-one problem. Or some major transcription and control file mismatch.
Arthur
Double checked. Doesn't seem the case of mismatch.
Actually both .fileids and .transcription fiiles are copied from an existing successful project.
Will take a closer look.
Here are some other observations:
1. I use a portion of training data to train a digit only model. Everything looks fine. The differences between the digit-only data and whole data set are:
7hrs vs. 80 hrs.
28 CI phones vs. 50 CI phones.
ERROR: "baum_welch.c", line 300: 0/ad_17_p14 ignored
utt 0.014x 1.025e upd 0.014x 1.007e fwd 0.014x 1.002e bwd -0.000x 0.000e gau 0.007x 0.963e rsts 0.000x 0.000e rstf 0.000x 0.000e rstu 0.000x 0.000e
so, what do those number mean?
Copied from successful project.. That's strange.
Feature mismatch?