I have been using the SphinxTrain documentation (http://www.speech.cs.cmu.edu/sphinxman/scriptman1.html) to train models. After I've created monophone (CI) models, I create a triphone model file, initialize the CD model files (mean, var, mixw, tmat) and re-estimate four times.
After the re-estimation is done, the means, vars and mixture weights of exactly one state have all become "nan" (not a number), according to printp. Does anyone know what could have caused this?
I'd appreciate any help you can give me.
Best regards,
Wout
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That's strange. The state where this problem occurs (there is only one) is a CI state. In the CI part of the training process this state is trained without any problems, so the state must have been observed. After adding the CD models and the first TODO re-estimations the means/vars/mixws are still ok, but during the TODO re-estimation, these values somehow become NaN.
That's strange. The state where this problem occurs (there is only one) is a CI state. In the CI part of the training process this state is trained without any problems, so the state must have been observed. After adding the CD models and the first re-estimation the means/vars/mixws are still ok, but during the second re-estimation, these values somehow become NaN.
Hello,
I have been using the SphinxTrain documentation (http://www.speech.cs.cmu.edu/sphinxman/scriptman1.html) to train models. After I've created monophone (CI) models, I create a triphone model file, initialize the CD model files (mean, var, mixw, tmat) and re-estimate four times.
After the re-estimation is done, the means, vars and mixture weights of exactly one state have all become "nan" (not a number), according to printp. Does anyone know what could have caused this?
I'd appreciate any help you can give me.
Best regards,
Wout
I think this can happen if that state is never observed. This leads to dividing zero by zero in the Baum-Welch re-estimation formulas, and 0/0 = nan.
That's strange. The state where this problem occurs (there is only one) is a CI state. In the CI part of the training process this state is trained without any problems, so the state must have been observed. After adding the CD models and the first TODO re-estimations the means/vars/mixws are still ok, but during the TODO re-estimation, these values somehow become NaN.
I am calling bw with the following parameters:
bw
-moddeffn hmms/mixtures_1/mdef
-mixwfn hmms/mixtures_1/mixw
-tmatfn hmms/mixtures_1/tmat
-meanfn hmms/mixtures_1/mean
-varfn hmms/mixtures_1/var
-ts2cbfn .cont.
-mwfloor 0
-dictfn pronunciation.words.dict
-fdictfn train_dict_filler
-ctlfn training/featurevs/control.ctl
-part 1
-npart 1
-cepext mfc
-lsnfn training/transcriptions
-accumdir /tmp/s3_train_temp_accumdir_reest
-varfloor 0
-topn 1
-abeam 3e-108
-bbeam 3e-108
-agc none
-cmn none -2passvar yes
-feat 1s_12c_12d_3p_12dd
-ceplen 13
-timing no
Could my values for -meanfloor and -mwfloor be the problem? I've set them to 0, instead of the (default) small value.
I checked all logs, but no special warnings are issued concerning this state.
Wout
Oops, I forgot to fill in the TODO's...
That's strange. The state where this problem occurs (there is only one) is a CI state. In the CI part of the training process this state is trained without any problems, so the state must have been observed. After adding the CD models and the first re-estimation the means/vars/mixws are still ok, but during the second re-estimation, these values somehow become NaN.
I am calling bw with the following parameters:
bw
-moddeffn hmms/mixtures_1/mdef
-mixwfn hmms/mixtures_1/mixw
-tmatfn hmms/mixtures_1/tmat
-meanfn hmms/mixtures_1/mean
-varfn hmms/mixtures_1/var
-ts2cbfn .cont.
-mwfloor 0
-dictfn pronunciation.words.dict
-fdictfn train_dict_filler
-ctlfn training/featurevs/control.ctl
-part 1
-npart 1
-cepext mfc
-lsnfn training/transcriptions
-accumdir /tmp/s3_train_temp_accumdir_reest
-varfloor 0
-topn 1
-abeam 3e-108
-bbeam 3e-108
-agc none
-cmn none -2passvar yes
-feat 1s_12c_12d_3p_12dd
-ceplen 13
-timing no
Could my values for -meanfloor and -mwfloor be the problem? I've set them to 0, instead of the (default) small value.
I checked all logs, but no special warnings are issued concerning this state.
Wout