In the last week i try to increase the dimension of the features vector for a custom training of sphinx model.
I operate as below:
In the first phase i generate my feature and i write it on a binary file (.mfc) as done in tutorial example (and in particular in wave2feat file) with the same protocol:
first 4 bytes: number of saved data (M=N*L) where M,N and L are respectively the number of float data save in the file, the number of block of signal analyzed and the length of the features
vector;
- M*(4 bytes): float data saved.
After the first phase in which i have omitted the standard features extraction phase (using make_feat.pl) replaced by my scripts, i generate the sphinx's model.
Before this phase i have modified all parameters refered to the length of the feature vector, i hope all.
Then i execute the training and all model have been created without problems.
The next phase is the decoding test as shown in tutorila. An error occur when i try to decode my testing_features with the model already created.
The error in dos shell is:
perl scripts_pl/decode/slave.pl
MODULE: DECODE Decoding using models previously trained
Decoding 60 segments starting at 0 (part 1 of 1)
0% FATAL_ERROR: "......\src\libs3decoder\libsearch\kbcore.c", line 607:
Feature streamlen(39) != mgau streamlen(126)
This step had 1 ERROR messages and 1 WARNING messages. Please check the log file
for details.
Failed to start bw
Can't open C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/resul
t/dade-1-1.match
SENTENCE ERROR: 100.000% (60/60)
In the log file related to the decoding there are same inconsistencies with the setted parameters.
In particular the features used are 42 not 13.
Also the Vector size of stream[0] is not 39 but 42*3=126.I think.
The question is "why the value of the decoding are set on default values when i set the parameters of the training to the newer ones???"
The log file is this:
INFO: ....\src\libsphinxutil\info.c(70): C:\Users\Davide\Desktop\Speech_recognition\SimpleREC\sphinx\dade\bin\sphinx3_decode.exe Compiled on: Apr 18 2009, AT: 08:44:54
I have simply added these lines in the "scripts/setup_tutorial.pl" in sphinx3 folder:
elsif (lc($task) eq "test") {
$langmodel = " test.ug.lm.DMP";
$langweight = X; // X=length of my features vector
$beam = 1e-120;
$wbeam = 1e-100;
}
Thanks a lot for all answers.
Davide
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Anonymous
-
2010-05-12
Hi, i see that since a year about no body help you, i have tha same problem..
i dont know if you resolved that or not, if you do, you can help me?
thanks
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sorry,
I can add new features for Sphinx Model Training.
In the configuration file where i must modify the parameters.
Hi Davide Riva
> I can add new features for Sphinx Model Training.
Congratulations
> In the configuration file where i must modify the parameters.
In the first line or in the second or even in the third. Don't you think it depends on what you are trying to get?
Sorry for the wrong question.
In the last week i try to increase the dimension of the features vector for a custom training of sphinx model.
I operate as below:
In the first phase i generate my feature and i write it on a binary file (.mfc) as done in tutorial example (and in particular in wave2feat file) with the same protocol:
vector;
- M*(4 bytes): float data saved.
After the first phase in which i have omitted the standard features extraction phase (using make_feat.pl) replaced by my scripts, i generate the sphinx's model.
Before this phase i have modified all parameters refered to the length of the feature vector, i hope all.
Then i execute the training and all model have been created without problems.
The next phase is the decoding test as shown in tutorila. An error occur when i try to decode my testing_features with the model already created.
The error in dos shell is:
perl scripts_pl/decode/slave.pl
MODULE: DECODE Decoding using models previously trained
Decoding 60 segments starting at 0 (part 1 of 1)
0% FATAL_ERROR: "......\src\libs3decoder\libsearch\kbcore.c", line 607:
Feature streamlen(39) != mgau streamlen(126)
This step had 1 ERROR messages and 1 WARNING messages. Please check the log file
for details.
Failed to start bw
Can't open C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/resul
t/dade-1-1.match
SENTENCE ERROR: 100.000% (60/60)
In the log file related to the decoding there are same inconsistencies with the setted parameters.
In particular the features used are 42 not 13.
Also the Vector size of stream[0] is not 39 but 42*3=126.I think.
The question is "why the value of the decoding are set on default values when i set the parameters of the training to the newer ones???"
The log file is this:
INFO: ....\src\libsphinxutil\info.c(70): C:\Users\Davide\Desktop\Speech_recognition\SimpleREC\sphinx\dade\bin\sphinx3_decode.exe Compiled on: Apr 18 2009, AT: 08:44:54
INFO: ....\src\libsphinxutil\cmd_ln.c(430): Parsing command line:
C:\Users\Davide\Desktop\Speech_recognition\SimpleREC\sphinx\dade\bin\sphinx3_decode.exe \
-mdef C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/model_architecture/dade.1000.mdef \
-senmgau .cont. \
-hmm C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/model_parameters/dade.cd_cont_1000 \
-lw 42 \
-feat 1s_c_d_dd \
-beam 1e-120 \
-wbeam 1e-100 \
-dict C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/etc/dade.dic \
-fdict C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/etc/dade.filler \
-lm C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/etc/dade.ug.lm.DMP \
-wip 0.2 \
-ctl C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/etc/dade_test.fileids \
-ctloffset 0 \
-ctlcount 60 \
-cepdir C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/feat \
-cepext .mfc \
-hyp C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/result/dade-1-1.match \
-agc none \
-varnorm no \
-cmn current
Current configuration:
[NAME] [DEFLT] [VALUE]
.
.
.
-ceplen 13 13
.
.
.
-ncep 13 13
.
.
.
-worddumpsf 200000000 200000000
INFO: ......\src\libs3decoder\libsearch\kbcore.c(404): Begin Initialization of Core Models:
INFO: ....\src\libsphinxutil\cmd_ln.c(430): Parsing command line:
\
-alpha 0.97 \
-dither yes \
-doublebw no \
-nfilt 40 \
-ncep 42 \
-lowerf 133.33334 \
-upperf 6855.4976 \
-nfft 512 \
-wlen 0.0256 \
-transform legacy \
-feat 1s_c_d_dd \
-agc none \
-cmn current \
-varnorm no
Current configuration:
[NAME] [DEFLT] [VALUE]
-agc none none
-alpha 0.97 9.700000e-001
-cep2spec no no
-ceplen 13 13
.
.
.
-ncep 13 42
.
.
.
-wlen 0.025625 2.560000e-002
INFO: ......\src\libs3decoder\libsearch\kbcore.c(422): Parsed model-specific feature parameters from C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/model_parameters/dade.cd_cont_1000/feat.params
INFO: ......\src\libs3decoder\libcommon\logs3.c(151): Initializing logbase: 1.000300e+000 (add table: 1)
INFO: Initialization of the log add table
INFO: Log-Add table size = 29350
INFO:
INFO: ....\src\libsphinxfeat\feat.c(835): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none'
INFO: ....\src\libsphinxfeat\cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0
INFO: ......\src\libs3decoder\libsearch\kbcore.c(446): .cont.
INFO: Initialization of feat_t, report:
INFO: Feature type = 1s_c_d_dd
INFO: Cepstral size = 13
INFO: Cepstral size Used = 13
.
INFO: Vector size of stream[0]: 39
INFO: Whether CMN is used = 1
INFO: Whether AGC is used = 0
INFO: Whether variance is normalized = 0
INFO:
INFO: Reading HMM in Sphinx 3 Model format
.
.
.
.
INFO: Read 18 transition matrices of size 3x4
INFO:
INFO: ......\src\libs3decoder\libdict\dict.c(471): Reading main dictionary: C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/etc/dade.dic
INFO: ......\src\libs3decoder\libdict\dict.c(474): 29 words read
INFO: ......\src\libs3decoder\libdict\dict.c(479): Reading filler dictionary: C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/etc/dade.filler
INFO: ......\src\libs3decoder\libdict\dict.c(482): 3 words read
INFO: Initialization of dict_t, report:
INFO: No of CI phone: 0
INFO: Max word: 4128
INFO: No of word: 32
INFO:
INFO: ......\src\libs3decoder\liblm\lm.c(593): LM read('C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/etc/dade.ug.lm.DMP', lw= 42.00, wip= 0.20, uw= 0.70)
INFO: ......\src\libs3decoder\liblm\lm.c(595): Reading LM file C:/Users/Davide/Desktop/Speech_recognition/SimpleREC/sphinx/dade/etc/dade.ug.lm.DMP (LM name "default")
INFO: ......\src\libs3decoder\liblm\lm_3g_dmp.c(628): Reading LM in 16 bits format
INFO: ......\src\libs3decoder\liblm\lm_3g_dmp.c(684): Read 27 unigrams [in memory]
INFO: ......\src\libs3decoder\liblm\lm_3g_dmp.c(757): 1 bigrams [on disk]
INFO: ......\src\libs3decoder\liblm\lm_3g_dmp.c(900): 2 bigram prob entries
INFO: ......\src\libs3decoder\liblm\lm_3g_dmp.c(1051): 27 word strings
INFO: ......\src\libs3decoder\liblm\lm.c(686): The LM routine is operating at 16 bits mode
ERROR: "......\src\libs3decoder\libdict\wid.c", line 282: <UNK> is not a word in dictionary and it is not a class tag.
INFO: ......\src\libs3decoder\libdict\wid.c(292): 1 LM words not in dictionary; ignored
INFO: Initialization of fillpen_t, report:
INFO: Language weight =42.000000
INFO: Word Insertion Penalty =0.200000
INFO: Silence probability =0.100000
INFO: Filler probability =0.100000
INFO:
INFO: ......\src\libs3decoder\libsearch\dict2pid.c(577): Building PID tables for dictionary
INFO: Initialization of dict2pid_t, report:
INFO: Dict2pid is in composite triphone mode
INFO: 122 composite states; 46 composite sseq
INFO:
INFO: ......\src\libs3decoder\libsearch\kbcore.c(602): Inside kbcore: Verifying models consistency ......
FATAL_ERROR: "......\src\libs3decoder\libsearch\kbcore.c", line 607: Feature streamlen(39) != mgau streamlen(126)
Wed May 20 13:10:43 2009
The question is "why the value of the decoding are set on default values when i set the parameters of the training to the newer ones???"
You forgot to add -ceplen 42 to feat.params.
yesterday afternoon i have resolved the problems.
I have simply added these lines in the "scripts/setup_tutorial.pl" in sphinx3 folder:
elsif (lc($task) eq "test") {
$langmodel = " test.ug.lm.DMP";
$langweight = X; // X=length of my features vector
$beam = 1e-120;
$wbeam = 1e-100;
}
Thanks a lot for all answers.
Davide
Hi, i see that since a year about no body help you, i have tha same problem..
i dont know if you resolved that or not, if you do, you can help me?
thanks