Hi
I have trained acoustic model for hindi language by folowing steps provided on CMU Sphinx wbsite and the training part of my model is completed but when decoding the model then receiving this error "word_align.pl failed with error code 65280 at /usr/local/lib/sphinxtrain/scripts/decode/slave.pl line 173.".
I have tried to search for the solution and found that if I am not training cd model then i need to change some values in .cfg file
$DEC_CFG_MODEL_NAME = "$CFG_EXPTNAME.ci_cont";
but this is not helpful for me. I am still getting that error and I have also tried to train CD model with CI model without changing any value in .cfg file, but then also Error prevails.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi. I had a look at your logs. They seem almost fine. If you check decode/hindi-1-1.log, you will see the recognized hypotheses for each test utterance.
There seems to be a problem at the very last stage, where the system computes word error rate (by aligning hypotheses and references).
You can check the recognizer output and reference manually. It seems there is a problem with encoding of some characters. But to make sure it is the case, we would need the whole experiment folder to reproduce your errors
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you Sir.
You mean that this error has nothing to do with my model performance.
If it is so, then my model is working fine but the accuracy is still very low, So how can I increase it's accuracy?
Last edit: Prince Dhanwan 2017-01-17
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'd start with geting more data to train context-dependent models.
Of course, you can also adapt some other acoustic model, improve language model, do transfer learning from another language, train neural networks. There is the universe of things that can be done. But the simplest is get more data and train CD models.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The same problem occurred with mine also. Whether the error occur due to model or with the decoder. I think that this error occurs due to decoder and proceed with using the models to do forced alignment. Is this would effect forced alignment result. Because I am getting large variation in the boundary. Please help me. Thank you.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hey there!
Hello everybody!
I am trying to build acoustic model using jsgf grammer, but I am havaing the same problem, the last line of training log is giving the same error, while the decoder log file doesn't show any error
I am working on Mint but when I sent my model to my friend everything worked fine under windows.
The model gives better results on Windows rather than Linux.
Any deas please ?
MODULE: 20 Training Context Independent models
Phase 1: Cleaning up directories:
accumulator...logs...qmanager...models...
Phase 2: Flat initialize
Phase 3: Forward-Backward
Baum welch starting for 1 Gaussian(s), iteration: 1 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 1
Current Overall Likelihood Per Frame = -157.15196743555
Baum welch starting for 1 Gaussian(s), iteration: 2 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 2
Current Overall Likelihood Per Frame = -152.781129318443
Convergence Ratio = 4.37083811710724
Baum welch starting for 1 Gaussian(s), iteration: 3 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 3
Current Overall Likelihood Per Frame = -149.734787600459
Convergence Ratio = 3.04634171798375
Baum welch starting for 1 Gaussian(s), iteration: 4 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 4
Current Overall Likelihood Per Frame = -148.820791149149
Convergence Ratio = 0.913996451309629
Baum welch starting for 1 Gaussian(s), iteration: 5 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 5
Current Overall Likelihood Per Frame = -148.607452249243
Convergence Ratio = 0.213338899905693
Baum welch starting for 1 Gaussian(s), iteration: 6 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 6
Current Overall Likelihood Per Frame = -148.502870264064
Convergence Ratio = 0.104581985178726
Baum welch starting for 1 Gaussian(s), iteration: 7 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 7
Current Overall Likelihood Per Frame = -148.440141947605
Training completed after 7 iterations
Skipped (set $CFG_CD_TRAIN = 'yes' to enable)
Skipped (set $CFG_CD_TRAIN = 'yes' to enable)
Skipped (set $CFG_CD_TRAIN = 'yes' to enable)
Skipped (set $CFG_CD_TRAIN = 'yes' to enable)
MODULE: 60 Lattice Generation
Skipped: $ST::CFG_MMIE set to 'no' in sphinx_train.cfg
MODULE: 61 Lattice Pruning
Skipped: $ST::CFG_MMIE set to 'no' in sphinx_train.cfg
MODULE: 62 Lattice Format Conversion
Skipped: $ST::CFG_MMIE set to 'no' in sphinx_train.cfg
MODULE: 65 MMIE Training
Skipped: $ST::CFG_MMIE set to 'no' in sphinx_train.cfg
MODULE: 90 deleted interpolation
Skipped for continuous models
MODULE: DECODE Decoding using models previously trained
Decoding 0 segments starting at 0 (part 1 of 1)
Aligning results to find error rate
word_align.pl failed with error code 65280 at /usr/local/lib/sphinxtrain/scripts/decode/slave.pl line 173.
Last edit: Bassel Zaity 2017-12-27
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hello
I'm also I have some problem please help me .
MODULE: DECODE Decoding using models previously trained
Decoding 0 segments starting at 0 (part 1 of 1)
Aligning results to find error rate
word_align.pl failed with error code 65280 at C:\ProjectSphinx\sphinxtrain\scripts\decode\slave.pl line 173.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi
I have trained acoustic model for hindi language by folowing steps provided on CMU Sphinx wbsite and the training part of my model is completed but when decoding the model then receiving this error "word_align.pl failed with error code 65280 at /usr/local/lib/sphinxtrain/scripts/decode/slave.pl line 173.".
I have tried to search for the solution and found that if I am not training cd model then i need to change some values in .cfg file
$DEC_CFG_MODEL_NAME = "$CFG_EXPTNAME.ci_cont";
but this is not helpful for me. I am still getting that error and I have also tried to train CD model with CI model without changing any value in .cfg file, but then also Error prevails.
You can check logdir for details. You can share your model folder to get further help.
Sir here is the the decode log, html file which is created and cfg file
Last edit: Prince Dhanwan 2017-01-12
Waiting for your reply @Nickolay V. Shmyrev.
Or is there any other file you need??
Last edit: Prince Dhanwan 2017-01-14
You can share your model folder to get further help.
Sir I have shared the model here in my next post
Here is the model link
https://www.dropbox.com/sh/71hwz8h0ccs0nho/AAAwJpifT9_oWBRp2owPTGruag?dl=0
Last edit: Prince Dhanwan 2017-01-19
Hi. I had a look at your logs. They seem almost fine. If you check decode/hindi-1-1.log, you will see the recognized hypotheses for each test utterance.
There seems to be a problem at the very last stage, where the system computes word error rate (by aligning hypotheses and references).
You can check the recognizer output and reference manually. It seems there is a problem with encoding of some characters. But to make sure it is the case, we would need the whole experiment folder to reproduce your errors
Thank you Sir.
You mean that this error has nothing to do with my model performance.
If it is so, then my model is working fine but the accuracy is still very low, So how can I increase it's accuracy?
Last edit: Prince Dhanwan 2017-01-17
I'd start with geting more data to train context-dependent models.
Of course, you can also adapt some other acoustic model, improve language model, do transfer learning from another language, train neural networks. There is the universe of things that can be done. But the simplest is get more data and train CD models.
Sir, with more data are we going to make more seperate CD models OR We can incresase the accuracy of the old model made by my previous data?
How?
Last edit: Prince Dhanwan 2017-01-17
check tutorial. with more data you can train more senones.
The same problem occurred with mine also. Whether the error occur due to model or with the decoder. I think that this error occurs due to decoder and proceed with using the models to do forced alignment. Is this would effect forced alignment result. Because I am getting large variation in the boundary. Please help me. Thank you.
Last edit: Prince Dhanwan 2017-01-17
Hey there!
Hello everybody!
I am trying to build acoustic model using jsgf grammer, but I am havaing the same problem, the last line of training log is giving the same error, while the decoder log file doesn't show any error
I am working on Mint but when I sent my model to my friend everything worked fine under windows.
The model gives better results on Windows rather than Linux.
Any deas please ?
MODULE: 20 Training Context Independent models
Phase 1: Cleaning up directories:
accumulator...logs...qmanager...models...
Phase 2: Flat initialize
Phase 3: Forward-Backward
Baum welch starting for 1 Gaussian(s), iteration: 1 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 1
Current Overall Likelihood Per Frame = -157.15196743555
Baum welch starting for 1 Gaussian(s), iteration: 2 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 2
Current Overall Likelihood Per Frame = -152.781129318443
Convergence Ratio = 4.37083811710724
Baum welch starting for 1 Gaussian(s), iteration: 3 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 3
Current Overall Likelihood Per Frame = -149.734787600459
Convergence Ratio = 3.04634171798375
Baum welch starting for 1 Gaussian(s), iteration: 4 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 4
Current Overall Likelihood Per Frame = -148.820791149149
Convergence Ratio = 0.913996451309629
Baum welch starting for 1 Gaussian(s), iteration: 5 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 5
Current Overall Likelihood Per Frame = -148.607452249243
Convergence Ratio = 0.213338899905693
Baum welch starting for 1 Gaussian(s), iteration: 6 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 6
Current Overall Likelihood Per Frame = -148.502870264064
Convergence Ratio = 0.104581985178726
Baum welch starting for 1 Gaussian(s), iteration: 7 (1 of 1)
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Normalization for iteration: 7
Current Overall Likelihood Per Frame = -148.440141947605
Training completed after 7 iterations
Skipped (set $CFG_CD_TRAIN = 'yes' to enable)
Skipped (set $CFG_CD_TRAIN = 'yes' to enable)
Skipped (set $CFG_CD_TRAIN = 'yes' to enable)
Skipped (set $CFG_CD_TRAIN = 'yes' to enable)
MODULE: 60 Lattice Generation
Skipped: $ST::CFG_MMIE set to 'no' in sphinx_train.cfg
MODULE: 61 Lattice Pruning
Skipped: $ST::CFG_MMIE set to 'no' in sphinx_train.cfg
MODULE: 62 Lattice Format Conversion
Skipped: $ST::CFG_MMIE set to 'no' in sphinx_train.cfg
MODULE: 65 MMIE Training
Skipped: $ST::CFG_MMIE set to 'no' in sphinx_train.cfg
MODULE: 90 deleted interpolation
Skipped for continuous models
MODULE: DECODE Decoding using models previously trained
Decoding 0 segments starting at 0 (part 1 of 1)
Aligning results to find error rate
word_align.pl failed with error code 65280 at /usr/local/lib/sphinxtrain/scripts/decode/slave.pl line 173.
Last edit: Bassel Zaity 2017-12-27
Hello
I'm also I have some problem please help me .
MODULE: DECODE Decoding using models previously trained
Decoding 0 segments starting at 0 (part 1 of 1)
Aligning results to find error rate
word_align.pl failed with error code 65280 at C:\ProjectSphinx\sphinxtrain\scripts\decode\slave.pl line 173.
Hello
I'm also I have some problem please help me .
MODULE: DECODE Decoding using models previously trained
Decoding 0 segments starting at 0 (part 1 of 1)
Aligning results to find error rate
word_align.pl failed with error code 65280 at C:\ProjectSphinx\sphinxtrain\scripts\decode\slave.pl line 173.
log file
INFO: pocketsphinx.c(153): Parsed model-specific feature parameters from C:/ProjectSphinx/B/model_parameters/B.ci_cont/feat.params
Current configuration:
[NAME] [DEFLT] [VALUE]
-agc none none
-agcthresh 2.0 2.000000e+00
-allphone
-allphone_ci yes yes
-alpha 0.97 9.700000e-01
-ascale 20.0 2.000000e+01
-aw 1 1
-backtrace no no
-beam 1e-48 1.000000e-80
-bestpath yes yes
-bestpathlw 9.5 1.000000e+01
-ceplen 13 13
-cmn live batch
-cmninit 40,3,-1 40,3,-1
-compallsen no no
-dict C:/ProjectSphinx/B/etc/B.dic
-dictcase no no
-dither no no
-doublebw no no
-ds 1 1
-fdict
-feat 1s_c_d_dd 1s_c_d_dd
-featparams
-fillprob 1e-8 1.000000e-08
-frate 100 100
-fsg
-fsgusealtpron yes yes
-fsgusefiller yes yes
-fwdflat yes yes
-fwdflatbeam 1e-64 1.000000e-80
-fwdflatefwid 4 4
-fwdflatlw 8.5 1.000000e+01
-fwdflatsfwin 25 25
-fwdflatwbeam 7e-29 1.000000e-40
-fwdtree yes yes
-hmm C:/ProjectSphinx/B/model_parameters/B.ci_cont
-input_endian little little
-jsgf
-keyphrase
-kws
-kws_delay 10 10
-kws_plp 1e-1 1.000000e-01
-kws_threshold 1e-30 1.000000e-30
-latsize 5000 5000
-lda
-ldadim 0 0
-lifter 0 22
-lm C:/ProjectSphinx/B/etc/B.lm.DMP
-lmctl
-lmname
-logbase 1.0001 1.000100e+00
-logfn
-logspec no no
-lowerf 133.33334 1.300000e+02
-lpbeam 1e-40 1.000000e-80
-lponlybeam 7e-29 1.000000e-80
-lw 6.5 1.000000e+01
-maxhmmpf 30000 30000
-maxwpf -1 -1
-mdef
-mean
-mfclogdir
-min_endfr 0 0
-mixw
-mixwfloor 0.0000001 1.000000e-07
-mllr
-mmap yes yes
-ncep 13 13
-nfft 512 512
-nfilt 40 25
-nwpen 1.0 1.000000e+00
-pbeam 1e-48 1.000000e-80
-pip 1.0 1.000000e+00
-pl_beam 1e-10 1.000000e-10
-pl_pbeam 1e-10 1.000000e-10
-pl_pip 1.0 1.000000e+00
-pl_weight 3.0 3.000000e+00
-pl_window 5 5
-rawlogdir
-remove_dc no no
-remove_noise yes yes
-remove_silence yes yes
-round_filters yes yes
-samprate 16000 1.600000e+04
-seed -1 -1
-sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03
-smoothspec no no
-svspec
-tmat
-tmatfloor 0.0001 1.000000e-04
-topn 4 4
-topn_beam 0 0
-toprule
-transform legacy dct
-unit_area yes yes
-upperf 6855.4976 6.800000e+03
-uw 1.0 1.000000e+00
-vad_postspeech 50 50
-vad_prespeech 20 20
-vad_startspeech 10 10
-vad_threshold 3.0 3.000000e+00
-var
-varfloor 0.0001 1.000000e-04
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wbeam 7e-29 1.000000e-40
-wip 0.65 2.000000e-01
-wlen 0.025625 2.562500e-02
INFO: feat.c(715): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='batch', VARNORM='no', AGC='none'
INFO: mdef.c(518): Reading model definition: C:/ProjectSphinx/B/model_parameters/B.ci_cont/mdef
INFO: bin_mdef.c(181): Allocating 100 * 8 bytes (0 KiB) for CD tree
INFO: tmat.c(149): Reading HMM transition probability matrices: C:/ProjectSphinx/B/model_parameters/B.ci_cont/transition_matrices
INFO: acmod.c(113): Attempting to use PTM computation module
INFO: ms_gauden.c(127): Reading mixture gaussian parameter: C:/ProjectSphinx/B/model_parameters/B.ci_cont/means
INFO: ms_gauden.c(242): 72 codebook, 1 feature, size:
INFO: ms_gauden.c(244): 1x39
INFO: ms_gauden.c(127): Reading mixture gaussian parameter: C:/ProjectSphinx/B/model_parameters/B.ci_cont/variances
INFO: ms_gauden.c(242): 72 codebook, 1 feature, size:
INFO: ms_gauden.c(244): 1x39
INFO: ms_gauden.c(304): 0 variance values floored
INFO: ptm_mgau.c(807): Number of codebooks doesn't match number of ciphones, doesn't look like PTM: 72 != 24
INFO: acmod.c(115): Attempting to use semi-continuous computation module
INFO: ms_gauden.c(127): Reading mixture gaussian parameter: C:/ProjectSphinx/B/model_parameters/B.ci_cont/means
INFO: ms_gauden.c(242): 72 codebook, 1 feature, size:
INFO: ms_gauden.c(244): 1x39
INFO: ms_gauden.c(127): Reading mixture gaussian parameter: C:/ProjectSphinx/B/model_parameters/B.ci_cont/variances
INFO: ms_gauden.c(242): 72 codebook, 1 feature, size:
INFO: ms_gauden.c(244): 1x39
INFO: ms_gauden.c(304): 0 variance values floored
INFO: acmod.c(117): Falling back to general multi-stream GMM computation
INFO: ms_gauden.c(127): Reading mixture gaussian parameter: C:/ProjectSphinx/B/model_parameters/B.ci_cont/means
INFO: ms_gauden.c(242): 72 codebook, 1 feature, size:
INFO: ms_gauden.c(244): 1x39
INFO: ms_gauden.c(127): Reading mixture gaussian parameter: C:/ProjectSphinx/B/model_parameters/B.ci_cont/variances
INFO: ms_gauden.c(242): 72 codebook, 1 feature, size:
INFO: ms_gauden.c(244): 1x39
INFO: ms_gauden.c(304): 0 variance values floored
INFO: ms_senone.c(149): Reading senone mixture weights: C:/ProjectSphinx/B/model_parameters/B.ci_cont/mixture_weights
INFO: ms_senone.c(200): Truncating senone logs3(pdf) values by 10 bits
INFO: ms_senone.c(207): Not transposing mixture weights in memory
INFO: ms_senone.c(268): Read mixture weights for 72 senones: 1 features x 1 codewords
INFO: ms_senone.c(320): Mapping senones to individual codebooks
INFO: ms_mgau.c(144): The value of topn: 4
WARN: "ms_mgau.c", line 148: -topn argument (4) invalid or > #density codewords (1); set to latter
INFO: phone_loop_search.c(114): State beam -225 Phone exit beam -225 Insertion penalty 0
INFO: dict.c(320): Allocating 4109 * 20 bytes (80 KiB) for word entries
INFO: dict.c(333): Reading main dictionary: C:/ProjectSphinx/B/etc/B.dic
INFO: dict.c(213): Dictionary size 10, allocated 0 KiB for strings, 0 KiB for phones
INFO: dict.c(336): 10 words read
INFO: dict.c(358): Reading filler dictionary: C:/ProjectSphinx/B/model_parameters/B.ci_cont/noisedict
INFO: dict.c(213): Dictionary size 13, allocated 0 KiB for strings, 0 KiB for phones
INFO: dict.c(361): 3 words read
INFO: dict2pid.c(396): Building PID tables for dictionary
INFO: dict2pid.c(406): Allocating 24^3 * 2 bytes (27 KiB) for word-initial triphones
INFO: dict2pid.c(132): Allocated 7008 bytes (6 KiB) for word-final triphones
INFO: dict2pid.c(196): Allocated 7008 bytes (6 KiB) for single-phone word triphones
INFO: ngram_model_trie.c(354): Trying to read LM in trie binary format
INFO: ngram_model_trie.c(365): Header doesn't match
INFO: ngram_model_trie.c(177): Trying to read LM in arpa format
INFO: ngram_model_trie.c(193): LM of order 3
INFO: ngram_model_trie.c(195): #1-grams: 12
INFO: ngram_model_trie.c(195): #2-grams: 20
INFO: ngram_model_trie.c(195): #3-grams: 10
INFO: lm_trie.c(474): Training quantizer
INFO: lm_trie.c(482): Building LM trie
INFO: ngram_search_fwdtree.c(74): Initializing search tree
INFO: ngram_search_fwdtree.c(101): 9 unique initial diphones
INFO: ngram_search_fwdtree.c(186): Creating search channels
INFO: ngram_search_fwdtree.c(323): Max nonroot chan increased to 128
INFO: ngram_search_fwdtree.c(333): Created 0 root, 0 non-root channels, 3 single-phone words
INFO: ngram_search_fwdflat.c(157): fwdflat: min_ef_width = 4, max_sf_win = 25
INFO: batch.c(867): TOTAL 0.00 seconds speech, 0.00 seconds CPU, 0.00 seconds wall
INFO: batch.c(869): AVERAGE -nan(ind) xRT (CPU), -nan(ind) xRT (elapsed)
INFO: ngram_search_fwdtree.c(429): TOTAL fwdtree 0.00 CPU -nan(ind) xRT
INFO: ngram_search_fwdtree.c(432): TOTAL fwdtree 0.00 wall -nan(ind) xRT
INFO: ngram_search_fwdflat.c(176): TOTAL fwdflat 0.00 CPU -nan(ind) xRT
INFO: ngram_search_fwdflat.c(179): TOTAL fwdflat 0.00 wall -nan(ind) xRT
INFO: ngram_search.c(303): TOTAL bestpath 0.00 CPU -nan(ind) xRT
INFO: ngram_search.c(306): TOTAL bestpath 0.00 wall -nan(ind) xRT
Sun May 17 13:09:20 2020