Non english LM is built, what is next ?

[Anonymous]

Hi, I have succesfully train a new non-english LM, what should I do next ?

Error has occur as below:

sphinx3-simple:
Demo CMU Sphinx-3 decoder called with command line arguments.

<executing /usr/local/bin/livedecode, please wait>
INFO: cmd_ln.c(277): Parsing command line:
\ -mdef /usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/Malay.6000.mdef \ -fdict /usr/local/share/sphinx3/model/lm/an4/Malay.filler \ -dict /usr/local/share/sphinx3/model/lm/an4/Malay.dic \ -mean /usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/means \ -var /usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/variances \ -mixw /usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/mixture_weights \ -tmat /usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/transition_matrices \ -subvq /usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/8gau.6000sen.quant \ -upperf 6855.49756 \ -lowerf 133.33334 \ -nfilt 40 \ -feat 1s_c_d_dd \ -samprate 16000 \ -agc none \ -varnorm no \ -cmn current \ -subvqbeam 1e-02 \ -epl 4 \ -fillprob 0.02 \ -lw 9.5 \ -maxwpf 10 \ -beam 1e-60 \ -wbeam 1e-35 \ -lm /usr/local/share/sphinx3/model/lm/an4/Malay.arpa.DMP

Configuration in effect:
[NAME] [DEFLT] [VALUE]
-agc max In arg dump, the argument name is -agc
none
-alpha 0.97 In arg dump, the argument name is -alpha
9.700000e-01
-beam 1.0e-55 In arg dump, the argument name is -beam
1.000000e-60
-bghist 0 In arg dump, the argument name is -bghist
0
-bptbldir In arg dump, the argument name is -bptbldir

-cepdir In arg dump, the argument name is -cepdir

-ci_pbeam 1e-80 In arg dump, the argument name is -ci_pbeam
0.000000e+00
-cmn current In arg dump, the argument name is -cmn
current
-cond_ds 0 In arg dump, the argument name is -cond_ds
0
-ctl In arg dump, the argument name is -ctl

-ctlcount 1000000000 In arg dump, the argument name is -ctlcount
1000000000
-ctloffset 0 In arg dump, the argument name is -ctloffset
0
-ctl_lm In arg dump, the argument name is -ctl_lm

-dict In arg dump, the argument name is -dict
/usr/local/share/sphinx3/model/lm/an4/Malay.dic
-ds 1 In arg dump, the argument name is -ds
1
-epl 3 In arg dump, the argument name is -epl
4
-fdict In arg dump, the argument name is -fdict
/usr/local/share/sphinx3/model/lm/an4/Malay.filler
-feat In arg dump, the argument name is -feat
1s_c_d_dd
-fillpen In arg dump, the argument name is -fillpen

-fillprob 0.1 In arg dump, the argument name is -fillprob
2.000000e-02
-frate 100 In arg dump, the argument name is -frate
100
-gs In arg dump, the argument name is -gs

-gs4gs 1 In arg dump, the argument name is -gs4gs
1
-hmmdump 0 In arg dump, the argument name is -hmmdump
0
-hmmhistbinsize 5000 In arg dump, the argument name is -hmmhistbinsize
5000
-hyp In arg dump, the argument name is -hyp

-hypseg In arg dump, the argument name is -hypseg

-latext lat.gz In arg dump, the argument name is -latext
lat.gz
-lextreedump 0 In arg dump, the argument name is -lextreedump
0
-lm In arg dump, the argument name is -lm
/usr/local/share/sphinx3/model/lm/an4/Malay.arpa.DMP
-lmctlfn In arg dump, the argument name is -lmctlfn

-lmdumpdir In arg dump, the argument name is -lmdumpdir

-lminmemory 0 In arg dump, the argument name is -lminmemory
0
-log3table 1 In arg dump, the argument name is -log3table
1
-logbase 1.0003 In arg dump, the argument name is -logbase
1.000300e+00
-lowerf 200 In arg dump, the argument name is -lowerf
1.333333e+02
-lw 8.5 In arg dump, the argument name is -lw
9.500000e+00
-maxcepvecs 256 In arg dump, the argument name is -maxcepvecs
256
-maxhistpf 100 In arg dump, the argument name is -maxhistpf
100
-maxhmmpf 20000 In arg dump, the argument name is -maxhmmpf
20000
-maxhyplen 1000 In arg dump, the argument name is -maxhyplen
1000
-maxwpf 20 In arg dump, the argument name is -maxwpf
10
-mdef In arg dump, the argument name is -mdef
/usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/Malay.6000.mdef
-mean In arg dump, the argument name is -mean
/usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/means
-mixw In arg dump, the argument name is -mixw
/usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/mixture_weights
-mixwfloor 0.0000001 In arg dump, the argument name is -mixwfloor
1.000000e-07
-nfft 256 In arg dump, the argument name is -nfft
256
-nfilt 31 In arg dump, the argument name is -nfilt
40
-Nlextree 3 In arg dump, the argument name is -Nlextree
3
-outlatdir In arg dump, the argument name is -outlatdir

-outlatoldfmt 1 In arg dump, the argument name is -outlatoldfmt
1
-pbeam 1.0e-50 In arg dump, the argument name is -pbeam
1.000000e-50
-pl_beam 1.0e-80 In arg dump, the argument name is -pl_beam
0.000000e+00
-pl_window 1 In arg dump, the argument name is -pl_window
1
-ptranskip 0 In arg dump, the argument name is -ptranskip
0
-samprate 8000 In arg dump, the argument name is -samprate
16000
-senmgau .cont. In arg dump, the argument name is -senmgau
.cont.
-silprob 0.1 In arg dump, the argument name is -silprob
1.000000e-01
-subvq In arg dump, the argument name is -subvq
/usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/8gau.6000sen.quant
-subvqbeam 3.0e-3 In arg dump, the argument name is -subvqbeam
1.000000e-02
-svq4svq 0 In arg dump, the argument name is -svq4svq
0
-tmat In arg dump, the argument name is -tmat
/usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/transition_matrices
-tmatfloor 0.0001 In arg dump, the argument name is -tmatfloor
1.000000e-04
-treeugprob 1 In arg dump, the argument name is -treeugprob
1
-upperf 3500 In arg dump, the argument name is -upperf
6.855498e+03
-utt In arg dump, the argument name is -utt

-uw 0.7 In arg dump, the argument name is -uw
7.000000e-01
-var In arg dump, the argument name is -var
/usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/variances
-varfloor 0.0001 In arg dump, the argument name is -varfloor
1.000000e-04
-varnorm no In arg dump, the argument name is -varnorm
no
-vqeval 3 In arg dump, the argument name is -vqeval
3
-wbeam 1.0e-35 In arg dump, the argument name is -wbeam
1.000000e-35
-wend_beam 1.0e-80 In arg dump, the argument name is -wend_beam
0.000000e+00
-wip 0.7 In arg dump, the argument name is -wip
7.000000e-01
-wlen 0.0256 In arg dump, the argument name is -wlen
2.560000e-02

INFO: kbcore.c(95): Initializing core models:
INFO: logs3.c(99): Initializing logbase: 1.000300e+00 (add table: 1)
INFO: logs3.c(161): Log-Add table size = 29350
INFO: feat.c(642): Initializing feature stream to type: '1s_c_d_dd', CMN='current', VARNORM='no', AGC='none'
INFO: mdef.c(594): Reading model definition: /usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/Malay.6000.mdef
INFO: mdef.c(771): 14 CI-phone, 152 CD-phone, 5 emitstate/phone, 70 CI-sen, 260 Sen, 64 Sen-Seq
INFO: dict.c(358): Reading main dictionary: /usr/local/share/sphinx3/model/lm/an4/Malay.dic
ERROR: "dict.c", line 192: Line 7: Bad ciphone: L; word DALAM ignored
ERROR: "dict.c", line 192: Line 8: Bad ciphone: UH; word UNTUK ignored
ERROR: "dict.c", line 192: Line 9: Bad ciphone: P; word KEPADA ignored
ERROR: "dict.c", line 192: Line 10: Bad ciphone: G; word NEGARA ignored
ERROR: "dict.c", line 192: Line 11: Bad ciphone: AW; word ORANG ignored
ERROR: "dict.c", line 192: Line 12: Bad ciphone: AW; word OLEH ignored
ERROR: "dict.c", line 192: Line 13: Bad ciphone: JH; word JUGA ignored
ERROR: "dict.c", line 192: Line 14: Bad ciphone: S; word SAYA ignored
ERROR: "dict.c", line 192: Line 15: Bad ciphone: P; word DARIPADA ignored
ERROR: "dict.c", line 192: Line 16: Bad ciphone: P; word TETAPI ignored
ERROR: "dict.c", line 192: Line 18: Bad ciphone: L; word TELAH ignored
ERROR: "dict.c", line 192: Line 22: Bad ciphone: L; word LAGI ignored
ERROR: "dict.c", line 192: Line 23: Bad ciphone: JH; word KERAJAAN ignored
ERROR: "dict.c", line 192: Line 24: Bad ciphone: P; word DAPAT ignored
ERROR: "dict.c", line 192: Line 25: Bad ciphone: L; word LEBIH ignored
ERROR: "dict.c", line 192: Line 26: Bad ciphone: P; word PADA ignored
ERROR: "dict.c", line 192: Line 27: Bad ciphone: S; word SUDAH ignored
ERROR: "dict.c", line 192: Line 28: Bad ciphone: S; word ISLAM ignored
ERROR: "dict.c", line 192: Line 29: Bad ciphone: AO; word ATAU ignored
ERROR: "dict.c", line 192: Line 30: Bad ciphone: B; word BOLEH ignored
ERROR: "dict.c", line 192: Line 31: Bad ciphone: L; word MALAYSIA ignored
ERROR: "dict.c", line 192: Line 32: Bad ciphone: L; word ADALAH ignored
ERROR: "dict.c", line 192: Line 35: Bad ciphone: JH; word MENJADI ignored
ERROR: "dict.c", line 192: Line 36: Bad ciphone: S; word SAHAJA ignored
ERROR: "dict.c", line 192: Line 37: Bad ciphone: B; word BAGI ignored
ERROR: "dict.c", line 192: Line 38: Bad ciphone: S; word SATU ignored
ERROR: "dict.c", line 192: Line 39: Bad ciphone: L; word LAIN ignored
ERROR: "dict.c", line 192: Line 40: Bad ciphone: S; word SEMUA ignored
ERROR: "dict.c", line 192: Line 41: Bad ciphone: B; word BAHAWA ignored
ERROR: "dict.c", line 192: Line 42: Bad ciphone: JH; word JIKA ignored
ERROR: "dict.c", line 192: Line 43: Bad ciphone: S; word SEPERTI ignored
ERROR: "dict.c", line 192: Line 44: Bad ciphone: B; word BUKAN ignored
ERROR: "dict.c", line 192: Line 46: Bad ciphone: S; word SEBAGAI ignored
ERROR: "dict.c", line 192: Line 47: Bad ciphone: HH; word TAHUN ignored
ERROR: "dict.c", line 192: Line 48: Bad ciphone: AW; word EKONOMI ignored
ERROR: "dict.c", line 192: Line 49: Bad ciphone: L; word IALAH ignored
ERROR: "dict.c", line 192: Line 50: Bad ciphone: P; word APA ignored
ERROR: "dict.c", line 192: Line 51: Bad ciphone: HH; word HARI ignored
ERROR: "dict.c", line 192: Line 52: Bad ciphone: S; word SENDIRI ignored
ERROR: "dict.c", line 192: Line 53: Bad ciphone: P; word PUN ignored
ERROR: "dict.c", line 192: Line 54: Bad ciphone: P; word PERLU ignored
ERROR: "dict.c", line 192: Line 55: Bad ciphone: B; word BAIK ignored
ERROR: "dict.c", line 192: Line 56: Bad ciphone: B; word BANYAK ignored
ERROR: "dict.c", line 192: Line 57: Bad ciphone: SH; word MASYARAKAT ignored
ERROR: "dict.c", line 192: Line 58: Bad ciphone: L; word MELAYU ignored
ERROR: "dict.c", line 192: Line 59: Bad ciphone: S; word MASIH ignored
ERROR: "dict.c", line 192: Line 60: Bad ciphone: S; word MASA ignored
ERROR: "dict.c", line 192: Line 61: Bad ciphone: B; word MEMBERI ignored
ERROR: "dict.c", line 192: Line 62: Bad ciphone: B; word BESAR ignored
ERROR: "dict.c", line 192: Line 64: Bad ciphone: S; word SUPAYA ignored
ERROR: "dict.c", line 192: Line 65: Bad ciphone: S; word SEKARANG ignored
ERROR: "dict.c", line 192: Line 66: Bad ciphone: P; word PULA ignored
ERROR: "dict.c", line 201: Line 72: dict_add_word (KE) failed (duplicate?); ignored
ERROR: "dict.c", line 201: Line 73: dict_add_word (DIA) failed (duplicate?); ignored
ERROR: "dict.c", line 201: Line 74: dict_add_word (ADA) failed (duplicate?); ignored
ERROR: "dict.c", line 201: Line 75: dict_add_word (KERANA) failed (duplicate?); ignored
ERROR: "dict.c", line 201: Line 76: dict_add_word (IA) failed (duplicate?); ignored
ERROR: "dict.c", line 201: Line 77: dict_add_word (DARI) failed (duplicate?); ignored
ERROR: "dict.c", line 201: Line 78: dict_add_word (RAKYAT) failed (duplicate?); ignored
ERROR: "dict.c", line 201: Line 79: dict_add_word (ANTARA) failed (duplicate?); ignored
INFO: dict.c(361): 18 words read
INFO: dict.c(366): Reading filler dictionary: /usr/local/share/sphinx3/model/lm/an4/Malay.filler
INFO: dict.c(369): 3 words read
INFO: lm.c(739): LM read('/usr/local/share/sphinx3/model/lm/an4/Malay.arpa.DMP', lw= 9.50, wip= -1188, uw= 0.70)
INFO: lm.c(553): 71 ug
INFO: lm.c(583): 68 bigrams [on disk]
INFO: lm.c(591): 68 trigrams [on disk]
INFO: lm.c(613): 2 bigram prob entries
INFO: lm.c(631): 3 trigram bowt entries
INFO: lm.c(647): 2 trigram prob entries
INFO: lm.c(662): 1 trigram segtable entries (512 segsize)
INFO: lm.c(696): 71 word strings
ERROR: "wid.c", line 171: <UNK> is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: ADALAH is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: APA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: ATAU is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: BAGI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: BAHAWA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: BAIK is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: BANYAK is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: BESAR is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: BOLEH is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: BUKAN is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: DALAM is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: DAPAT is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: DARIPADA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: EKONOMI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: HARI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: IALAH is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: ISLAM is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: JIKA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: JUGA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: KEPADA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: KERAJAAN is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: LAGI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: LAIN is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: LEBIH is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: MALAYSIA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: MASA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: MASIH is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: MASYARAKAT is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: MELAYU is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: MEMBERI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: MENJADI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: NEGARA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: OLEH is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: ORANG is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: PADA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: PERLU is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: PULA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: PUN is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SAHAJA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SATU is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SAYA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SEBAGAI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SEKARANG is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SEMUA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SENDIRI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SEPERTI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SUDAH is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: SUPAYA is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: TAHUN is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: TELAH is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: TETAPI is not a word in dictionary and it is not a class tag.
ERROR: "wid.c", line 171: UNTUK is not a word in dictionary and it is not a class tag.
INFO: wid.c(178): 53 LM words not in dictionary; ignored
INFO: cont_mgau.c(90): Reading mixture gaussian file '/usr/local/share/sphinx3/model/hmm/hub4_cd_continuous_8gau_1s_c_d_dd/means'
FATAL_ERROR: "cont_mgau.c", line 128: #Features streams(4) != 1 in continuous HMM

Any help on this?

Regards,Chee Leong