Error sphinx_lm_convert format

[Ana Alfaro]

Hi!
I'm using sphinxbase-5prealpha to convert LM file into bin file. I'm executing in ubuntu:

sphinx_lm_convert -i myModel.lm -o myModel.lm.bin

And I receive these error and output file doesn't create:
ERROR: "ngram_model.c", line 183: language model file type not supported
ERROR: "sphinx_lm_convert.c", line 212: Failed to write language model in format (null) to myModel.lm.bin

How can I solve this? I'm doing the same wiki says in:
http://cmusphinx.sourceforge.net/wiki/tutoriallm

I need to convert my lm into bin file because my generated language model is tooooooo big.

Thanks for all!!
Ana

[Nickolay V. Shmyrev]

Please share the file causing problem.

Please share the complete log output.

Please make sure you are using latest sphinxbase version and do not have older version installed.

[sheharyar masood]

Hi ana alfaro!
I am facing problem with sphinx_lm_convert -i myModel.lm -o myModel.lm.bin it dont recognizes sphinx_lm_convert ...... do you have any idea? Please tell me! I am stuck here for really very long time! :-(

[sheharyar masood]

Hi ana alfaro!
I am facing problem with sphinx_lm_convert -i myModel.lm -o myModel.lm.bin it dont recognizes sphinx_lm_convert ...... do you have any idea? Please tell me! I am stuck here for really very long time! :-(

[Ana Alfaro]

Hi Nickolay! Thanks for your answer.

I've downloaded last sphinxbase-master version, I think, from this url:
https://github.com/cmusphinx/sphinxbase

So I've uncompressed this in ubuntu and I've done:
$ cd src/sphinx_lmtools
I've copied my LM in this directory, and next I've done:
$ sphinx_lm_convert -i voxforge_es.lm -o voxforge_es.lm.bin

The voxforge_es.lm file is the file can be downloaded from http://cmusphinx.sourceforge.net/ in Spanish language. I have another LM own file but I get the same result, I mean no result.

I attach voxforge_es.lm file and test.log file with the result I get in console when I execute sphinx_lm_convert.

Do you have any idea?
Thank you very much.

Ana Alfaro

[Nickolay V. Shmyrev]

Hello Ana

Your log demonstrates that you have an old version of sphinxbase installed. You can figure out which exactly binary do you run with the following command

       which sphinx_lm_convert

you can remove sphinxbase package from distribution with apt-get package manager. Once you removed package you need to install sphinxbase per instructions in README with the commands:

     unzip sphinxbase-master.zip
     cd sphinxbase-master
     ./configure
     make
     sudo make install

You can find detailed instruction in README. It is not enough just to unpack the source.

[Ana Alfaro]

Hello Nickolay,

I have followed all the steps to install sphinxbase. If I run which sphinx_lm_convert, I see:
/usr/local/bin/sphinx_lm_convert

And wherever I run the command:
sphinx_lm_convert -i myModel.lm -o myModel.lm.bin
I see:
sphinx_lm_convert: error while loading shared libraries: libsphinxbase.so.3: cannot open shared object file: No such file or directory

I found your answer here:
http://stackoverflow.com/questions/10630747/converting-the-lm-to-dmp-file-for-building-the-language-model-for-speech-rec

I've tried the two first points and I've got the same error... do i need to recompile? I don't understand very well your third point (with /usr prefix?)...

Thanks you very much.

Ana

[Nickolay V. Shmyrev]

1) Make sure that the file libsphinbase.so.3 is present in the folder /usr/local/lib

2) Open the terminal and type

export LD_LIBRARY_PATH=/usr/local/lib

3) then in the same terminal without closing it run sphinx_lm_convert.

4) If something fails, ask local Linux expert

[Ana Alfaro]

Hi Nickolay!!

Thanks for your answer, it works fine and generates bin file.

But when I run sphinx_lm_convert with my own huge LM I receive this message, although bin file is apparently created ok:

ERROR: "lm_trie.c", line 105: Sorry, this does not support more than 33554432 n-grams of a particular order. Edit util/bit_packing.hh and fix the bit packing functions
INFO: lm_trie.c(474): Training quantizer
INFO: lm_trie.c(482): Building LM trie

Where util/bit_packing.hh is located? I can't find it.
Do you have any advice any way?

Thank you very much.
Ana

[Nickolay V. Shmyrev]

Prune your huge language model with srilm to make it smaller.

[Ana Alfaro]

Thank you Nickolay, we try to prune it with srilm.

If I run Sphinx4 using that BIN Language Model generated, I receive a NullPointerException in line 95 into BinaryLoader class because tables (float[]) is null (...and quantType is NO_QUANT). It 's trying to allocate LM with NgramTrieModel class.
Do you have any idea why Sphinx4 is not working?

Thanks for all.
Ana

[Ana Alfaro]

Even I run Sphinx4 using BIN Language Model generated from your default Spanish LM I get same error. But It works If Sphinx4 uses your default English BIN LM.
I attach both BIN LM you to try it please!

Thank you very much.
Ana

[Nickolay V. Shmyrev]

I have just fixed this bug in sphinxbase, please update sphinxbase and try to convert lm to bin again.

Thank you and sorry for inconvenience.

[Ana Alfaro]

Thank you Nickolay!!
I've just downloaded sphinxbase from: https://github.com/cmusphinx/sphinxbase
Is that the newer version? If it isn't, how can I get it?

For this version, I get these errors when I exec command "make":

$ make
/bin/bash: -c: line 0: syntax error near unexpected token )' /bin/bash: -c: line 0:fail=; if (arget_optioon=k; case ${target_option-} in ?) ;; ) echo "ammake_running_with_option: internal error: invalid" "target option '${target_option-}' specified" >&2; exit 1;; esac; has_opt=no; sane_makeflags=$MAKEFLAGS; if test -n '' && test -n '0'; then sane_makeflags=$MFLAGS; else case $MAKEFLAGS in \[\ \ ]) bs=\; sane_makeflags=printf '%s\n' "$MAKEFLAGS" | sed "s/$bs$bs[$bs $bs ]*//g";; esac; fi; skip_next=no; strip_trailopt () { flg=printf '%s\n' "$flg" | sed "s/$1.*$//"; }; for flg in $sane_makeflags; do test $skip_next = yes && { skip_next=no; continue; }; case $flg in =|--) continue;; -I) strip_trailopt 'I'; skip_next=yes;; -I?) strip_trailopt 'I';; -O) strip_trailopt 'O'; skip_next=yes;; -O?) strip_trailopt 'O';; -l) strip_trailopt 'l'; skip_next=yes;; -l?) strip_trailopt 'l';; -[dEDm]) skip_next=yes;; -[JT]) skip_next=yes;; esac; case $flg in $target_option) has_opt=yes; break;; esac; done; test $has_opt = yes); then failcom='fai==yes'; else \ failcoom='exit 1'; fi; dtseen==no; taret=echo alll-recursive | se /-recursive///; case "all-rrecursive" in disclean- | maitainer-clean-)) list='src doc include test swig' ;; *) list'r docc includde test swig' ;; \ esac; for subdr in $llist; d echo "Making $target in $subdir"; iff tes "$subbdir" =""; then dot_een=yess; \ local_targett="$target-am"; \ else loca_targett="$ttarget"; fi; CDPAATH="${ZHVERSION+.}:" && cd $subdir && make $local_target) || eval$alcomm; done; \ if testt ""$$dot_seen" = "no"; then make "targett-am" || exit 1; fi; tes -z ffaaail"'
make: ** [all-recursive] Error 1

And when I exec command "make install":

$ sudo make install
/bin/bash: -c: line 1: syntax error: unexpected end of file
make: *** [install-recursive] Error 1

Is about a new bug?

Thank you.
Ana

[Nickolay V. Shmyrev]

I've just downloaded sphinxbase from: https://github.com/cmusphinx/sphinxbase

This is correct version

/bin/bash: -c: line 1: syntax error: unexpected end of file

I'm sorry, I have no idea how your file got corrupted, I suggest you to clone a clean version once again and go through installation process with autogen.sh and make and make install. In case you still see the troubles please provide the complete build log.

[aditya vikram]

hello nickolay or ana,
i am a newbi can u tell the step by step procedure

[aditya vikram]

hello nickolay or ana,
i am a newbi can u tell the step by step procedure

[Pooja Daga]

Hello Aditya,
Have you got any way to overcome this problem?
I had just started with pocketsphinx, if you have any solution to fix this please share.
It will be highly helpful.
Thanks!

[aditya vikram]

i just did on ur other post if it help it will be gr8 otherwise warlockadityaa@gmail.com
my email u can contact me there i am always active
cheers!