I have downloaded gigaword lm and trying to build the decoding graph with the help of instructions in http://kaldi.sourceforge.net/graph_recipe_test.html#graph_symtab.
I was able to create L.fst and G.fst. But when I am trying to prepare LG.fst it is failing with std::bad_alloc error. Size of G.fst is around 430 mb. I am running it on ubuntu 32-bit machine. I thought of reducing the lm size and tried to prune lm using the command ngram -prune 3e-7 -lm lm.arpa -write-lm out.lm. But ngram is exiting with some corruption ( Error in `ngram': free(): invalid pointer:). Is it possible to construct the decoding graph without pruning the huge lm? what am I doing wrong while pruning lm using ngram? Or is it required to run the commands on 64 bit machine ?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It's likely that you do need to prune G.fst, as it sounds quite large. You
might want to either try debugging the SRILM tools and figuring out what's
happening, or running the pruning on a different machine.
Dan
I have downloaded gigaword lm and trying to build the decoding graph with
the help of instructions in http://kaldi.sourceforge.net/graph_recipe_test.html#graph_symtab.
I was able to create L.fst and G.fst. But when I am trying to prepare
LG.fst it is failing with std::bad_alloc error. Size of G.fst is around 430
mb. I am running it on ubuntu 32-bit machine. I thought of reducing the lm
size and tried to prune lm using the command ngram -prune 3e-7 -lm lm.arpa
-write-lm out.lm. But ngram is exiting with some corruption ( Error in
`ngram': free(): invalid pointer:). Is it possible to construct the
decoding graph without pruning the huge lm? what am I doing wrong while
pruning lm using ngram? Or is it required to run the commands on 64 bit
machine ?
I have downloaded gigaword lm and trying to build the decoding graph with the help of instructions in http://kaldi.sourceforge.net/graph_recipe_test.html#graph_symtab.
I was able to create L.fst and G.fst. But when I am trying to prepare LG.fst it is failing with std::bad_alloc error. Size of G.fst is around 430 mb. I am running it on ubuntu 32-bit machine. I thought of reducing the lm size and tried to prune lm using the command ngram -prune 3e-7 -lm lm.arpa -write-lm out.lm. But ngram is exiting with some corruption ( Error in `ngram': free(): invalid pointer:). Is it possible to construct the decoding graph without pruning the huge lm? what am I doing wrong while pruning lm using ngram? Or is it required to run the commands on 64 bit machine ?
It's likely that you do need to prune G.fst, as it sounds quite large. You
might want to either try debugging the SRILM tools and figuring out what's
happening, or running the pruning on a different machine.
Dan
On Thu, Oct 9, 2014 at 4:22 AM, K R Srinidhi srinidhikrs@users.sf.net
wrote: