Everything goes great, but when I try to load the ngram:
_logComputation = ps_get_logmath(_pocketsphinxDecoder);
if (_logComputation == NULL)
throw(SpeechException("logmath object in sphinx decoder not succesfully created"));
ngram_model_t *nGramModel = ngram_model_read(_pocketsphinxConfig,_options->getSphinxNGramModel().c_str(),NGRAM_AUTO,_logComputation);
I get a 'Floating point exception' (8) error in the following line of the
lm3g_model.c file:
for(i=0;i<lm3g->n_prob2;++i){int32prob2;/* Can't just punt this back to general code since it is quantized. */prob2=(int32)((lm3g->prob2[i].l-base->log_wip)/base->lw);[b]lm3g->prob2[i].l=(int32)(prob2*lw)+log_wip;[/b]}
Using the same language model file with pocketsphinx_continuous gives no
error, and executing my program with valgrind also does not give the error.
Did someone experienced a similar problem?
The Log resulting from the execution of the program is as follows:
because if so it does not try to apply the weights. However, with this
solution I want be able to change the default wwight of the model, or is there
another way to do it?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You can use ps_get_config to get config object from decoder, there is no need
to use your own copy which might be corrupted in this partiuclar situation
/ Can't just punt this back to general code since it is quantized. / prob2
= (int32)((lm3g->prob2.l - base->log_wip) / base->lw); lm3g->prob2.l =
(int32)(prob2 * lw) + log_wip;
You need to compile without optimization to debug the application. The crash
actually happens in the division, most likely base->lw is 0. There could be
multiple reasons for that, for example a corrupted config or config structure
without entry for lw option. It's easier to debug this situation to find why
base->lw wasn't set properly.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Finally I discovered that the crash has a relation with the -lw value in the
cmd_ln_t config:
If I run the following code, the decoder initializes without problem:
voidlm3g_apply_weights(ngram_model_t*base,lm3g_model_t*lm3g,float32lw,float32wip,float32uw){......for(i=0;i<lm3g->n_prob2;++i){int32prob2;/* Can't just punt this back to general code since it is quantized. */prob2=(int32)((lm3g->prob2[i].l-base->log_wip)/base->lw);[b]lm3g->prob2[i].l=(int32)(prob2*lw)+log_wip;[/b]}......}
The debugger shows there are the values of the variables when executing the
line in bold are:
Looks like the problem is that prob2 is going out of scope. But I am not sure.
If this is the case.
prob2 comes from the model loaded, doesnt it?
But the file I am loading work when called from the pocketsphinx_continuous
and batch.
It's often happens that beginner developers create a programs with memory
corruption. It's very often happens that this corruption breaks the
functionality of the library. Often such developers complain about the library
while library works perfectly.
Instead you should try to find a bug in your applicaiton. You can use debugger
to do that. You can use watchpoints to track memory values. You can also use
specialized tools like valgrind to catch memory corruption.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It looks like the error is related to some uninitialised values in the
decdoer. If no default language model is set in the initialitsation and then a
language model set is updated. The following code:
But this only happens if the decoder cannnot find the language model
hub4.5000.DMP. If it finds it, although no default language model is specified
in the config, it loads it and initializes correctly. If the mentioned file is
not there. Then it is not loaded and something goes wrong in the
initialization.
I am trying to implement Ngram model switching but I did not have the default
hub model. And this ended up in using unitized values. I guess. Of course,
there are a lot of changes that I am wrong.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I just noticed that in the framework I am working in, the math exceptions are
enabled by default. While in most environments are disabled by default, as it
is if you use GCC straight away. So, yes, there is a problem in the
initialization,. What happens is that you just don't see it. If you compile
the following code. Which is the one you mentioned, but with math exceptions
enabled. You will get the same arithmetic exception as I do.
Exactly the one I got in my code. lm3g->prob2_.l tries to get a value that is
out of the scope of the variable definition.
Lower than the minimum in my case. What happens is that if you use -lw == 1
then you do not chenge the value, and that is it.
I do not know why, but if you use valgrind, this kind of exceptions are not
thrown.
I did no catch all the details related with this function, but I think in this
point lm3g->prob2 is used uninitialised. Or otherwise, might be a problem of
the -logbase used to load the model. I am not sure yet. If this is intended,
then it would convenient to add a numneric_limits check in there.
I used:
GCC version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
GNU gdb (GDB) 7.1-ubuntu
valgrind-3.6.0.SVN-Debian
Regards.
_
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
To clarify a bit everything. There are two issues:
1) Ngram loading generates and arithmetic exception when applying weights if
math exceptions are enables. (last post)
2) If there is not an initial model, loaded with the -lm option int ps_init, a
segmentation fault is produced when updating the lmset. This is because base
is null in ngram_model_set_map_words(..). This only happens if sphinx is
compiled with -O3 option. (previous to last post)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I integrated pocketsphinx in our code. I can perform recognition and change
grammar (FSG) online., with fsg_update and select.
However, in another mode I want the decoder to load an ngram model instead of
an fsg. I setup the cmd_ln_t * _pocketsphinxConfig, in this way:
Then when I initialise the decoder:
Everything goes great, but when I try to load the ngram:
I get a 'Floating point exception' (8) error in the following line of the
lm3g_model.c file:
Using the same language model file with pocketsphinx_continuous gives no
error, and executing my program with valgrind also does not give the error.
Did someone experienced a similar problem?
The Log resulting from the execution of the program is as follows:
Current configuration:
-agc none none
-agcthresh 2.0 2.000000e+00
-alpha 0.97 9.700000e-01
-ascale 20.0 2.000000e+01
-aw 1 1
-backtrace no no
-beam 1e-48 1.000000e-48
-bestpath yes yes
-bestpathlw 9.5 9.500000e+00
-bghist no no
-ceplen 13 13
-cmn current current
-cmninit 8.0 8.0
-compallsen no no
-debug 0
-dict /opt/sphinx/share/pocketsphinx/model/lm/en_US/cmu07a.dic
-dictcase no no
-dither no no
-doublebw no no
-ds 1 1
-fdict
-feat 1s_c_d_dd 1s_c_d_dd
-featparams
-fillprob 1e-8 1.000000e-08
-frate 100 100
-fsg
-fsgusealtpron yes yes
-fsgusefiller yes yes
-fwdflat yes yes
-fwdflatbeam 1e-64 1.000000e-64
-fwdflatefwid 4 4
-fwdflatlw 8.5 8.500000e+00
-fwdflatsfwin 25 25
-fwdflatwbeam 7e-29 7.000000e-29
-fwdtree yes yes
-hmm /opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k/
-input_endian little little
-jsgf
-kdmaxbbi -1 -1
-kdmaxdepth 0 0
-kdtree
-latsize 5000 5000
-lda
-ldadim 0 0
-lextreedump 0 0
-lifter 0 0
-lm
-lmctl
-lmname default default
-logbase 1.0001 1.000100e+00
-logfn
-logspec no no
-lowerf 133.33334 1.333333e+02
-lpbeam 1e-40 1.000000e-40
-lponlybeam 7e-29 7.000000e-29
-lw 6.5 1.100000e+00
-maxhmmpf -1 -1
-maxnewoov 20 20
-maxwpf -1 -1
-mdef /opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//mdef
-mean
-mfclogdir
-min_endfr 0 0
-mixw
-mixwfloor 0.0000001 1.000000e-07
-mllr
-mmap yes yes
-ncep 13 13
-nfft 512 512
-nfilt 40 40
-nwpen 1.0 1.000000e+00
-pbeam 1e-48 1.000000e-48
-pip 1.0 1.000000e+00
-pl_beam 1e-10 1.000000e-10
-pl_pbeam 1e-5 1.000000e-05
-pl_window 0 0
-rawlogdir
-remove_dc no no
-round_filters yes yes
-samprate 16000 1.600000e+04
-seed -1 -1
-sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03
-smoothspec no no
-svspec
-tmat
-tmatfloor 0.0001 1.000000e-04
-topn 4 4
-topn_beam 0 0
-toprule
-transform legacy legacy
-unit_area yes yes
-upperf 6855.4976 6.855498e+03
-usewdphones no no
-uw 1.0 1.100000e+00
-var
-varfloor 0.0001 1.000000e-04
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wbeam 7e-29 7.000000e-29
-wip 0.65 1.100000e+00
-wlen 0.025625 2.562500e-02
16:47:45:957: Initialising the Activity detector
INFO: cmd_ln.c(691): Parsing command line:
\
-nfilt 20 \
-lowerf 1 \
-upperf 4000 \
-wlen 0.025 \
-transform dct \
-round_filters no \
-remove_dc yes \
-svspec 0-12/13-25/26-38 \
-feat 1s_c_d_dd \
-agc none \
-cmn current \
-cmninit 56,-3,1 \
-varnorm no
Current configuration:
-agc none none
-agcthresh 2.0 2.000000e+00
-alpha 0.97 9.700000e-01
-ceplen 13 13
-cmn current current
-cmninit 8.0 56,-3,1
-dither no no
-doublebw no no
-feat 1s_c_d_dd 1s_c_d_dd
-frate 100 100
-input_endian little little
-lda
-ldadim 0 0
-lifter 0 0
-logspec no no
-lowerf 133.33334 1.000000e+00
-ncep 13 13
-nfft 512 512
-nfilt 40 20
-remove_dc no yes
-round_filters yes no
-samprate 16000 1.600000e+04
-seed -1 -1
-smoothspec no no
-svspec 0-12/13-25/26-38
-transform legacy dct
-unit_area yes yes
-upperf 6855.4976 4.000000e+03
-varnorm no no
-verbose no no
-warp_params
-warp_type inverse_linear inverse_linear
-wlen 0.025625 2.500000e-02
INFO: acmod.c(242): Parsed model-specific feature parameters from
/opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//feat.params
INFO: feat.c(684): Initializing feature stream to type: '1s_c_d_dd',
ceplen=13, CMN='current', VARNORM='no', AGC='none'
INFO: cmn.c(142): mean= 12.00, mean= 0.0
INFO: acmod.c(163): Using subvector specification 0-12/13-25/26-38
INFO: mdef.c(520): Reading model definition:
/opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//mdef
INFO: mdef.c(531): Found byte-order mark BMDF, assuming this is a binary mdef
file
INFO: bin_mdef.c(330): Reading binary model definition:
/opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//mdef
INFO: bin_mdef.c(507): 50 CI-phone, 143047 CD-phone, 3 emitstate/phone, 150
CI-sen, 5150 Sen, 27135 Sen-Seq
INFO: tmat.c(205): Reading HMM transition probability matrices: /opt/sphinx/sh
are/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//transition_matrices
INFO: acmod.c(117): Attempting to use SCHMM computation module
INFO: ms_gauden.c(198): Reading mixture gaussian parameter:
/opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//means
INFO: ms_gauden.c(292): 1 codebook, 3 feature, size:
INFO: ms_gauden.c(294): 256x13
INFO: ms_gauden.c(294): 256x13
INFO: ms_gauden.c(294): 256x13
INFO: ms_gauden.c(198): Reading mixture gaussian parameter:
/opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//variances
INFO: ms_gauden.c(292): 1 codebook, 3 feature, size:
INFO: ms_gauden.c(294): 256x13
INFO: ms_gauden.c(294): 256x13
INFO: ms_gauden.c(294): 256x13
INFO: ms_gauden.c(354): 0 variance values floored
INFO: s2_semi_mgau.c(908): Loading senones from dump file
/opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//sendump
INFO: s2_semi_mgau.c(932): BEGIN FILE FORMAT DESCRIPTION
INFO: s2_semi_mgau.c(1027): Using memory-mapped I/O for senones
INFO: s2_semi_mgau.c(1304): Maximum top-N: 4 Top-N beams: 0 0 0
INFO: dict.c(306): Allocating 137542 * 20 bytes (2686 KiB) for word entries
INFO: dict.c(321): Reading main dictionary:
/opt/sphinx/share/pocketsphinx/model/lm/en_US/cmu07a.dic
INFO: dict.c(212): Allocated 1010 KiB for strings, 1664 KiB for phones
INFO: dict.c(324): 133436 words read
INFO: dict.c(330): Reading filler dictionary:
/opt/sphinx/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k//noisedict
INFO: dict.c(212): Allocated 0 KiB for strings, 0 KiB for phones
INFO: dict.c(333): 11 words read
INFO: dict2pid.c(396): Building PID tables for dictionary
INFO: dict2pid.c(404): Allocating 50^3 * 2 bytes (244 KiB) for word-initial
triphones
INFO: dict2pid.c(131): Allocated 30200 bytes (29 KiB) for word-final triphones
INFO: dict2pid.c(195): Allocated 30200 bytes (29 KiB) for single-phone word
triphones
16:47:46:687: Loading and setting the language model from
/home/jordiadell/work/asrEvaluation/sphinx/cmuarctic.arpa
INFO: ngram_model_arpa.c(477): ngrams 1=2774, 2=8228, 3=10597
INFO: ngram_model_arpa.c(135): Reading unigrams
INFO: ngram_model_arpa.c(516): 2774 = #unigrams created
INFO: ngram_model_arpa.c(195): Reading bigrams
INFO: ngram_model_arpa.c(533): 8228 = #bigrams created
INFO: ngram_model_arpa.c(534): 339 = #prob2 entries
INFO: ngram_model_arpa.c(542): 547 = #bo_wt2 entries
INFO: ngram_model_arpa.c(292): Reading trigrams
INFO: ngram_model_arpa.c(555): 10597 = #trigrams created
INFO: ngram_model_arpa.c(556): 152 = #prob3 entries
SigHandler captured signal 'Floating point exception' (8). Giving opportunity
to close nicely.
SigHandler will quit the application after receiving signal Floating point
exception (8)
si_errno 0si_code: 7 Signal generated by kernel FPE_FLTINV: floating point
invalid operation si_addr 0xb781a0f5
Regards.
Jordi Adell
I just discovered that calling the reading function with NULL insteand of the
command line configurations. This is:
instead of:
because if so it does not try to apply the weights. However, with this
solution I want be able to change the default wwight of the model, or is there
another way to do it?
Please provide software version when you report about errors
It doesn't seem right. You are using uninitialized variable. First arugment
must be NULL.
You can use ps_get_config to get config object from decoder, there is no need
to use your own copy which might be corrupted in this partiuclar situation
You need to compile without optimization to debug the application. The crash
actually happens in the division, most likely base->lw is 0. There could be
multiple reasons for that, for example a corrupted config or config structure
without entry for lw option. It's easier to debug this situation to find why
base->lw wasn't set properly.
Sorry for ḿissing the version, at end I will learn how to post a question ...
I'm using:
pocketsphinx-0.7/
sphinxtrain-1.0.7/
Thanks a lot for your help, I will refactor the code so that no pointers are
stored.
Finally I discovered that the crash has a relation with the -lw value in the
cmd_ln_t config:
If I run the following code, the decoder initializes without problem:
However, if the code is this one:
It crashes applying weights:
The HMM models I load are:
the Ngram is:
and the dictionary:
I do not understand why this happens since -lw 6.5 is the value that is
recomended in the pockersphinx command line:
I am now debugging with out optionmization and will report if I find something
useful.
I am working with pockesphinx 0.7
If I load a grammar this way:
and then I try to set weights:
An error occurs here:
The debugger shows there are the values of the variables when executing the
line in bold are:
Looks like the problem is that prob2 is going out of scope. But I am not sure.
If this is the case.
prob2 comes from the model loaded, doesnt it?
But the file I am loading work when called from the pocketsphinx_continuous
and batch.
The ngram model used here can be found in:
http://dl.dropbox.com/u/818449/cmuarctic.arpa
Do you have any hint?
Do you think the file is malformed?, did I found a bug?
I verified the following small example with the version you've mentioned
It's often happens that beginner developers create a programs with memory
corruption. It's very often happens that this corruption breaks the
functionality of the library. Often such developers complain about the library
while library works perfectly.
Instead you should try to find a bug in your applicaiton. You can use debugger
to do that. You can use watchpoints to track memory values. You can also use
specialized tools like valgrind to catch memory corruption.
I am sorry for having bothered you so much.
I'll keep working on it, let's see if I am good enough to find the error.
Regards.
It looks like the error is related to some uninitialised values in the
decdoer. If no default language model is set in the initialitsation and then a
language model set is updated. The following code:
gives the following error:
with the following backtrace:
because base=0x0
But this only happens if the decoder cannnot find the language model
hub4.5000.DMP. If it finds it, although no default language model is specified
in the config, it loads it and initializes correctly. If the mentioned file is
not there. Then it is not loaded and something goes wrong in the
initialization.
I am trying to implement Ngram model switching but I did not have the default
hub model. And this ended up in using unitized values. I guess. Of course,
there are a lot of changes that I am wrong.
Finally!
I just noticed that in the framework I am working in, the math exceptions are
enabled by default. While in most environments are disabled by default, as it
is if you use GCC straight away. So, yes, there is a problem in the
initialization,. What happens is that you just don't see it. If you compile
the following code. Which is the one you mentioned, but with math exceptions
enabled. You will get the same arithmetic exception as I do.
If you run this with GDB, then you'll get the following error:
Exactly the one I got in my code. lm3g->prob2_.l tries to get a value that is
out of the scope of the variable definition.
Lower than the minimum in my case. What happens is that if you use -lw == 1
then you do not chenge the value, and that is it.
I do not know why, but if you use valgrind, this kind of exceptions are not
thrown.
I did no catch all the details related with this function, but I think in this
point lm3g->prob2 is used uninitialised. Or otherwise, might be a problem of
the -logbase used to load the model. I am not sure yet. If this is intended,
then it would convenient to add a numneric_limits check in there.
I used:
GCC version 4.4.3 (Ubuntu 4.4.3-4ubuntu5)
GNU gdb (GDB) 7.1-ubuntu
valgrind-3.6.0.SVN-Debian
Regards.
_
To clarify a bit everything. There are two issues:
1) Ngram loading generates and arithmetic exception when applying weights if
math exceptions are enables. (last post)
2) If there is not an initial model, loaded with the -lm option int ps_init, a
segmentation fault is produced when updating the lmset. This is because base
is null in ngram_model_set_map_words(..). This only happens if sphinx is
compiled with -O3 option. (previous to last post)
That's something to fix. Many thanks.