I am using SimpleLM script to make a class based LM. (data already transformed to class string).
idngram2lm is crashing at the following line of code:
if (!ng->context_cue[current_ngram.id_array[0]]) {
ng->n_unigrams += ng_count[0];
I do not know where context_cue is initialized. it's first vallue is "0" and the value of
current_ngram.id_array[0] is 65535 which looks suspicious.
Any help in debugging will be greatly appreciated. I have tested this bug to be re-producible
in cygwin, Linux and Windows enviornment. I will be happy to send all the relevant files.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am using SimpleLM script to make a class based LM. (data already transformed to class string).
idngram2lm is crashing at the following line of code:
if (!ng->context_cue[current_ngram.id_array[0]]) {
ng->n_unigrams += ng_count[0];
I do not know where context_cue is initialized. it's first vallue is "0" and the value of
current_ngram.id_array[0] is 65535 which looks suspicious.
Any help in debugging will be greatly appreciated. I have tested this bug to be re-producible
in cygwin, Linux and Windows enviornment. I will be happy to send all the relevant files.
Oh... Never mind. It appears that the bug appears because the input to idngram2lm is bad in
cygwin enviornment.
If I run the entire script in linux enviornment, it does move forward smoothly.
so the .idngram file generated by the previous step should be corrupt.
Oh... Never mind. It appears that the bug appears because the input to idngram2lm is bad in
cygwin enviornment.
If I run the entire script in linux enviornment, it does move forward smoothly.
so the .idngram file generated by the previous step should be corrupt.