i use cygwin to run sml toolkit v 2. there was an error about ( gunzip: stdin: unexpected end of file
gunzip: stdin: not in gzip format ) while i running wngram2idngram.exe. this is whai i got...
User@KP /cygdrive/c/delete/cmu/src
$ ./wngram2idngram -vocab test.vocab -buffer 100 -hash 200000 -temp /usr/tmp/ -
files 20 -gzip -verbosity 2 -n 3 -write_ascii < /cygdrive/c/testlm/testtrig.wng
ram > /cygdrive/c/testlm/testtrig2.idngram
Vocab : test.vocab
Buffer size : 100
Hash table size : 200000
Temp directory : /usr/tmp/
Max open files : 20
n : 3
FOF size : 10
buffer size = 4166600
Initialising hash table...
Reading vocabulary...
Allocating memory for the buffer...
Writing non-OOV counts to temporary file /usr/tmp/wngram2idngram.temp.KP.3080.1.
gz
Sorting final buffer...
Writing sorted buffer to temporary file /usr/tmp/wngram2idngram.temp.KP.3080.2.g
z
Merging temporary files...
i use cygwin to run sml toolkit v 2. there was an error about ( gunzip: stdin: unexpected end of file
gunzip: stdin: not in gzip format ) while i running wngram2idngram.exe. this is whai i got...
User@KP /cygdrive/c/delete/cmu/src
$ ./wngram2idngram -vocab test.vocab -buffer 100 -hash 200000 -temp /usr/tmp/ -
files 20 -gzip -verbosity 2 -n 3 -write_ascii < /cygdrive/c/testlm/testtrig.wng
ram > /cygdrive/c/testlm/testtrig2.idngram
Vocab : test.vocab
Buffer size : 100
Hash table size : 200000
Temp directory : /usr/tmp/
Max open files : 20
n : 3
FOF size : 10
buffer size = 4166600
Initialising hash table...
Reading vocabulary...
Allocating memory for the buffer...
Writing non-OOV counts to temporary file /usr/tmp/wngram2idngram.temp.KP.3080.1.
gz
Sorting final buffer...
Writing sorted buffer to temporary file /usr/tmp/wngram2idngram.temp.KP.3080.2.g
z
Merging temporary files...
gunzip: stdin: unexpected end of file
gunzip: stdin: not in gzip format
2-grams occurring: N times > N times Sug. -spec_num value
0 2 12
1 0 2 12
2 0 2 12
3 0 2 12
4 0 2 12
5 0 2 12
6 0 2 12
7 0 2 12
8 0 2 12
9 0 2 12
10 0 2 12
3-grams occurring: N times > N times Sug. -spec_num value
0 2 12
1 0 2 12
2 0 2 12
3 0 2 12
4 0 2 12
5 0 2 12
6 0 2 12
7 0 2 12
8 0 2 12
9 0 2 12
10 0 2 12
$
Still have the test case? Please send it to me. -Arthur Chan
Sir
I am getting some error during Language model.
Help me out.
Error
ahmad@HSRD14:~/Speech/CMU-Cam_Toolkit_v2/bin$ text2idngram -vocab $bn.tmp.vocab < $bn.tmp.closed > $bn.tmp.idngram
text2idngram
Vocab : weather.txt.tmp.vocab
N-gram buffer size : 100
Hash table size : 200000
Temp directory : /usr/tmp/
Max open files : 20
FOF size : 10
n : 3
Initialising hash table...
Reading vocabulary...
Allocating memory for the n-gram buffer...
Reading text into the n-gram buffer...
20,000 n-grams processed for each ".", 1,000,000 for each line.
..................................................
........................
Sorting n-grams...
Writing sorted n-grams to temporary file /usr/tmp/text2idngram.temp.HSRD14.6331.1
rr_fopen: problems opening /usr/tmp/text2idngram.temp.HSRD14.6331.1 for "w".