i use cygwin to run sml toolkit v 2. there was an error about ( gunzip: stdin: unexpected end of file
gunzip: stdin: not in gzip format ) while i running wngram2idngram.exe. this is whai i got...
User@KP /cygdrive/c/delete/cmu/src
$ ./wngram2idngram -vocab test.vocab -buffer 100 -hash 200000 -temp /usr/tmp/ -
files 20 -gzip -verbosity 2 -n 3 -write_ascii < /cygdrive/c/testlm/testtrig.wng
ram > /cygdrive/c/testlm/testtrig2.idngram
Vocab : test.vocab
Buffer size : 100
Hash table size : 200000
Temp directory : /usr/tmp/
Max open files : 20
n : 3
FOF size : 10
buffer size = 4166600
Initialising hash table...
Reading vocabulary...
Allocating memory for the buffer...
Writing non-OOV counts to temporary file /usr/tmp/wngram2idngram.temp.KP.3080.1.
gz
Sorting final buffer...
Writing sorted buffer to temporary file /usr/tmp/wngram2idngram.temp.KP.3080.2.g
z
Merging temporary files...
i use cygwin to run sml toolkit v 2. there was an error about ( gunzip: stdin: unexpected end of file
gunzip: stdin: not in gzip format ) while i running wngram2idngram.exe. this is whai i got...
User@KP /cygdrive/c/delete/cmu/src
$ ./wngram2idngram -vocab test.vocab -buffer 100 -hash 200000 -temp /usr/tmp/ -
files 20 -gzip -verbosity 2 -n 3 -write_ascii < /cygdrive/c/testlm/testtrig.wng
ram > /cygdrive/c/testlm/testtrig2.idngram
Vocab : test.vocab
Buffer size : 100
Hash table size : 200000
Temp directory : /usr/tmp/
Max open files : 20
n : 3
FOF size : 10
buffer size = 4166600
Initialising hash table...
Reading vocabulary...
Allocating memory for the buffer...
Writing non-OOV counts to temporary file /usr/tmp/wngram2idngram.temp.KP.3080.1.
gz
Sorting final buffer...
Writing sorted buffer to temporary file /usr/tmp/wngram2idngram.temp.KP.3080.2.g
z
Merging temporary files...
gunzip: stdin: unexpected end of file
gunzip: stdin: not in gzip format
2-grams occurring: N times > N times Sug. -spec_num value
0 2 12
1 0 2 12
2 0 2 12
3 0 2 12
4 0 2 12
5 0 2 12
6 0 2 12
7 0 2 12
8 0 2 12
9 0 2 12
10 0 2 12
3-grams occurring: N times > N times Sug. -spec_num value
0 2 12
1 0 2 12
2 0 2 12
3 0 2 12
4 0 2 12
5 0 2 12
6 0 2 12
7 0 2 12
8 0 2 12
9 0 2 12
10 0 2 12
$