Menu
▾
▴
#270 htmerge 3.1.6, words.db=4GB corrupted
It looks like words.db file can't be bigger than 4GB.
i use htdig 3.1.6 to index lots of documents (100 GO).
htmerge stops when words.db file is exactly 4 GB.
(other htdig files are > 4GO).
Is it a limitation of berkeley db 2.6.4?
Does htdig 3.2 have this limitation?
Regards,
Logged In: YES
user_id=149687
Yes, I believe it is a limitation of Berkeley DB 2.6.4, at
least on some systems. We've had spotty reports of some
people having success with database files larger than 2 GB
on Solaris systems with ht://Dig 3.1.6, but database files
larger than 2 GB are known not to work on Linux with this
version. (You don't specify the platform on which you're
running it.) Generally though, we don't recommend using
3.1.6 with large database files, as it wasn't designed with
those in mind.
I'm pretty sure that ht://Dig 3.2.0b6 works fine with large
database files on Linux as well as other systems.
Logged In: YES
user_id=799288
It is on a Linux Mandrake 2006, kernel 2.6.12-12mdk .
.docdb file is 1,1 GO
.wordlist file is 7GO
.wordlist.new file (sorted by sort) is 6 GO
.words.db file is exactly 4 GO when htmerge stops.
since i have to switch to htdig 3.2, is there a way to
convert 3.1.6 files into 3.2 files, or am i obliged to
launch again the htdig process, which took several days to
build the files??
Some guys talk about mysql instead of db2: is it possible?
i have not found the patches to modify htdig 3.1.6 neither
3.2...
Regards,