From: Robert R. <ri...@li...> - 2004-09-05 10:52:24
|
Hello Developers, What is the maximum filesize htdig (3.1.6) can access? Does sizeof long happen to be 2^30? Please take a look at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=269831 Any idea how we can acto on such an error? Robert |
From: Jim <li...@yg...> - 2004-09-05 11:45:59
|
On Sun, 5 Sep 2004, Robert Ribnitz wrote: > What is the maximum filesize htdig (3.1.6) can access? Out of the box under Linux there is a 2 GB limit. > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=269831 > > Any idea how we can acto on such an error? There are at least a couple issues related to the maximum file size. The first is ht://Dig's direct handling of on-disk files, which is the one in play here. The db.wordlist.work file is a text file directly handled with system calls such as fopen(). Since the ht://Dig package isn't compiled with largefile support by default, such access is affected by the 2 GB limit. A couple people have reported that adding '-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64' has successfully provided a workaround for the limit on db.wordlist. The second issue is one of support at the Berkeley DB level. The version provided with 3.1.6 doesn't provide support for large files under Linux. There is an --enable-bigfile, but its support is limited to AIX, HP/UX, and Solaris. Some time ago attempts were made to force the option for a Linux build, but it was reported that the attempts were unsuccessful. A more recent post implied that the limit at the BDB level is 4 GB. So by building ht://Dig with largefile support you can most likely bypass problem with limits on the size of db.wordlist. However there will still be a limit on the size of the actual databases (presumably 4 GB). Jim |