Menu

#5 Option to limit FTP download amount

open
nobody
Options (4)
5
2005-07-28
2005-07-28
No

After setting FTP_PASSIVE=1 at the beginning of the
checkbot command line, we no longer receive Internal
Server errors related to FTP. This is very good. However...

We have many large files on our site referenced with
ftp:// URLs and Checkbot seems to download the whole
file before moving on to check the next link. This
slows down our site checking by a factor of 10 or more.

This is a feature request for a command line switch to
Checkbot to limit the amount of a file downloaded
before moving on to the next link. The assumption is
that if Checkbot can successfully start downloading the
file, it must exist, it is readible, and the FTP server
is working; we don't need to download the whole file.
We're willing to pass on the full download in favor of
speed of a partial download. Something like:

--ftp-thresh=10K

Thanks!!

Discussion


Log in to post a comment.