RoboBrowser is a webkit powered browser which built for web scraping purposes. It loads requested webpage, saves page source to disk, and sends it's path to a php script as first parameter.
APC Anti Crawler is a php5 class based on APC which can be used to limit the amount of http request per IP. It stop web crawler to download your entire website.
Larbin is a Web crawler intended to fetch a large number of Web pages, it should be able to fetch more than 100 millions pages on a standard PC with much u/d. This set of PHP and Perl scripts, called webtools4larbin, can handle the output of Larbin and p