PHPCrawl is a high configurable webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
Rate This ProjectLogin To Rate This Project
Wow, this crawler has it all. It is - even with a point zero release faster and more mature and feature rich than every other I tried. Especially useful is the test interface, where you can try out all the parameters, without coding. This is excellent work!
Great tool to crawl sites, excelent support
Could use a "Per domain page-limit" :)
Very good job. Hard to find better!