Download
PHPCrawl is a high configurable webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
License
GNU General Public License version 2.0 (GPLv2)Follow PHPCrawl
Other Useful Business Software
Our Free Plans just got better! | Auth0
You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
Rate This Project
Login To Rate This Project
User Reviews
-
***A*W*E*S*O*M*E***
-
Wow, this crawler has it all. It is - even with a point zero release faster and more mature and feature rich than every other I tried. Especially useful is the test interface, where you can try out all the parameters, without coding. This is excellent work!
-
Great tool to crawl sites, excelent support
-
Could use a "Per domain page-limit" :)
-
Very good job. Hard to find better!