PHPCrawl is a high configurable webcrawler/webspider-library written in PHP. It supports filters, limiters, cookie-handling, robots.txt-handling, multiprocessing and much more.
License
GNU General Public License version 2.0 (GPLv2)Follow PHPCrawl
Other Useful Business Software
$300 Free Credits to Build on Google Cloud
Start your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
Rate This Project
Login To Rate This Project
User Reviews
-
***A*W*E*S*O*M*E***
-
Wow, this crawler has it all. It is - even with a point zero release faster and more mature and feature rich than every other I tried. Especially useful is the test interface, where you can try out all the parameters, without coding. This is excellent work!
-
Great tool to crawl sites, excelent support
-
Could use a "Per domain page-limit" :)
-
Very good job. Hard to find better!