Crawler.NET is a component-based distributed framework for web traversal intended for the .NET platform. It comprises of loosely coupled units each realizing a specific web crawler task. The main design goals are efficiency and flexibility.
License
GNU General Public License version 2.0 (GPLv2)Follow Crawler.NET
Other Useful Business Software
Fully Managed MySQL, PostgreSQL, and SQL Server
Cloud SQL handles your database ops end to end, so you can focus on your app.
Rate This Project
Login To Rate This Project
User Reviews
-
Webcrawler is wonderful! Thanks.