Multiple server/webapp vulnerability scanner
Distributed high-concurrency crawler software written in pure golang
Magnet link aggregation search
BotSlayer Community Edition
WebSploit is a high level MITM Framework
The next web scraper, see through the <html> noise
Python ShadowSocks framework
A powerful Spider(Web Crawler) system in Python
Open source web crawler for Java
An open source search engine with RESTFul API and crawlers
Perl Web Scraping Project
File system crawler and disk space usage software
Catberry is an isomorphic framework
a java-based crawler
get web page. include html、css and js files
Web Crawler/Spider for NodeJS + server-side jQuery
WebCollector is an open source web crawler framework based on Java.
IOSec Addons are enhancements for web security and crawler detection
Modular web site spider for web developers.
Zoozle 2008 - 2010 Webpage, Tools and SQL Files