Distributed Crawler Management Framework Based on Scrapy
Library for Rapid (Web) Crawler and Scraper Development
A Python library for automating interaction with websites
An extension that allows for hassle-free website citation/referencing.
Web Scraper in Go, similar to BeautifulSoup
Twitter Intelligence OSINT project performs tracking and analysis
A powerful Spider(Web Crawler) system in Python
Ever wanted to download only a part of a Git repository.
WebCollector is an open source web crawler framework based on Java.
anonymous tools [uncontinued]
Generate wordlists using different methods
Python Web Crawler Library