A Python library for automating interaction with websites
The easiest way to parse and modify URLs in Python
Distributed Crawler Management Framework Based on Scrapy
Twitter Intelligence OSINT project performs tracking and analysis
A powerful Spider(Web Crawler) system in Python
Python Web Crawler Library