This is the most powerful software taking into account CIS location
Python & command-line tool to gather text on the Web
Distributed Crawler Management Framework Based on Scrapy
A service daemon to run Scrapy spiders
Creating Scrapy scrapers via the Django admin interface
Twitter Intelligence OSINT project performs tracking and analysis
A powerful Spider(Web Crawler) system in Python