A fast, high-level web crawling and web scraping framework
Easily turn large sets of image urls to an image dataset
dude uncomplicated data extraction: A simple framework
Easy Spider is a distributed Perl Web Crawler Project from 2006
Automate the download of entire Twitch.tv channels
Finds all the security information for a given domain name
Open source Search Engine and Enterprise Search
Python Web Crawler Library