Free batch downloader for image, wallpaper, video, audio, document,
Multiple server/webapp vulnerability scanner
Distributed high-concurrency crawler software written in pure golang
Magnet link aggregation search
Distributed crawler powered by Headless Chrome
A powerful Spider(Web Crawler) system in Python
An open source search engine with RESTFul API and crawlers
File system crawler and disk space usage software
IOSec Addons are enhancements for web security and crawler detection
Arch Crawler is a pre-configured install of Arch Linux
Python Web Crawler Library