Open source file indexing & storage analytics powered by Elasticsearch
Lightweight .NET framework for fast web crawling and data scraping
A fast, high-level web crawling and web scraping framework
Python HTTP client with TLS and HTTP/2 fingerprint emulation support
Open source enterprise search server for websites, files, and data
Python tool for crawling and extracting structured data from news site
Collection of Python ecommerce and website crawler examples projects
Distributed proxy IP pool for web crawlers using Scrapy and Redis