Download
Geziyor is a high-performance web crawling and web scraping framework built for the Go programming language. It is designed to help developers crawl websites and extract structured information from web pages efficiently. It focuses on speed and scalability, allowing large numbers of requests to be processed concurrently. Geziyor supports use cases such as data mining, monitoring web content, and automated testing workflows. It provides a flexible architecture where developers define parsing functions that process responses and extract the desired data. Geziyor includes features for managing requests, handling cookies, respecting robots rules, and exporting collected data in multiple formats. With built-in tools for caching, metrics collection, and proxy management, it enables developers to build robust and customizable scraping systems using Go.
Features
- High-speed crawling capable of handling thousands of requests per second
- JavaScript rendering support for scraping dynamic web pages
- Built-in caching options using memory, disk, or LevelDB
- Automatic data exporting to formats such as JSON or CSV
- Concurrency control with global and per-domain request limits
- Proxy management with single, round-robin, or custom strategies
Project Samples
