Spider Files

High-performance Rust web crawler and scraper for large-scale data

This is an exact mirror of the Spider project, hosted at https://github.com/spider-rs/spider. SourceForge is not affiliated with Spider.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
README.md	2026-10-30	162 Bytes	0
v2.53.6 source code.tar.gz	2026-10-30	1.2 MB	0
v2.53.6 source code.zip	2026-10-30	1.4 MB	0
Totals: 3 Items		2.7 MB	0

Commits since v2.53.5

fix(agent): utf-8 safe head truncation + drop unused openai deps (v2.53.6) (8ec195da)

Crates: https://crates.io/crates/spider/2.53.6

Source: README.md, updated 2026-10-30

Other Useful Business Software

Build Agents and Models on One Platform

Everything you need to build production-ready agents and models. Access 200+ Google and third-party AI models and tools.

Gemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.

Start Free

One Monitoring Tool for IT, OT and Cloud | Free Trial

Vendor-agnostic monitoring across on-prem servers, cloud platforms and OT devices, all in one dashboard. No more tool sprawl.

Modern infrastructure spans data centers, cloud platforms and factory floors, and every blind spot between them is a risk. PRTG supports SNMP, WMI, SSH and other standard protocols to monitor IT, OT and hybrid environments through one customizable dashboard. Build the views your team needs, from network health to application performance, without switching tools. Try PRTG free for 30 days now.

Try PRTG Free

Build Agents and Models on One Platform

Everything you need to build production-ready agents and models. Access 200+ Google and third-party AI models and tools.

Start Free

Recommended Projects

Abot
Fast and flexible C# framework for building customizable web crawlers
GoSpider
Gospider - Fast web spider written in Go
watercrawl
AI-ready web crawler that extracts and structures website content
grab-site
Web crawler for archiving and backing up sites into WARC archives
Firecrawl
Turn entire websites into LLM-ready markdown or structured data