Spidertron webcrawler API download

Spidertron is a multithreaded web crawling API for web sites of moderate size (hundreds of thousands of pages) that allows you to focus not on the crawling but on processing of the information retreived.

Project Activity

See All Activity >

License

GNU General Public License version 2.0 (GPLv2)

Follow Spidertron webcrawler API

Spidertron webcrawler API Web Site

Other Useful Business Software

Custom VMs From 1 to 96 vCPUs With 99.95% Uptime

General-purpose, compute-optimized, or GPU/TPU-accelerated. Built to your exact specs.

Live migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.

Try Free

Rate This Project

User Reviews

Be the first to post a review of Spidertron webcrawler API!

Additional Project Details

Intended Audience

Advanced End Users, Developers

Programming Language

Java

Related Categories

Java Search Engines

Registered

2005-03-05

Similar Business Software

Perplexity

Perplexity is an AI-powered search and answer engine designed to provide accurate, real-time information. It combines natural language processing with web search to deliver concise and reliable answers. Users can ask questions conversationally and receive responses backed by cited sources. The...

See Software
Perplexity Pro

Perplexity Pro is the most powerful way to search the internet with unlimited Pro Search, upgraded AI models, unlimited file upload, image generation, and API credits. Perplexity Pro is a premium offering from the Perplexity AI platform, designed to provide users with a more advanced and...

See Software
Google

Our mission is to organize the world's information so that everyone can access and use it. Every time you search, there are thousands, sometimes millions, of webpages with helpful information. How Google figures out which results to show starts long before you even type, and is guided by a...

See Software
Yahoo Search

Yahoo dials up what people love most with apps and sites like Yahoo Sports, Yahoo Finance, Yahoo Fantasy, Yahoo Mail and more in the works everyday. We reach millions of people around the globe and connect advertisers with just the right audiences to help build their brands and businesses. Yahoo...

See Software
MyAllSearch

MyAllSearch is a privacy search engine that helps you to protect your privacy online. We believe in a search experience that doesn't result in an invasion of your online privacy. While other search engines store every search you ever type, we hold your privacy above everything else. We know...

See Software
Luxxle

Luxxle is a next-generation search engine that prioritizes privacy, quality, and real-time results without tracking users or storing their data. It uses real-time crawling and indexing, ensuring the most up-to-date and relevant results, unlike search engines relying on outdated databases. Luxxle...

See Software