Spidertron webcrawler API download

Spidertron is a multithreaded web crawling API for web sites of moderate size (hundreds of thousands of pages) that allows you to focus not on the crawling but on processing of the information retreived.

Project Activity

See All Activity >

License

GNU General Public License version 2.0 (GPLv2)

Follow Spidertron webcrawler API

Spidertron webcrawler API Web Site

Other Useful Business Software

Gen AI apps are built with MongoDB Atlas

The database for AI-powered applications.

MongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.

Start Free

Rate This Project

User Reviews

Be the first to post a review of Spidertron webcrawler API!

Additional Project Details

Intended Audience

Advanced End Users, Developers

Programming Language

Java

Related Categories

Java Search Engines

Registered

2005-03-05

Similar Business Software

CoRepo

CoRepo is a web search engine enriched with company data to let you filter your results better and find the right companies faster. We crawl the web and identify all websites which are associated with an incorporated and actively running company while filtering out all the content websites. The...

See Software
YaCy

YaCy is free software for your own search engine. Join a community of search engines or make your own search portal! There are these three use cases you can choose from: Web Search by the people, for the people: decentralized, all users are equal, no central, no search request storage, shared...

See Software
Perplexity

Where knowledge begins. Perplexity is an AI search engine that gives you quick answers. Available for free at as a web app, desktop app, or on the go on iPhone or Android. Perplexity AI is an advanced search and question-answering tool that leverages large language models to provide...

See Software
Yahoo Search

Yahoo dials up what people love most with apps and sites like Yahoo Sports, Yahoo Finance, Yahoo Fantasy, Yahoo Mail and more in the works everyday. We reach millions of people around the globe and connect advertisers with just the right audiences to help build their brands and businesses. Yahoo...

See Software
Google

Our mission is to organize the world's information so that everyone can access and use it. Every time you search, there are thousands, sometimes millions, of webpages with helpful information. How Google figures out which results to show starts long before you even type, and is guided by a...

See Software
Dekoo

Dekoo is the Asia's independent global search engine company offering Unique search, Unbiased results on the web having its own crawler, indexing, search and ranking algorithms along with AI/ML technologies with Tier 4 Datacenters and Content Delivery Network around the world. It aims to...

See Software