SmartCrawler

SmartCrawler is a java-based fully configurable, multi-threaded and extensible crawler, which is able to fetch and analyze the contents of a web site by using dinamically pluggable filters

Project Activity

See All Activity >

License

GNU General Public License version 2.0 (GPLv2)

Follow SmartCrawler

SmartCrawler Web Site

nel_h2

Gen AI apps are built with MongoDB Atlas

Build gen AI apps with an all-in-one modern database: MongoDB Atlas

MongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.

Start Free

Rate This Project

User Reviews

Be the first to post a review of SmartCrawler!

Additional Project Details

Languages

English

Intended Audience

Developers, End Users/Desktop

User Interface

Command-line

Programming Language

Java

Related Categories

Java Search Engines

Registered

2005-06-14

Similar Business Software

Deepfind

Deepfind is an innovative search engine and content analysis platform that leverages advanced AI technologies, focusing on deep learning and natural language processing. Search Capabilities: Unlike traditional search engines, Deepfind allows users to perform searches using natural language...

See Software
Yahoo Search

Yahoo dials up what people love most with apps and sites like Yahoo Sports, Yahoo Finance, Yahoo Fantasy, Yahoo Mail and more in the works everyday. We reach millions of people around the globe and connect advertisers with just the right audiences to help build their brands and businesses. Yahoo...

See Software
Infospace

InfoSpace is a leading provider of white label search and monetization solutions. We serve a network of more than 100 web publishers globally, offering core search solutions as a proven means to monetize audience via a search-based advertising revenue stream. Our technology selects from the top...

See Software
Perplexity

Where knowledge begins. Perplexity is an AI search engine that gives you quick answers. Available for free at as a web app, desktop app, or on the go on iPhone or Android. Perplexity AI is an advanced search and question-answering tool that leverages large language models to provide...

See Software
Dekoo

Dekoo is the Asia's independent global search engine company offering Unique search, Unbiased results on the web having its own crawler, indexing, search and ranking algorithms along with AI/ML technologies with Tier 4 Datacenters and Content Delivery Network around the world. It aims to...

See Software
Google

Our mission is to organize the world's information so that everyone can access and use it. Every time you search, there are thousands, sometimes millions, of webpages with helpful information. How Google figures out which results to show starts long before you even type, and is guided by a...

See Software