Ex-Crawler

Ex-Crawler is divided into 3 subprojects (Crawler Daemon, distributed gui Client, (web) search engine) which together provide a flexible and powerful search engine supporting distributed computing. More informations: http://ex-crawler.sourceforge.net

Features

Fast web and image crawler written in Java which follows new links automaticly on a priority algorithm and can run 24h a day.
Uses MySQL, MSSQL, PostgreSQL or even SQLlite Database to store informations
Comes with a web frontend search engine, direct website analysis and more (in development - SVN)
Support for distributed crawling with a graphical frontend for non-advanced users (de.excrawler.distributed.client in svn)
Comes with it's own server where you can add new sites and much more including UserManagement and more.
Comes with an plugin interface, with it you can easily add your own features in no time
And much more

Project Samples

Ex-Crawler Distributed GUI Client with first run assistant

Overview over some of the server commands in telnet session

Ex-Crawler Search Engine screenshot (PHP, in Browser)

Project Activity

See All Activity >

License

GNU General Public License version 3.0 (GPLv3)

Follow Ex-Crawler

Ex-Crawler Web Site

Other Useful Business Software

Gen AI apps are built with MongoDB Atlas

The database for AI-powered applications.

MongoDB Atlas is the developer-friendly database used to build, scale, and run gen AI and LLM-powered apps—without needing a separate vector database. Atlas offers built-in vector search, global availability across 115+ regions, and flexible document modeling. Start building AI apps faster, all in one place.

Start Free

Rate This Project

User Ratings

5.0 out of 5 stars

★★★★★

★★★★

★★★

★★

★

ease 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5

features 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5

design 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5

support 1 of 5 2 of 5 3 of 5 4 of 5 5 of 5 0 / 5

User Reviews

Filter Reviews:

All

marcophoenix Posted 2013-06-20

Ex-crawler is good! Thanks.

1 user found this review helpful.

Additional Project Details

Operating Systems

Linux, Mac, Solaris, Windows

Languages

English

Intended Audience

Advanced End Users, Developers, Education, End Users/Desktop, Science/Research, System Administrators

User Interface

Command-line, Java Swing, Non-interactive (Daemon), Web-based

Programming Language

Java, PHP

Database Environment

JDBC, Microsoft SQL Server, MySQL, PostgreSQL (pgsql), SQLite

Related Categories

PHP Search Engines, PHP Distributed Computing Software, PHP Information Analysis Software, Java Search Engines, Java Distributed Computing Software, Java Information Analysis Software

Registered

2010-03-02

Similar Business Software

Toutiao

Today Toutiao is a general information platform dedicated to connecting people and information, allowing high-quality and rich information to be efficiently and accurately distributed, and promoting information to create value. Today’s Toutiao currently has a variety of distribution methods such...

See Software
Dekoo

Dekoo is the Asia's independent global search engine company offering Unique search, Unbiased results on the web having its own crawler, indexing, search and ranking algorithms along with AI/ML technologies with Tier 4 Datacenters and Content Delivery Network around the world. It aims to...

See Software
Infospace

InfoSpace is a leading provider of white label search and monetization solutions. We serve a network of more than 100 web publishers globally, offering core search solutions as a proven means to monetize audience via a search-based advertising revenue stream. Our technology selects from the top...

See Software
Gigablast

Gigablast is a search engine designed to provide users with efficient and fast search results. It aggregates data from various sources, giving access to web pages, images, and more, allowing for a comprehensive search experience. Gigablast is particularly focused on simplicity and speed,...

See Software
YaCy

YaCy is free software for your own search engine. Join a community of search engines or make your own search portal! There are these three use cases you can choose from: Web Search by the people, for the people: decentralized, all users are equal, no central, no search request storage, shared...

See Software
Dogpile

InfoSpace created the Dogpile search engine because your time is important to us. Powered by Metasearch technology, Dogpile returns all the best results from leading search engines including Google and Yahoo!, so you find what you’re looking for faster. Each search engine has its own method of...

See Software