Ex-Crawler is divided into 3 subprojects (Crawler Daemon, distributed gui Client, (web) search engine) which together provide a flexible and powerful search engine supporting distributed computing. More informations: http://ex-crawler.sourceforge.net
- Fast web and image crawler written in Java which follows new links automaticly on a priority algorithm and can run 24h a day.
- Uses MySQL, MSSQL, PostgreSQL or even SQLlite Database to store informations
- Comes with a web frontend search engine, direct website analysis and more (in development - SVN)
- Support for distributed crawling with a graphical frontend for non-advanced users (de.excrawler.distributed.client in svn)
- Comes with it's own server where you can add new sites and much more including UserManagement and more.
- Comes with an plugin interface, with it you can easily add your own features in no time
- And much more
Ex-crawler is good! Thanks.