Search Results for "linux spider solitaire"

ahCrawler is a set to implement your own search on your website and an analyzer for your web content. It can be used on a shared hosting. It consists of * crawler (spider) and indexer * search for your website(s) * search statistics * website analyzer (http header, short titles and keywords, linkchecker, ...) You need to install it on your own server. So all crawled data stay in your environment. You never know when an external webspider updated your content. Trigger a rescan...

OpenWebSpider is an Open Source multi-threaded Web Spider (robot, crawler) and search engine with a lot of interesting features!

Crawler is a bare-bones spider designed to quickly and effectively build an index of all files and pages on a given Web site as well as the link relationship (both incoming and outgoing) between each page. More open source at https://github.com/fcc.

WEB-PA is a spider indexing a set of web sites for collecting statistics. It is currently being run on sites of Italian Public Administrations and studies/reports on their WWW standards compliance, but it can serve for a number of purposes.

Funnel is a project for use on intranets, or selected sites on the Internet to gather together and index information from several different sources and make it available through a sane, usable interface.

OpenAnonymity consists of a module for apache 2.0 Webserver and a framework that enables you to control search engine spider indexing on a word level, contrary to on file level as in Robots exclusion. OA could force Spiders to follow this rules.

Castor is a php tool for webmaster to submit an url on multiple search engines or repositories through a web interface.

phpSitemapNG is a free Google Sitemaps generator written in PHP, but also generates RSS-based, txt-based and HTML-based sitemap files. It will spider your website and can also index the filesystem.