Search Results for "spider"

...The spider is a CLI tool and must be added as a cronjob. In a web based backend you can control all data and analyze your data. You can handle multiple websites in the same backend. PHP 7 or 8 + PDO (Mysql/ Sqlite)

...Tasks are scheduled by the task scheduler module in the master node, and received by the task handler module in worker nodes, which executes these tasks in task runners. Task runners are actually processes running spider or crawler programs, and can also send data through gRPC (integrated in SDK) to other data sources, e.g. MongoDB.

OpenWebSpider is an Open Source multi-threaded Web Spider (robot, crawler) and search engine with a lot of interesting features!

This is fuskit A Fusker clone it's a working clone but it needs to optimized for todays technology so please submit your optimized versions so we can make fuskit better

Sitemap Creator is a PHP class which creates XML sitemaps files compatible with the standard sitemaps.org protocol supported by Google and Bing. Features Uses PHPCrawl class to crawl/spider the website and creates URLs set while all PHPCrawl methods and options are accessible through class. Ability to calculate Priority, Frequency and Last-Modified date with variety of options. Creates sitemaps in gzip format or uncompressed XML. Pings search engines with sitemaps locations. Reads from CSV files and exports entries in CSV format.

PGbuild is a Phonegap development system that automates the development process by connecting your CMS/web server with the online service [Phonegap Build](http://build.phonegap.com). PGBuild is essentially a web spider that make off-line versions of web pages. The off-line version is zippped and send to the Phonegap Build service. The spider is controlled by a project file that sets the rules for the spider and the options for the phonebap build service. You may create and manage your phonegap project source files manually on your webserver or use PGBuild to connect to a CMS system to extract content. ...

Bigforum is a free message board in German. Bigforum ist eine kostenlose Foren-Software. Man benötigt keine Vorkenntnisse, die Codes lassen sich leicht bearbeiten und es gibt ständig neue Funktionen.

Graidle is a project that is proposed to create a web application to generate graph diagrams. There are several classes specialized in generating histograms(horizontal),line(filled) pie and spider charts.

Crawler is a bare-bones spider designed to quickly and effectively build an index of all files and pages on a given Web site as well as the link relationship (both incoming and outgoing) between each page. More open source at https://github.com/fcc.

NZBGetter is a PHP Script for linux based systems to spider NZB index sites for NZB files matching your predefined search patterns. The script downloads matching NZB files and passes them to your Usenet Reader.

WEB-PA is a spider indexing a set of web sites for collecting statistics. It is currently being run on sites of Italian Public Administrations and studies/reports on their WWW standards compliance, but it can serve for a number of purposes.

An web-based application for active network devices record creation and SNMP management of it.

It's multi protocol web chat with using modern web technologies such as ajax,css2(3). It also supports connect to IRC(based on SmartIRC) and Jabber Conference and may be will supports some other networks

Funnel is a project for use on intranets, or selected sites on the Internet to gather together and index information from several different sources and make it available through a sane, usable interface.

The Spider Web Network Xoops Module Team provides modules for the Xoops community written in the PHP coding language. We develop mods and or take existing php script and port it into the Xoops format. High quality mods is our goal.

OpenAnonymity consists of a module for apache 2.0 Webserver and a framework that enables you to control search engine spider indexing on a word level, contrary to on file level as in Robots exclusion. OA could force Spiders to follow this rules.

Castor is a php tool for webmaster to submit an url on multiple search engines or repositories through a web interface.

phpSitemapNG is a free Google Sitemaps generator written in PHP, but also generates RSS-based, txt-based and HTML-based sitemap files. It will spider your website and can also index the filesystem.