Search Results for "crawler php"

Crawler Detect is a PHP library that detects bots, crawlers, and spiders by analyzing user-agent headers and comparing them against a constantly updated list of known crawlers. It's useful for analytics, rate-limiting, or displaying alternative content for automated tools. It is fast, lightweight, and easy to integrate into any PHP application.

Spatie Crawler is a PHP library that allows developers to crawl websites and extract information efficiently. It can be used for web scraping, link checking, or automated testing of web pages. The library is simple to use and supports customizable crawling strategies, including controlling crawl depth and handling redirects. It’s suitable for building crawlers that navigate large or dynamically generated websites.

github: https://github.com/samedog/phpmvs

OpenSearchServer is a powerful, enterprise-class, search engine program. Using the web user interface, the crawlers (web, file, database, etc.) and the client libraries (REST/API , Ruby, Rails, Node.js, PHP, Perl) you will be able to integrate quickly and easily advanced full-text search capabilities in your application: Full-text with basic semantic, join queries, boolean queries, facet and filter, document (PDF, Office, etc.) indexation, web scrapping,etc. OpenSearchServer runs on...

OpenWebSpider is an Open Source multi-threaded Web Spider (robot, crawler) and search engine with a lot of interesting features!

IOSEC PHP HTTP FLOOD PROTECTION ADDONS IOSEC is a php component that allows you to simply block unwanted access to your webpage. if a bad crawler uses to much of your servers resources iosec can block that. IOSec Enhanced Websites: https://www.artikelschreiber.com/en/ https://www.unaique.net/en/ https://www.unaique.com/ https://www.artikelschreiber.com/marketing/ https://www.paraphrasingtool1.com/ https://www.artikelschreiben.com/ https://www.buzzerstar.com/ https://iosecaddons.sourceforge.io/ https://www.artikelschreiber.com/opensource/ https://www.sebastianenger.com/ Added Setup Instructions for Wordpress Content Management System. ...

Download search engine and directory with Rapidshare and Torrent - zoozle Download Suchmaschine All The files that run the World Leading German Download Search Engine in 2010 with 500 000 unique visitors a day - all the tools you need to set up a clone. Code Contains: - PHP Files for zoozle - Perl Crawler for gathering new content to database and all other cool tools i have created https://www.artikelschreiber.com/en/ https://www.unaique.net/en/ https://www.unaique.com/ https://www.unaique.de/ https://www.openinsider.de/ https://www.artikelschreiber.com/ https://zoozle.sourceforge.io/ https://www.artikelschreiben.com/ https://www.artikelschreiber.com/opensource/ It was very successful and 2008/2009 one of the biggest German download engines! ...

TubeKit is a toolkit for creating YouTube crawlers. It allows one to build one's own crawler that can crawl YouTube based on a set of seed queries and collect up to 17 different attributes.

Ani-Shell is a PHP remote shell, basically used for remote access and security pen testing. Ani-Shell provides a robust and a basic interface to access the file system, do some networking tweaks and even test your server for some common security vulnerabilities. The developer has tried to follow a coding standard which makes the code a little clean and easier to understand, Note: How you use this shell is exactly on you, and author pays no responsibility for what you use it for and what may...

Ex-Crawler is divided into 3 subprojects (Crawler Daemon, distributed gui Client, (web) search engine) which together provide a flexible and powerful search engine supporting distributed computing. More informations: http://ex-crawler.sourceforge.net

** Guys I have built a much more powerful Fully Featured CMS system at: https://github.com/MacdonaldRobinson/FlexDotnetCMS Macs CMS is a Flat File ( XML and SQLite ) based AJAX Content Management System. It focuses mainly on the Edit In Place editing concept. It comes with a built in blog with moderation support, user manager section, roles manager section, SEO / SEF URL

Discontinued lightweight Desktop-Files/SMB/FTP crawler and search engine.

bee-rain is a web crawler that harvest and index file over the network. You can see result by bee-rain website : http://bee-rain.internetcollaboratif.info/

APC Anti Crawler is a php5 class based on APC which can be used to limit the amount of http request per IP. It stop web crawler to download your entire website.

PHP Crawler is a simple website search script for small-to-medium websites. The only requrements are PHP and MySQL, no shell access required.

PK-Torrents is a PHP torrent lister based on torrenthoster v1.0. It has the functionality to crawl the top torrent sites, Meganova, Mininova, Piratebay, Snarf, Torrentportal, Torrentspy.

SNT is a search engine for SMB and FTP shares with crawler running on Win32. Web interface is provided for searching files and browsing shares contents. Also provided shared films list with users rates and comments.

Larbin is a Web crawler intended to fetch a large number of Web pages, it should be able to fetch more than 100 millions pages on a standard PC with much u/d. This set of PHP and Perl scripts, called webtools4larbin, can handle the output of Larbin and p

Fast File Search is a crawler of FTP servers and SMB shares (Windows shares and UNIX systems running Samba). WWW interface is provided for searching files. FFS is similar to FemFind but optimized for speed.

The Jobcrawler search engine is a research project in order to index the available applications on the internet. Our mission is to really help people who seek a job or employee on a one to one basis and rule mediators (job agencies) out.

MyNewsGroups :) is a Web based USENET news crawler, news reader and news poster. With the use of a DB backend, the crawler fetch the newsgroups messages ONCE only. Web based environment, SPAM Filters, Search Engine, Subscriptions and much more.

FTPList/FTPCrawler is a Multi-threaded and MySQL based FTP Crawler, it has a PHP interface which you can use to search in the database, see FTP status (up, full, down), and more. It's designed for enviorements like big LAN-Partys i.e Remedy.

studiMaps is a web based application for visualization and analysis of social networks. It consists of two software components: a web-crawler for getting data and the web based application for visualization.

FTP Scanner, FTP Crawler and web-based interface to search files and browse FTP-servers through database index (offline-browsing). Originally designed for ethernet network segments with anonymously accessing FTP servers to easily find needle files.