Search Results for "website spider"
Sort By:
Showing 11 open source projects for "website spider"
View related business solutions-
For minor company expenses, you might utilize a spend management solution or track everything in spreadsheets. For everything more, you'll need Precoro. We help companies achieve procurement excellence and budget efficiency by building transparent, predictable, automated spending workflows.
-
ManageEngine's Endpoint Central is a Unified Endpoint Management Solution, that takes care of enterprise mobility management (including all features of mobile application management and mobile device management), as well as client management for a diversified range of endpoints - mobile devices, laptops, computers, tablets, server machines etc. With ManageEngine Endpoint Central, users can automate their regular desktop management routines like distributing software, installing patches, managing IT assets, imaging and deploying OS, and more.
-
1
ahCrawler
A PHP search engine for your website and web analytics tool. GNU GPL3
ahCrawler is a set to implement your own search on your website and an analyzer for your web content. It can be used on a shared hosting. It consists of * crawler (spider) and indexer * search for your website(s) * search statistics * website analyzer (http header, short titles and keywords, linkchecker, ...) You need to install it on your own server. So all crawled data stay in your environment. You never know when an external webspider updated your content. Trigger a rescan... -
2
Grab Framework Project
Web Scraping Framework
Grab is a python framework for building web scrapers. With Grab you can build web scrapers of various complexity, from simple 5-line scripts to complex asynchronous website crawlers processing millions of web pages. Grab provides an API for performing network requests and for handling the received content e.g. interacting with DOM tree of the HTML document. The single request/response API that allows you to build network request, perform it and work with the received content. The API is built... -
3
Site monitoring
Monitoring of websites with spider and email notifications
Free website monitoring software, easy to set up and use for monitoring web sites. It is a web application programmed in Java programming language. You can monitor HTML pages, JSON and XML, pages in sitemap and even your whole web site using spider. Naturally you can check multiple websites. You can check HTTP result codes and even contents of the checked pages. Website checking is done periodically using build-in cron mechanism. In case of a check failure, application will automatically... -
4
sitecheck
Modular web site spider for web developers.
More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation... -
Chat with your business data with Looker. More than just a modern business intelligence platform, you can turn to Looker for self-service or governed BI, build your own custom applications with trusted metrics, or even bring Looker modeling to your existing BI environment.
-
5
Sitemap Creator is a PHP class which creates XML sitemaps files compatible with the standard sitemaps.org protocol supported by Google and Bing. Features Uses PHPCrawl class to crawl/spider the website and creates URLs set while all PHPCrawl methods and options are accessible through class. Ability to calculate Priority, Frequency and Last-Modified date with variety of options. Creates sitemaps in gzip format or uncompressed XML. Pings search engines with sitemaps locations. Reads from...
-
6
SEO Tools provides tools for search engine optimization. Spider SEO: generates meta tags for all pages of a website. Web Page Optimizer performs analysis and changes in a web page. A graphical user interface is linked with a scriptol program that gene
-
7
This project will provide a tool for users to get a better understanding of the content and structure of an existing website. It will do this by providing a customised web spider as well as extensions to the GUESS graph visualisation application.
-
8
This CGI program can trap malicious robots that spider your website. The program works by blocking access from the bot's ip address. It can also provide an unlimited number of false e-mail addresses to muck up databases of email harvesting bots.
-
9
Sperowider Website Archiving Suite is a set of Java applications, the primary purpose of which is to spider dynamic websites, and to create static distributable archives with a full text search index usable by an associated Java applet.
-
Evolia is a web and mobile platform that connects enterprises with 1000’s of local shift workers and offers free workforce scheduling and time and attendance solutions. Is your business on Evolia?
-
10
Robust featureful multi-threaded CLI web spider using apache commons httpclient v3.0 written in java. ASpider downloads any files matching your given mime-types from a website. Tries to reg.exp. match emails by default, logging all results using log4j.
-
11
phpSitemapNG is a free Google Sitemaps generator written in PHP, but also generates RSS-based, txt-based and HTML-based sitemap files. It will spider your website and can also index the filesystem.
- Previous
- You're on page 1
- Next