Search Results for "website spider"

Free website monitoring software, easy to set up and use for monitoring web sites. It is a web application programmed in Java programming language. You can monitor HTML pages, JSON and XML, pages in sitemap and even your whole web site using spider. Naturally you can check multiple websites. You can check HTTP result codes and even contents of the checked pages. Website checking is done periodically using build-in cron mechanism. In case of a check failure, application will automatically...

More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation...

SQLSentinel is an opensource tool that automates the process of finding the sql injection on a website. SQLSentinel includes a spider web and sql errors finder. You give in input a site and SQLSentinel crawls and try to exploit parameters validation error for you. When job is finished, it can generate a pdf report which contains the url vuln found and the url crawled. Please remember that SQLSentinel is not an exploiting tool. It can only finds url Vulnerabilities SQLSentinel official...

Sitemap Creator is a PHP class which creates XML sitemaps files compatible with the standard sitemaps.org protocol supported by Google and Bing. Features Uses PHPCrawl class to crawl/spider the website and creates URLs set while all PHPCrawl methods and options are accessible through class. Ability to calculate Priority, Frequency and Last-Modified date with variety of options. Creates sitemaps in gzip format or uncompressed XML. Pings search engines with sitemaps locations. Reads from...

This project will provide a tool for users to get a better understanding of the content and structure of an existing website. It will do this by providing a customised web spider as well as extensions to the GUESS graph visualisation application.

This CGI program can trap malicious robots that spider your website. The program works by blocking access from the bot's ip address. It can also provide an unlimited number of false e-mail addresses to muck up databases of email harvesting bots.

Sperowider Website Archiving Suite is a set of Java applications, the primary purpose of which is to spider dynamic websites, and to create static distributable archives with a full text search index usable by an associated Java applet.

phpSitemapNG is a free Google Sitemaps generator written in PHP, but also generates RSS-based, txt-based and HTML-based sitemap files. It will spider your website and can also index the filesystem.