Search Results for "web crawler source code"

LittleLink Custom is an open source self-hosted alternative to services like Linktree and many.link. LittleLink Custom allows you to link all your social media platforms easily accessible on one page. No coding or command line knowledge required. Just easy plug and play.

...This application is open source and free, and is only used for crawler technology exchange and learning. The search results are all from the source site, and no responsibility is assumed. The project complies with GNU General Public License v3.0. Online playback is performed in conjunction with the webtorrent desktop version. It needs to be downloaded separately.

PHP Server Monitor is a script that checks whether your websites and servers are up and running. It comes with a web based user interface where you can manage your services and websites, and you can manage users for each server with a mobile number and email address. For services a connection will be made to the entered ip or domain, on the given port. This way you can check if certain services on your machine are still running. To check your IMAP service for example, enter port...

More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation...

Checks given webpages for backlinks and scans for image links and keywords. Uses DOM-based methods to scan for backlinks, which are more sophisticated than simple text scanning (one example: they ignore commented out source code).

ZeroSearch World Wide Web it's a crawler that found and download all file in site we insert to start the search. See all image, video and other to your preferite site or create your personal internet database to found news or information.

A C# coded web crawler that analyzes pagerank, total links, no follow links, and make a ranking for their best-positioned pages in Google

This is simple link checker. It can crawl any site and help to find broken links. It also having download CSV report option.The CSV file includes url ,parent page url and status of page [broken or ok]. It is be very useful for search engine optimization.

404SEF is a component for Mambo CMS (4.5.x right now, 4.6.x soon) to provide Human Readable URLs. Works with apache and IIS. Provides proper 404 status code for missing content, logs 404 errors, and user-defined custom redirection via special shortcuts

RoboWeb is set of tools to automate the creation of test suites for any kind of web sites and web applications. RoboWeb uses a proxy recorder to capture user actions as she navigates a site, which later can be reproduced automatically. Site responses

Harvest is a web indexing package, originally disigned for distributed indexing, it can form a powerful system for indexing both large and small web sites. Also now includes Harvest-NG a highly efficient, modular, perl-based web crawler.

Framework for Java code generation of web applications is for the above framework mentawai, done to expedite the process of creating a project in Java technologies with the JPA, Hibernate, Mentawai, AJAX with DWR among others.