Search Results for "website crawler"
Sort By:
Showing 29 open source projects for "website crawler"
View related business solutions-
Stop messing with tools that aren’t designed to amplify volunteer programs. With VolunteerMatters, it’s a delight to manage everything in one place.
-
BrightGauge, a ConnectWise solution, was started in 2011 to fill a missing need in the small-to-medium IT Services industry: a better way to manage data and provide the value of work to clients. BrightGauge Software allows you to display all of your important business metrics in one place through the use of gauges, dashboards, and client reports. Used by more than 1,800 companies worldwide, BrightGauge integrates with popular business solutions on the market, like ConnectWise, Continuum, Webroot, QuickBooks, Datto, IT Glue, Zendesk, Harvest, Smileback, and so many more. Dig deeper into your data by adding, subtracting, multiplying, and dividing one metric against another. BrightGauge automatically computes these formulas for you. Want to show your prospects how quick you are to respond to tickets? Show off your data with embeddable gauges on public sites.
-
1
SiteOne Crawler
SiteOne Crawler is a website analyzer and exporter
SiteOne Crawler is a very useful and easy-to-use tool you'll ♥ as a Dev/DevOps, website owner or consultant. Works on all popular platforms - Windows, macOS, and Linux (x64 and arm64 too). It will crawl your entire website in depth, analyze and report problems, show useful statistics and reports, generate an offline version of the website, generate sitemaps, or send reports via email. Watch a detailed video with a sample report for Astro. build website. This crawler can be used as a command... -
2
crwlr
Library for Rapid (Web) Crawler and Scraper Development
This library provides kind of a framework and a lot of ready-to-use, so-called steps, that you can use as building blocks, to build your own crawlers and scrapers with. Before diving into the library, let's have a look at the terms crawling and scraping. For most real-world use cases, those two things go hand in hand, which is why this library helps with and combines both. A (web) crawler is a program that (down)loads documents and follows the links in it to load them as well. A crawler could... -
3
DocSearch
The easiest way to add search to your documentation
... with the interaction patterns of each OS. We scrape your documentation or technical blog, configure the Algolia application and send you the snippet you'll have to integrate. It's that simple. You don't need to configure any settings or even have an Algolia account. We take care of this for you! We'll send you a small snippet to integrate DocSearch to your website and an invite to your fully configured Algolia application. -
4
WFDownloader App
Free batch downloader for image, wallpaper, video, audio, document,
Use as an image gallery, wallpaper, audio/music, video, document, and other media bulk downloader from supported websites. Also use to download sequential website urls that have a certain pattern (e.g. image01.png to image100.png). Also use app's built-in site crawler for advanced link search or extraction. There is also special support for forum media and open directory downloading. It's a programmable downloader and also works with password protected sites. Say goodbye to downloading one... -
Speedy Claims became the top CMS-1500 Software by providing the best customer service imaginable to our thousands of clients all over America. Medical billing isn't the kind of thing most people get excited about - it is just a tedious task you have to do. But while it will never be a fun task, it doesn't have to be as difficult or time consumimg as it is now. With Speedy Claims CMS-1500 software you can get the job done quickly and easily, allowing you to focus on the things you love about your job, like helping patients. With a simple interface, powerful features to eliminate repetitive work, and unrivaled customer support, it's simply the best HCFA 1500 software available on the market. A powerful built-in error checking helps ensure your HCFA 1500 form is complete and correctly filled out, preventing CMS-1500 claims from being denied.
-
5
ahCrawler
A PHP search engine for your website and web analytics tool. GNU GPL3
ahCrawler is a set to implement your own search on your website and an analyzer for your web content. It can be used on a shared hosting. It consists of * crawler (spider) and indexer * search for your website(s) * search statistics * website analyzer (http header, short titles and keywords, linkchecker, ...) You need to install it on your own server. So all crawled data stay in your environment. You never know when an external webspider updated your content. Trigger a rescan... -
6
ScrapBot 1.40 64bits
Task automation software for accessing and manipulating website data.
ScrapBot is a task automation software that allows you to access, authenticate, extract, and insert data on any website. The software utilizes JavaScript to execute tasks, eliminating the need for server or additional software installations. The system can control the accessed webpage through JavaScript, and the entire navigation can be viewed in the program window. The main.js script runs in a separate frame from the navigation frame but can access all page content without any restrictions. -
7
PHP mini vulnerability suite
Multiple server/webapp vulnerability scanner
github: https://github.com/samedog/phpmvs -
8
sponge
A website crawler and links downloader command line tool
sponge is a website crawler and links downloader command line tool -
9
ShadowSocksShare
Python ShadowSocks framework
This project obtains the shared ss(r) account from the ss(r) shared website crawler, redistributes the account and generates a subscription link by parsing and verifying the account connectivity. Since Google plus will be closed on April 2, 2019, almost all the available accounts crawled before come from Google plus. So if you are building your own website, please keep an eye on the updates of this project and redeploy using the latest source code. -
eloomi combines Learning Management (LMS), Onboarding, Authoring, and continuous Performance Management tools in a cloud-based solution. It allows companies to optimize skill training, onboarding and employee development with strong user experience to enhance productivity and employee satisfaction. As a white label solution, the platform can be customized to mirror a company's branding and logo.
-
10
A Java Web crawler: multi-threaded, scalable, with high performance, extensible and polite. It can be used to crawl and index any web or enterprise domain and is configurable through a XML configuration file.
-
11
The "ToroSearch Search Engine" ist a project, I created in my leisure. You can add websites of your search engine or pages of your website, and you can search for websites on your own search machine or you can search for pages of your website. ATTENTION: This is not a crawler. It just lists websites or pages. Originally I hosted it myself, and nobody knew the source code. But now I don't have the time anymore to host and program it myself. And on SourceForge anyone can see it and change...
-
12
Products of the project: Java HTMLParser - VietSpider Web Data Extractor - Extractor VietSpider News. Click on "Show project details" to see more feature about each product.
-
13
crawly
Simple website crawler
soon. -
14
Addons for IOSEC - DoS HTTP Security
IOSec Addons are enhancements for web security and crawler detection
IOSEC PHP HTTP FLOOD PROTECTION ADDONS IOSEC is a php component that allows you to simply block unwanted access to your webpage. if a bad crawler uses to much of your servers resources iosec can block that. IOSec Enhanced Websites: https://www.artikelschreiber.com/en/ https://www.unaique.net/en/ https://www.unaique.com/ https://www.artikelschreiber.com/marketing/ https://www.paraphrasingtool1.com/ https://www.artikelschreiben.com/ https://buzzerstar.com/ https -
15
sitecheck
Modular web site spider for web developers.
More than just a link checker, sitecheck is a website spider (also known as a crawler) which can assist with SEO by testing an entire site plus both inbound links from search engines and outbound links to other sites for the following issues: looping redirects (HTTP 301/302), broken links (HTTP 404), server errors (HTTP 500), spelling mistakes, low readability scores (using the Flesch Reading Ease test), missing/empty/duplicate meta tags, duplicate content, slow page speed, W3C validation... -
16
Zoozle Search & Download Suchmaschine
Zoozle 2008 - 2010 Webpage, Tools and SQL Files
Download search engine and directory with Rapidshare and Torrent - zoozle Download Suchmaschine All The files that run the World Leading German Download Search Engine in 2010 with 500 000 unique visitors a day - all the tools you need to set up a clone. Code Contains: - PHP Files for zoozle - Perl Crawler for gathering new content to database and all other cool tools i have created https://www.artikelschreiber.com/en/ https://www.unaique.net/en/ https://www.unaique.com/ https... -
17
anonme.sh
anonymous tools [uncontinued]
anonme.sh {bash script} V1.0 Operative Systems Suported: Linux Dependencies: slowloris macchanger decrypter.py description of the script * this script makes it easy tasks such as DoS attacks, change you MAC address, inject XSS on target website, file upload vulns, MD5 decrypter, webcrawler (scan websites for vulns) and we can use WGET to download files from target domain or retrieve the all website... tutorial:http://www.youtube.com/watch?v=PrlrBuioCMc -
18
Aerc Crawler
A software used to crawler websites and make sitemaps.
Desenvolvido em Delphi X2 mas com total compatibilidade para Delphi 7, esta ferramenta vasculha um site determinado pelo usuario e atravez das tags "<a href='..." inicia um processo de Crawling no mesmo. Armazenando as informacoes em um banco de dados SQLli existe a possibilidade da criacao de um mapa do website. Developed in Delphi X2 but with full compatibility for Delphi 7, this tool scans a site given by the User and trough tags "<a href = '..." begins the process of Crawling therein... -
19
This 5 generation selenium web crawler crawl through web page of a host website searching for static and dynamic links and able to detect honeypot links.
-
20
Web Crawler Security Tool
A web crawler oriented to information security.
Last update on tue mar 26 16:25 UTC 2012 The Web Crawler Security is a python based tool to automatically crawl a web site. It is a web crawler oriented to help in penetration testing tasks. The main task of this tool is to search and list all the links (pages and files) in a web site. The crawler has been completely rewritten in v1.0 bringing a lot of improvements: improved the data visualization, interactive option to download files, increased speed in crawling, exports list of found... -
21
The archive-crawler project is building Heritrix: a flexible, extensible, robust, and scalable web crawler capable of fetching, archiving, and analyzing the full diversity and breadth of internet-accesible content.
-
22
Ex-Crawler is divided into 3 subprojects (Crawler Daemon, distributed gui Client, (web) search engine) which together provide a flexible and powerful search engine supporting distributed computing. More informations: http://ex-crawler.sourceforge.net
-
23
bee-rain is a web crawler that harvest and index file over the network. You can see result by bee-rain website : http://bee-rain.internetcollaboratif.info/
-
24
APC Anti Crawler is a php5 class based on APC which can be used to limit the amount of http request per IP. It stop web crawler to download your entire website.
-
25
The Java Sitemap Parser can parse a website's Sitemap (http://www.sitemaps.org/). This is useful for web crawlers that want to discover URLs from a website that is using the Sitemap Protocol. This project has been incorporated into crawler-commons (https://github.com/crawler-commons/crawler-commons) and is no longer being maintained.