With up to 25k MAUs and unlimited Okta connections, our Free Plan lets you focus on what you do best—building great apps.
You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
Try free now
Cloud tools for web scraping and data extraction
Deploy pre-built tools that crawl websites, extract structured data, and feed your applications. Reliable web data without maintaining scrapers.
Automate web data collection with cloud tools that handle anti-bot measures, browser rendering, and data transformation out of the box. Extract content from any website, push to vector databases for RAG workflows, or pipe directly into your apps via API. Schedule runs, set up webhooks, and connect to your existing stack. Free tier available, then scale as you need to.
The Backlinkchecker is used to analyze how many inbound links there are for your website. Its developed in PHP, JavaScript and MySQL. It uses the great Javascript lib extjs and the php lib snoopy.php.
The purpose of this project is to gauge website page loading speed as aspect of client through web management system, Leech. Leech says, "Leech bot reachs hosts via URL Information and sucks http responses!"
FoxBAT was made in an attempt to see if Naïve Bayesian filtering commonly used for spam filtering could be employed in the World Wide Web context. The application consists of a Firefox extension (.XPI package) and a Perl server script.
Atera all-in-one platform IT management software with AI agents
Ideal for internal IT departments or managed service providers (MSPs)
Atera’s AI agents don’t just assist, they act. From detection to resolution, they handle incidents and requests instantly, taking your IT management from automated to autonomous.
MWQL is an extension to MediaWiki, providing (end) users with a language for structural queries, so that they can build dynamic pages as seen in the Special pages of Wikipedia.
SpongeStats est un outil développé en PHP/MySQL/AJAX pour visualiser les statistiques de fréquentation et analyser le référencement d'un site Internet. SpongeStats is a analysis tools for your web site developped in PHP/MySQL/AJAX.
Ruya is a Python-based breadth-first, level-, delayed, event-based-crawler for crawling English, Japanese websites. It is targeted solely towards developers who want crawling functionality in their projects using API, and crawl control.
The OpenWebsuite.org contains the powerful HTML-Editor (X)HTML-Format and the Site analysing tool WebAnalyse. It includes a project management, css editor, link-check (for whole domains) and much more. It was released under the GNU GPL to find helper.
Get notified when specific parts of web/ftp pages have been modified.
Usefull to get new release of softwares, updates of files or pages...
Made with perl scripts. Need a sql database. (inspired from dodchecker (c)dodfr)
Aracnis is a Java based framework for building distributed web spiders. These spiders can be used to accomplish a variety of tasks, for example, screen-scraping and link integrity checking.
Auto Proxy Filter Test (APFT) automates the testing of safe and unsafe URLs against a content filtering proxy (such as Dansguardian) and helps prevent regressions. APFT is useful to people who are designing filter rules.
JLinkCheck is an Ant Task written in Java for checking links in websites. It is not just checking one single page, but crawling a whole site like a spider, generating a report in XML and (X)HTML. JReptator will be its succesor with many more features
Toke is a webmining toolkit for web exploring, indexing and searching for Java. Toke allows to you crawl public or private web sites, in order to create web estatistics, web Pajek graphs, Lucene indexs and word frequency files for data clustering.
This project provides a system tray application that monitors the status of a project which uses a DART dashboard. Status is displayed by color-coded icons, and message dialogs alert the user when the build status changes.
InSite is a Web site management tool written in perl. It checks link integrity and does some basic content monitoring of your site's files directly on the local disk, which gives it a huge speed advantage over similar tools.
404SEF is a component for Mambo CMS (4.5.x right now, 4.6.x soon) to provide Human Readable URLs. Works with apache and IIS. Provides proper 404 status code for missing content, logs 404 errors, and user-defined custom redirection via special shortcuts
Automatic link management program. Has three functions: List links in database in html format, add links to database using browser and optionaly check for bad links (by cron job). This eliminates the need for the "Report bad link" on too many web sites