Search Results for "web crawler bot"
Sort By:
Showing 14 open source projects for "web crawler bot"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
1
IPRanges
Daily updated lists of cloud, bot, and service IP ranges
ipranges is an open source repository that provides continuously updated lists of IP address ranges associated with major cloud providers, search engine crawlers, and online services. ipranges collects IP ranges from publicly available sources and organizes them into structured files that can be easily used in security, networking, and automation workflows. It includes address ranges from providers such as Google Cloud, Amazon AWS, Microsoft, Oracle Cloud, and DigitalOcean, as well as well... -
2
SafeLine
Serve as a reverse proxy to protect your web services from attacks
SafeLine is a self-hosted WAF(Web Application Firewall) to protect your web apps from attacks and exploits. A web application firewall helps protect web apps by filtering and monitoring HTTP traffic between a web application and the Internet. It typically protects web apps from attacks such as SQL injection, XSS, code injection, os command injection, CRLF injection, LDAP injection, XPath injection, RCE, XXE, SSRF, path traversal, backdoor, brute force, HTTP-flood, bot abuse, among others. ... -
3
ngx_waf
Handy, High performance, ModSecurity compatible Nginx firewall module
Handy, High-performance Nginx firewall module. Such as black and white list of IPs or IP range, uri black and white list, and request body black list, etc. Directives and rules are easy to write and readable. The IP detection is a constant-time operation. Most of the remaining inspections use caching to improve performance. Compatible with ModSecurity's rules, you can use OWASP ModSecurity Core Rule Set. Supports verifying Google, Bing, Baidu and Yandex crawlers and allowing them... -
4
Coraza
OWASP Coraza WAF is a golang modsecurity compatible firewall library
...CRS protects from many common attack categories including: SQL Injection (SQLi), Cross Site Scripting (XSS), PHP & Java Code Injection, HTTPoxy, Shellshock, Scripting/Scanner/Bot Detection & Metadata & Error Leakages. Coraza is a library at its core, with many integrations to deploy on-premise Web Application Firewall instances. -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
5
Niubi Guard
Open-source GitHub repository abuse detection and response system
Niubi Guard is an open-source defense system for GitHub maintainers who need to detect and respond to repository abuse. It focuses on spam, harassment, coordinated attacks, suspicious issue activity, and bot-like comment patterns. The project gives maintainers control over rules, allowlists, detection signals, confidence thresholds, and response modes. It includes both a command-line workflow and a web console for configuring scans and reviewing results. AI-assisted detection can be connected through OpenAI-compatible models, but review-only behavior is the default. ... -
6
tirreno
Security framework for devs and product teams
tirreno is a security framework. tirreno [tir.ˈrɛ.no] helps understand, monitor, and protect your product from threats, fraud, and abuse. While classic cybersecurity focuses on infrastructure and network perimeter, most breaches occur through compromised accounts and application logic abuse that bypasses firewalls, SIEM, WAFs, and other defenses. tirreno detects threats where they actually happen: inside your product. tirreno is a few-dependency, "low-tech" PHP/PostgreSQL application.... -
7
Hakrawler
Fast Go web crawler for discovering URLs and web app endpoints
hakrawler is a lightweight command-line web crawler built in Go that is designed to quickly discover URLs, endpoints, and assets within web applications. It is primarily used during the reconnaissance phase of security testing, bug bounty hunting, and penetration testing. It works by automatically crawling web pages and extracting links, JavaScript file locations, and other resources that may reveal additional attack surface or hidden functionality. hakrawler is implemented as a simple and efficient crawler using the Gocolly library, which allows it to perform fast and concurrent crawling of web pages. ... -
8
ReconSpider
Most Advanced Open Source Intelligence (OSINT) Framework
...Reconnaissance is a mission to obtain information by various detection methods, about the activities and resources of an enemy or potential enemy, or geographic characteristics of a particular area. A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). -
9
PHP mini vulnerability suite
Multiple server/webapp vulnerability scanner
github: https://github.com/samedog/phpmvs -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
WebSploit Framework
WebSploit is a high level MITM Framework
WebSploit Advanced MITM Framework [+]Autopwn - Used From Metasploit For Scan and Exploit Target Service [+]wmap - Scan,Crawler Target Used From Metasploit wmap plugin [+]format infector - inject reverse & bind payload into file format [+]phpmyadmin Scanner [+]CloudFlare resolver [+]LFI Bypasser [+]Apache Users Scanner [+]Dir Bruter [+]admin finder [+]MLITM Attack - Man Left In The Middle, XSS Phishing Attacks [+]MITM - Man In The Middle Attack [+]Java Applet Attack [+]MFOD Attack Vector [+]ARP Dos Attack [+]Web Killer Attack [+]Fake Update Attack [+]Fake Access point Attack [+]Wifi Honeypot [+]Wifi Jammer [+]Wifi Dos [+]Wifi Mass De-Authentication Attack [+]Bluetooth POD Attack Project In Github : https://github.com/websploit -
11
Photon
Incredibly fast crawler designed for OSINT
Photon is an extremely fast web crawler built specifically for OSINT and reconnaissance use cases. It is designed to extract URLs, endpoints, files, and other intelligence artifacts from target websites with minimal overhead. The crawler prioritizes speed and breadth, making it suitable for mapping web attack surfaces and discovering hidden resources. Photon is commonly used during early reconnaissance phases to build a comprehensive inventory of reachable assets. ... -
12
BitCrazy's Faucet-in-a-Box Mod
Mod of the original Faucet-in-a-Box faucet PHP script
Mod of the original Faucet-in-a-Box faucet PHP script (based on its revision 69). Includes pre-installed Anti-Bot Links 5.75, full support of user balances, countdown timer and some other improvements and cool features. ======================================== Minimum requirements: PHP 5.4.4+ (5.5 recommended) MySQL (5.6 recommended) ======================================== CHANGELOG { [F] - bugfixes, [+] - new features, [!] - known issues } Current release (version... -
13
SauceWalk Proxy Helper
Enumeration and automation of file discovery for your sec tools.
SauceWalk is a freeware(.exe)/Open Source(.py) tool for aiding in the enumeration of web application structure. It consists of 2 parts a local executable (walk.exe) and a remote agent. Walk.exe iterates through the local files and folders of your target web application (for example a local copy of Wordpress) and generates requests via your favourite proxy (for example burp suite) against a given target url. The remote agent can be used to identify target files and folders on a live system via a PHP script on the target server(ASP/JSP coming soon). ... -
14
** Guys I have built a much more powerful Fully Featured CMS system at: https://github.com/MacdonaldRobinson/FlexDotnetCMS Macs CMS is a Flat File ( XML and SQLite ) based AJAX Content Management System. It focuses mainly on the Edit In Place editing concept. It comes with a built in blog with moderation support, user manager section, roles manager section, SEO / SEF URL
- Previous
- You're on page 1
- Next