Search Results for "extensible web spider"
Sort By:
Showing 29 open source projects for "extensible web spider"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
Atera - an All-in-one platform for IT managementYour IT essentials, integrated & elevated. Take your IT management from automated to autonomous, download Atera's agent to start your free trial!
-
1
Nikto
Web server vulnerability scanner for security assessments
Nikto is an open-source web server scanner that performs comprehensive tests to detect potentially dangerous files, outdated server software, and configuration issues. It’s widely used by penetration testers and security professionals for auditing web applications and infrastructure. Nikto supports multiple output formats and can integrate with other tools for automated scanning workflows. -
2
PentestGPT
Automated Penetration Testing Agentic Framework Powered by LLMs
...Built with a modular and extensible architecture, PentestGPT supports cloud and local LLMs, making it suitable for research, education, and authorized security testing. -
3
Luakit
Fast, small, webkit based browser framework extensible by Lua
Luakit is a highly configurable browser framework based on the WebKit web content engine and the GTK+ toolkit. It is very fast, extensible with Lua, and licensed under the GNU GPLv3 license. It is primarily targeted at power users, developers and anyone who wants to have fine-grained control over their web browser’s behavior and interface. While switching to the WebKit 2 API means a vastly improved security situation, not all distributions of Linux package the most up-to-date version of WebKitGTK+, and several package very outdated versions that have many known vulnerabilities. ... -
4
ZAP
The OWASP ZAP core project
The OWASP Zed Attack Proxy (ZAP) is one of the world’s most popular free security tools and is actively maintained by a dedicated international team of volunteers. It can help you automatically find security vulnerabilities in your web applications while you are developing and testing your applications. It's also a great tool for experienced pentesters to use for manual security testing. ZAP is an easy to use integrated penetration testing tool for finding vulnerabilities in web... -
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
5
XRAY
XRay for recon, mapping and OSINT gathering from public networks
XRAY is a modular security toolset that helps developers and security professionals analyze, fuzz, and test web applications, protocols, and network services for vulnerabilities. It provides a framework for writing and executing inspection modules that can parse structured data (JSON, XML, HTML), traverse graphs of endpoints, and perform intelligent probing guided by discovered surface area. XRay is typically used as a reconnaissance and vulnerability discovery engine in red-team or app-security workflows: it leverages extensible plugins to adapt to different protocols, inject payloads, and detect common bug classes such as injection flaws, misconfigurations, and unsafe endpoints. ... -
6
Tookie-OSINT
Username OSINT tool for discovering accounts across many websites
Tookie-OSINT is an open source intelligence tool designed to help security researchers, ethical hackers, and investigators discover online accounts associated with a specific username. It automates the process of searching for usernames across multiple websites, making it easier to identify a person's presence on different platforms. By entering a target username, Tookie-OSINT scans a list of supported sites and checks whether the username exists on those platforms. This approach removes the... -
7
OWASP Maryam
Modular OSINT framework for automated open-source intelligence gatheri
Maryam is an open source intelligence (OSINT) framework designed to automate the process of gathering and analyzing publicly available information from the internet. It provides a modular environment that enables users to collect data from search engines, open data sources, and various online services for reconnaissance and investigative purposes. Written in Python, Maryam is built to provide a flexible and extensible framework for harvesting information quickly and efficiently from open... -
8
ModSecurity Nginx Connector
ModSecurity v3 Nginx Connector
ModSecurity-nginx is the connector that embeds the ModSecurity v3 (libmodsecurity) web application firewall engine into NGINX. It integrates WAF processing into the NGINX request/response phases, allowing rules to inspect headers, bodies, and even streaming request data before it reaches upstream apps. Operators can load the OWASP Core Rule Set or custom rules to detect and block common attacks such as SQLi, XSS, RCE patterns, and protocol anomalies. The module exposes directives for... -
9
PhoneInfoga
Information gathering framework for phone numbers
PhoneInfoga is an open-source intelligence framework focused on gathering and analyzing information related to international phone numbers. The tool aggregates data from multiple scanners and external services to provide contextual intelligence such as country, carrier, line type, and potential VoIP provider details. It is designed primarily for investigators, analysts, and security researchers who need structured phone-number reconnaissance rather than real-time tracking. PhoneInfoga... -
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
10
ReconSpider
Most Advanced Open Source Intelligence (OSINT) Framework
...ReconSpider can be used by Infosec Researchers, Penetration Testers, Bug Hunters, and Cyber Crime Investigators to find deep information about their target. ReconSpider aggregate all the raw data, visualize it on a dashboard, and facilitate alerting and monitoring on the data. Recon Spider also combines the capabilities of Wave, Photon and Recon Dog to do a comprehensive enumeration of attack surfaces. Reconnaissance is a mission to obtain information by various detection methods, about the activities and resources of an enemy or potential enemy, or geographic characteristics of a particular area. A Web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web, typically for the purpose of Web indexing (web spidering). -
11
GoSpider
Gospider - Fast web spider written in Go
GoSpider - Fast web spider written in Go. Fast web crawling. Brute force and parse sitemap.xml. Parse robots.txt. Generate and verify link from JavaScript files. Link Finder. Find AWS-S3 from response source. Find subdomains from the response source. Get URLs from Wayback Machine, Common Crawl, Virus Total, Alien Vault. Format output easy to Grep. Support Burp input. -
12
BlackWidow
Python web scanner for OSINT gathering and OWASP vulnerability fuzzing
BlackWidow is a Python-based web application scanning tool designed to crawl target websites and collect open-source intelligence (OSINT) while identifying potential security vulnerabilities. It functions as a web spider that systematically explores a site to gather valuable information such as URLs, dynamic parameters, subdomains, email addresses, and phone numbers associated with the target domain. -
13
Photon
Incredibly fast crawler designed for OSINT
...Despite its speed focus, the tool still provides useful filtering and extraction capabilities for analysts who need structured results. Overall, Photon functions as a lightweight yet powerful reconnaissance spider for web intelligence gathering. -
14
The Veronica Security Suite
Veronica Security Suite
In 2007 I was greatly interested in using biometrics (fingerprints, etc.) to unlock my encrypted partition. After scouring the web searching for something that could accomplish this and finding nothing (even in 2017), my frustration had finally led me to start this project. Its my first cross-platform project and my first ever LINUX project. This project aims to provide a unique security layer for both Linux and Windows. It is written completely in C and aims to be simple and easily extensible for anyone wanting to expand on its functionality. -
15
The PHP Web Toolkit enables the rapid development of multi-layered web applications and is designed to be easy to use, extensible, reliable, reusable, scalable and secure. It integrates with ADOdb, FCKeditor, kses, Libmcrypt, Libmhash and Smarty.
-
16
DACS - Distributed Access Control System
Authentication, Access Control, and Single Sign-On
DACS,a light-weight single sign-on and role-based security system for Apache or server-based software, provides comprehensive authentication capabilities, and powerful, transparent rule-based authorization checking for any web service or CGI program. The latest release is DACS 1.5.1 (26-Oct-2025) It is not available here! Get it at https://dacs.dss.ca/download.html -
17
WiKID Two-Factor Authentication System
Two-factor authentication system
The WiKID Strong Authentication System is a public-key based two-factor authentication system. It is a flexible, extensible, and secure alternative to tokens, certs and passwords. Application & API support exists for Java, ASP, PHP, Ruby, OpenVPN, TACACS+, etc. Read our eGuide on how to setup your network with two-factor authentication: http://www.wikidsystems.com/learn-more/two-factor-authentication-white-papers -
18
Kingdom multiuser password manager
An extensible pki based enterprise password utility
-
19
SauceWalk Proxy Helper
Enumeration and automation of file discovery for your sec tools.
SauceWalk is a freeware(.exe)/Open Source(.py) tool for aiding in the enumeration of web application structure. It consists of 2 parts a local executable (walk.exe) and a remote agent. Walk.exe iterates through the local files and folders of your target web application (for example a local copy of Wordpress) and generates requests via your favourite proxy (for example burp suite) against a given target url. The remote agent can be used to identify target files and folders on a live system via a PHP script on the target server(ASP/JSP coming soon). ... -
20
Safe3WVS is one of the most powerful web vulnerability scanner with AI on-the-fly web spider crawling technology,especially web portals ,it is the most fast tool to dig such as sql injection, upload vulnerability, and more.http://www.safe3.com.cn/en
-
21
Monkey-Spider
Moved to https://github.com/aikinci/monkeyspider
The Monkey-Spider is a crawler based low-interaction Honeyclient Project. It is not only restricted to this use but it is developed as such. The Monkey-Spider crawles Web sites to expose their threats to Web clients. -
22
OpenRCA is a highly flexible and modular Certificate management and Registration Authority based on J2EE technology. OpenRCA is designed to be a very flexible, extensible and low cost enterprise PKI solution.
-
23
Antispyd is an HTTP/HTTPS threats filtering proxy server written in pure C under Linux. The main goal of the project is to develop a free, secure, fast and extensible solution to fight against current Web threats, anonymize and control web traffic.
-
24
Extensible User Folder is a Zope user folder that requires the authentication of users to be removed from the storage of properties for users. Writing new authentication or property sources requires no intimate Zope authentication knowledge.
-
25
GridAuth is a user credential management system for distributed data and computational grids. GridAuth is configurable and extensible to just about any system requiring credential management, advanced authorization and secure authentication.