Search Results for "python web crawler"
Sort By:
Showing 142 open source projects for "python web crawler"
View related business solutions-
Simple, Secure Domain RegistrationRegister or renew your domain and pay only what we pay. No markups, hidden fees, or surprise add-ons. Choose from over 400 TLDs (.com, .ai, .dev). Every domain is integrated with Cloudflare's industry-leading DNS, CDN, and free SSL to make your site faster and more secure. Simple, secure, at-cost domain registration.
-
Gen AI apps are built with MongoDB AtlasMongoDB Atlas provides built-in vector search and a flexible document model so developers can build, scale, and run gen AI apps without stitching together multiple databases. From LLM integration to semantic search, Atlas simplifies your AI architecture—and it’s free to get started.
-
1
AWS Secrets Manager Python caching
Enables in-process caching of secrets for Python applications
The AWS Secrets Manager Python caching client enables in-process caching of secrets for Python applications. To use this client you must have Python 3.6 or newer. Use of Python versions 3.5 or older are not supported. An Amazon Web Services (AWS) account to access secrets stored in AWS Secrets Manager. To create an AWS account, go to Sign In or Create an AWS Account and then choose I am a new user. -
2
mitmproxy
A free and open source interactive HTTPS proxy
mitmproxy is an open source, interactive SSL/TLS-capable intercepting HTTP proxy, with a console interface fit for HTTP/1, HTTP/2, and WebSockets. It's the ideal tool for penetration testers and software developers, able to debug, test, and make privacy measurements. It can intercept, inspect, modify and replay web traffic, and can even prettify and decode a variety of message types. Its web-based interface mitmweb gives you a similar experience as Chrome's DevTools, with the addition of... -
3
Wifipumpkin3
Powerful framework for rogue access point attack
wifipumpkin3 is powerful framework for rogue access point attack, written in Python, that allow and offer to security researchers, red teamers and reverse engineers to mount a wireless network to conduct a man-in-the-middle attack. -
4
WAFW00F
WAFW00F allows one to identify and fingerprint Web App Firewall
The Web Application Firewall Fingerprinting Tool. Sends a normal HTTP request and analyses the response; this identifies a number of WAF solutions. If that is not successful, it sends a number of (potentially malicious) HTTP requests and uses simple logic to deduce which WAF it is. If that is also not successful, it analyses the responses previously returned and uses another simple algorithm to guess if a WAF or security solution is actively responding to our attacks. For further details,... -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
5
Django OAuth Toolkit
OAuth2 goodies for the Djangonauts!
...OAuth is an open standard for access delegation, commonly used as a way for Internet users to grant websites or applications access to their information on other websites but without giving them the passwords. Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design. Built by experienced developers, it takes care of much of the hassle of Web development, so you can focus on writing your app without needing to reinvent the wheel. Your Django app exposes a web API you want to protect with OAuth2 authentication. You need to implement an OAuth2 authorization server to provide tokens management for your infrastructure. -
6
Payloads All The Things
A list of useful payloads and bypass for Web Application Security
A list of useful payloads and bypasses for Web Application Security. Feel free to improve with your payloads and techniques. The API key is a unique identifier that is used to authenticate requests associated with your project. Some developers might hardcode them or leave it on public shares. -
7
OAuthLib
A generic, spec-compliant, thorough implementation of the OAuth
A generic, spec-compliant, thorough implementation of the OAuth request-signing logic for Python 3.8+. OAuthLib is a framework which implements the logic of OAuth1 or OAuth2 without assuming a specific HTTP request object or web framework. Use it to graft OAuth client support onto your favorite HTTP library, or provide support onto your favourite web framework. If you're a maintainer of such a library, write a thin veneer on top of OAuthLib and get OAuth support for very little effort. -
8
Maltrail
Malicious traffic detection system
Maltrail is a malicious traffic detection system, utilizing publicly available (black)lists containing malicious and/or generally suspicious trails, along with static trails compiled from various AV reports and custom user-defined lists, where trail can be anything from domain name, URL, IP address (e.g. 185.130.5.231 for the known attacker) or HTTP User-Agent header value (e.g. sqlmap for automatic SQL injection and database takeover tool). Also, it uses (optional) advanced heuristic... -
9
GRR
GRR Rapid Response, remote live forensics for incident response
GRR Rapid Response is an incident response framework focused on remote live forensics. It consists of a python client (agent) that is installed on target systems, and python server infrastructure that can manage and talk to clients. The goal of GRR is to support forensics and investigations in a fast, scalable manner to allow analysts to quickly triage attacks and perform analysis remotely. GRR client is deployed on systems that one might want to investigate. -
Build Securely on Azure with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
TorBot
Dark Web OSINT Tool
Contributions to this project are always welcome. To add a new feature fork the dev branch and give a pull request when your new feature is tested and complete. If its a new module, it should be put inside the modules directory. The branch name should be your new feature name in the format <Feature_featurename_version(optional)>. On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using chmod +x... -
11
Alerta
Alerta monitoring system
Email was not designed to be used as an alert console. It is not a scalable solution when it comes to monitoring and alert visualization. A minimal installation of Alerta can be deployed quickly and easily as monitoring requirements and confidence grow. There are integrations available with Prometheus, Riemann, Nagios, Zabbix, netdata, Sensu, Pingdom and Cloudwatch. Integrating bespoke systems is easy using the API or command-line tool. Alerts are submitted in JSON format to an HTTP API.... -
12
CTFd
CTFs as you need them
CTFd is a Capture The Flag framework focusing on ease of use and customizability. It comes with everything you need to run a CTF and it's easy to customize with plugins and themes. Create your own challenges, categories, hints, and flags from the Admin Interface. Dynamic Scoring Challenges. Unlockable challenge support. Challenge plugin architecture to create your own custom challenges. Static & Regex-based flags. Custom flag plugins. Unlockable hints. File uploads to the server or an Amazon... -
13
Scout Suite
Multi-cloud security auditing tool
Scout Suite is an open-source multi-cloud security-auditing tool, which enables security posture assessment of cloud environments. Using the APIs exposed by cloud providers, Scout Suite gathers configuration data for manual inspection and highlights risk areas. Rather than going through dozens of pages on the web consoles, Scout Suite presents a clear view of the attack surface automatically. Scout Suite was designed by security consultants/auditors. It is meant to provide a point-in-time... -
14
GTFOBins
GTFOBins is a curated list of Unix binaries
GTFOBins is a curated catalog of Unix / POSIX system binaries and how they can be misused to bypass restrictions, escalate privileges, exfiltrate data, spawn shells, or otherwise act as “living off the land” tools in a compromised environment. It collects documented techniques for how everyday binaries (e.g. awk, bash, tar, scp) can be abused under constrained conditions. Indexed list of Unix binaries and documented misuse techniques. Examples of command invocations to exploit... -
15
ClusterFuzz
Scalable fuzzing infrastructure
ClusterFuzz is a scalable fuzzing infrastructure that finds security and stability issues in software. Google uses ClusterFuzz to fuzz all Google products and as the fuzzing backend for OSS-Fuzz. ClusterFuzz provides many features which help seamlessly integrate fuzzing into a software project's development process. Can run on any size cluster (e.g. OSS-Fuzz instance runs on 100,000 VMs). Fully automatic bug filing, triage and closing for various issue trackers (e.g. Monorail, Jira).... -
16
ConsoleMe
A central control plane for AWS permissions and access
ConsoleMe is a web service that makes AWS IAM permissions and credential management easier for end-users and cloud administrators. ConsoleMe provides numerous ways to log in to the AWS Console. An IAM Self-Service Wizard lets users request IAM permissions in plain English. Cross-account resource policies will be automatically generated and can be applied with a single click for certain resource types. Weep (ConsoleMe’s CLI) supports 5 different ways of serving AWS credentials locally. Cloud... -
17
Digna Web Scanner
A tool to check web apps for vulnerabilty
This web application scanner is a powerful tool designed to identify potential security vulnerabilities in websites with full GUI (no need to cli). It currently performs checks for: SQL Injection (SQLi): Detects vulnerabilities that could allow attackers to inject malicious SQL code and manipulate the database. XSS Cross-site-scripting: Detect vulnerability that allow attackers to inject client-side scripts into web pages Cross-Site Request Forgery (CSRF): Helps discover... -
18
YoungerSibling
YoungerSibling: Cross-platform OSINT tool for quick data gathering.
YoungerSibling is a Python-based terminal utility script designed for educational purposes. It provides a set of useful tools to perform tasks like searching the web, performing lookups (Google search, IP lookup, username lookup, etc.), and extracting metadata from images, directly from the terminal. This project aims to help students, developers, and hobbyists learn about web scraping, API usage, and terminal interaction with Python. -
19
Network Security Toolkit (NST)
A network security analysis and monitoring toolkit Linux distribution.
Network Security Toolkit (NST) is a bootable ISO image (Live USB Flash Drive) based on Fedora 42 providing easy access to best-of-breed Open Source Network Security Applications and should run on most x86_64 systems. The main intent of developing this toolkit was to provide the security professional and network administrator with a comprehensive set of Open Source Network Security Tools. The majority of tools published in the article: Top 125 Security Tools by INSECURE.ORG are available... -
20
WebPin
Advanced Modern Web Application Manager for Linux
Advanced Modern Web Application Manager for Linux Transform any website into a native desktop application -
21
Wapiti
Wapiti is a web-application vulnerability scanner
Wapiti is a vulnerability scanner for web applications. It currently search vulnerabilities like XSS, SQL and XPath injections, file inclusions, command execution, XXE injections, CRLF injections, Server Side Request Forgery, Open Redirects... It use the Python 3 programming language. -
22
ufonet
UFONet - Denial of Service Toolkit
UFONet - Is a set of hacktivist tools that allow launching coordinated DDoS and DoS attacks and combine both in a single offensive. It also works as an encrypted DarkNET to publish and receive content by creating a global client/server network based on a direct-connect P2P architecture. + FAQ: https://ufonet.03c8.net/FAQ.html -------------------------------------------- -> UFONet-v1.8 [DPh] "DarK-PhAnT0m!" (.zip) -> md5 = [ c8ab016f6370c8391e2e6f9a7cbe990a ] -> UFONet-v1.8... -
23
Plum Cave Twofish
A version of Plum Cave that uses the ChaCha20 and Twofish ciphers
A version of Plum Cave that employs the "ChaCha20 + Twofish-256 CBC + HMAC-SHA3-512" authenticated encryption scheme for data encryption and ML-KEM-1024 for quantum-resistant key exchange. -
24
justniffer is a TCP sniffer. It reassembles and reorders packets and displays the tcp flow in a customizable way. It can log network traffic in web server log format. It can also log network services performances (e.g. web server response times) and extract http content (images, html, scripts, etc)
-
25
Robolinux
Announcing Robolinux Cinnamon 14.1
Announcing The New Robolinux Series 14 Mate LTS - 2034 Robolinux is very pleased to announce a completely new 14.1 Mate privacy centered 1inux operating system you can download freely while also offering our users an optional 14+ advanced upgrade which comes with our Untracker and FAAST Boot along with one click popular privacy software installers like Tor and I2p, Wireshark and Bleachbit plus many more for a fair and reasonable price. Robolinux14.1-Mate is ideal for beginners and...
