Search Results for "jailbreak"

In-The-Wild Jailbreak Prompts on LLMs is an open-source research repository that provides datasets and analytical tools for studying jailbreak prompts used to bypass safety restrictions in large language models. The project is part of a research effort to understand how users attempt to circumvent alignment and safety mechanisms built into modern AI systems.

...It is positioned more as a prompt experimentation archive than a traditional software library, emphasizing the study of how instruction phrasing can influence AI outputs. The project reflects the growing interest in prompt security, jailbreak testing, and model alignment research within the AI community. Its materials are often used by researchers and enthusiasts studying robustness, safety, and adversarial prompting dynamics. Because of its unconventional focus, it functions primarily as a research and exploration resource rather than a production tool. Overall, L1B3RT4S serves as a niche but widely referenced collection for studying advanced prompt manipulation patterns.

AI-Infra-Guard is a powerful open-source security platform from Tencent’s Zhuque Lab designed to assess the safety and resilience of AI infrastructures, codebases, and components through automated scanning and evaluation tools. It brings together AI infrastructure vulnerability scanning, MCP server risk analysis, and jailbreak evaluation into a unified workflow so that enterprises and individuals can identify critical security issues without relying on external services. Users can deploy it via Docker or scripts to get a modern web UI that guides them through tasks like scanning third-party frameworks for known CVEs and experimenting with prompt security against attack vectors. ...

...The router operates as an intelligent layer between users and model infrastructure, capturing signals from prompts, responses, and contextual data to improve decision-making. It can also integrate safety and monitoring mechanisms that detect issues such as jailbreak attempts, hallucinations, or sensitive information exposure.

FuzzyAI is an open-source fuzzing framework designed to test the security and reliability of large language model applications. The tool automates the process of generating adversarial prompts and input variations to identify vulnerabilities such as jailbreaks, prompt injections, or unsafe model responses. It allows developers and security researchers to systematically evaluate the robustness of LLM-based systems by simulating a wide range of malicious or unexpected inputs. The framework can...

Github: https://github.com/Tencent/AI-Infra-Guard A.I.G (AI-Infra-Guard) integrates capabilities such as AI infra vulnerability scan, MCP Server risk scan, and Jailbreak Evaluation, aiming to provide users with the most comprehensive, intelligent, and user-friendly solution for AI security risk self-examination. We are committed to making A.I.G(AI-Infra-Guard) the industry-leading AI red teaming platform. More stars help this project reach a wider audience, attracting more developers to contribute, which accelerates iteration and improvement. ...