Search Results for "safety"
Sort By:
Showing 62 open source projects for "safety"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
1
Purple Llama
Set of tools to assess and improve LLM security
Purple Llama is an umbrella safety initiative that aggregates tools, benchmarks, and mitigations to help developers build responsibly with open generative AI. Its scope spans input and output safeguards, cybersecurity-focused evaluations, and reference shields that can be inserted at inference time. The project evolves as a hub for safety research artifacts like Llama Guard and Code Shield, along with dataset specs and how-to guides for integrating checks into applications. ... -
2
PKU Beaver
Constrained Value Alignment via Safe Reinforcement Learning
PKU Beaver is an open-source research project focused on improving the safety alignment of large language models through reinforcement learning from human feedback under explicit safety constraints. The framework introduces techniques that separate helpfulness and harmlessness signals during training, allowing models to optimize for useful responses while minimizing harmful behavior. To support this process, the project provides datasets containing human-labeled examples that encode both performance preferences and safety constraints across multiple dimensions. ... -
3
SENTINEL
Open-source behavioral intelligence platform for detecting child groom
...Key capabilities: - Behavioral scoring across four signal types: linguistic register shifts, relationship graph evolution, temporal escalation patterns, and fairness gates - Tamper-evident audit logs (cryptographically chained, 7-year retention) for legal proceedings and regulatory compliance - Privacy-preserving federation: platforms share encrypted threat signatures without exposing raw messages or user identities - Demographic parity audits enforced as a deployment gate before any detection model goes live - Built-in compliance for EU Digital Services Act, UK Online Safety Act, COPPA, and GDPR Free for organizations under $100k annual revenue and all non-commerci -
4
ODMantic
Sync and Async ODM (Object Document Mapper) for MongoDB
Odmantic is an Object-Document Mapper (ODM) for MongoDB, designed for Python applications using Pydantic models, providing a seamless integration with type safety and validation. -
Atera - an All-in-one platform for IT managementYour IT essentials, integrated & elevated. Take your IT management from automated to autonomous, download Atera's agent to start your free trial!
-
5
FuzzyAI Fuzzer
A powerful tool for automated LLM fuzzing
...FuzzyAI provides testing tools, datasets, and evaluation workflows that help researchers measure how well models resist harmful instructions or attempts to bypass safety mechanisms. -
6
Crosvm
The Chrome OS Virtual Machine Monitor
...Unlike general-purpose emulators like QEMU, crosvm avoids full hardware emulation and focuses on modern paravirtualized I/O using the virtio standard, reducing complexity and attack surface. Written in Rust, it emphasizes memory safety and modularity, allowing sandboxed device emulation with fine-grained privilege separation. crosvm underpins several ChromeOS subsystems, including Android Runtime for Chrome (ARCVM) and Crostini Linux containers, enabling rich application compatibility within a tightly controlled environment. -
7
NautilusTrader
A high-performance algorithmic trading platform
...This helps to address the parity challenge of keeping the Python research/backtest environment, consistent with the production live trading environment. NautilusTraders design, architecture and implementation philosophy holds software correctness and safety at the highest level, with the aim of supporting Python native, mission-critical, trading system backtesting and live deployment workloads. -
8
In-The-Wild Jailbreak Prompts on LLMs
A dataset consists of 15,140 ChatGPT prompts from Reddit
In-The-Wild Jailbreak Prompts on LLMs is an open-source research repository that provides datasets and analytical tools for studying jailbreak prompts used to bypass safety restrictions in large language models. The project is part of a research effort to understand how users attempt to circumvent alignment and safety mechanisms built into modern AI systems. The repository includes a large collection of prompts gathered from real-world platforms such as Reddit, Discord, prompt-sharing communities, and other public sources. ... -
9
Nothing Ever Happens
Focused async Python bot for Polymarket
...Its core concept is based on statistical observations that a majority of prediction market outcomes resolve negatively, and it attempts to exploit this base-rate bias through systematic participation rather than predictive modeling. The bot includes a safety-oriented design with explicit environment variable requirements to enable live trading, ensuring that users consciously opt into real financial risk, along with a paper trading mode for testing without capital exposure. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
git-delete-merged-branches
Command-line tool to delete merged Git branches
...Supports workflows with multiple release branches, e.g. only delete branches that have been merged to all of master, dev and staging. Quick interactive configuration. Provider agnostic: Works with GitHub, GitLab, Gitea and any other Git hosting. Takes safety seriously. Deletion is a sharp knife that requires care. While git reflog would have your back in most cases, git-delete-merged-branches takes safety seriously. git push is used with --force-with-lease so if the server and you have a different understanding of that branch, it is not deleted. There is no use of os.system or shell code to go wrong. -
11
Claude Code Tools
Practical productivity tools for Claude Code, Codex-CLI
...Some components enable Claude Code to interact with terminal multiplexers such as tmux so that it can run programs, debug applications, and interact with scripts that require user input. The toolkit also provides safety mechanisms that prevent potentially dangerous shell commands from being executed automatically by AI agents. -
12
ZAPI
ZAPI by Adopt AI is an open-source Python library
ZAPI is a developer-centric API framework that streamlines building, testing, and deploying APIs with strong type safety and minimal boilerplate, helping teams deliver backend services faster with fewer errors. It emphasizes a declarative router and schema model that uses types to define request and response formats, providing clear contracts for frontend and backend teams while automatically generating documentation. Zapi abstracts many repetitive tasks such as validation, authentication flows, and error handling so developers can focus on business logic instead of infrastructure plumbing. ... -
13
typed-ffmpeg
Type-safe FFmpeg bindings for Python & TypeScript
...It supports operations such as transcoding, filtering, and stream manipulation while maintaining compatibility with FFmpeg’s underlying capabilities. typed-ffmpeg is particularly useful in large-scale applications where maintainability and type safety are important. It integrates well with modern JavaScript and Node.js ecosystems, enabling seamless backend media processing. Overall, it simplifies FFmpeg usage while preserving flexibility and control. -
14
AI Agents Masterclass
Follow along with my AI Agents Masterclass videos
...The project includes structured lessons, code examples, and practical exercises that cover foundational concepts like prompt engineering, chaining agents, tool usage, plan execution, evaluation, and safety considerations. It breaks down how autonomous agents interact with external systems, handle iterative reasoning, and integrate with third-party services or APIs to perform real tasks — for example, web search, browsing, scheduling, or coding assistance. Students of the masterclass can follow written modules or Jupyter notebooks that illustrate concepts step by step and progressively build more capable agents. ... -
15
Sideband
LXMF client for Android, Linux and macOS to communicate with people
Sideband is a privacy-focused, peer-to-peer mesh messaging protocol and platform that allows users to communicate directly over local networks or the internet without servers. It emphasizes metadata resistance and secure message delivery, working effectively in offline, low-bandwidth, or disaster recovery scenarios. Built-in Rust, Sideband supports encrypted group messaging, file transfers, and network self-discovery. It is ideal for use in community networks, crisis response, and... -
16
Pearl
A Production-ready Reinforcement Learning AI Agent Library
Pearl is a production-ready reinforcement learning and contextual bandit agent library built for real-world sequential decision making. It is organized around modular components—policy learners, replay buffers, exploration strategies, safety modules, and history summarizers—that snap together to form reliable agents with clear boundaries and strong defaults. The library implements classic and modern algorithms across two regimes: contextual bandits (e.g., LinUCB, LinTS, SquareCB, neural bandits) and fully sequential RL (e.g., DQN, PPO-style policy optimization), with attention to practical concerns like nonstationarity and dynamic action spaces. ... -
17
Inspect Petri
An alignment auditing agent capable of exploring alignment hypothesis
Inspect Petri is an open-source alignment auditing agent that lets researchers rapidly test concrete safety hypotheses against target models using realistic, multi-turn scenarios. Instead of building bespoke evals, Inspect Petri automatically generates audit environments from seed “special instructions,” orchestrates an auditor model to probe a target model, and simulates tool use and rollbacks to surface risky behaviors. Each interaction transcript is then scored by a judge model using a consistent rubric so results are comparable across runs and models. ... -
18
Claw Code
AI agent harness for AI coding agents
...It emphasizes harness engineering—how agents are structured, how they interact with tools, and how they maintain context during execution. The system is being actively expanded, with a Rust-based runtime in development to improve performance and memory safety. Overall, Claw Code serves as a research-driven platform for advancing agent-based software development systems. -
19
OpenHands
Open-source autonomous AI software engineer
...So we're building all our agents in the open on GitHub, under the MIT license. Our agents can do anything a human developer can: they write code, run commands, and use the web. We're partnering with AI safety experts like Invariant Labs to balance innovation with security. -
20
Heretic
Fully automatic censorship removal for language models
Heretic is an open-source Python tool that automatically removes the built-in censorship or “safety alignment” from transformer-based language models so they respond to a broader range of prompts with fewer refusals. It works by applying directional ablation techniques and a parameter optimization strategy to adjust internal model behaviors without expensive post-training or altering the core capabilities. Designed for researchers and advanced users, Heretic makes it possible to study and experiment with uncensored model responses in a reproducible, automated way. ... -
21
Supabase MCP Server
Query MCP enables end-to-end management of Supabase via chat interface
An open-source MCP server that enables comprehensive management of Supabase projects through natural language interactions, providing capabilities such as SQL execution, schema management, and API integration. -
22
highway-env
A minimalist environment for decision-making in autonomous driving
HighwayEnv is an OpenAI Gym-compatible environment focused on autonomous driving scenarios. It provides flexible simulations for testing decision-making algorithms in highway, intersection, and merging traffic situations. -
23
openpilot
Open source driver assistance system
openpilot is an open-source driver assistance system designed to improve upon the existing driver assistance of most modern cars today. openpilot gives you Tesla Autopilot-like functionality with functions like Adaptive Cruise Control (ACC), Automated Lane Centering (ALC), Forward Collision Warning (FCW) and Lane Departure Warning (LDW). All these with just a push of a button. openpilot also includes a camera-based driver monitoring feature that alerts distracted and asleep drivers while it... -
24
Django Migration Linter
Detect backward incompatible migrations for your django project
django-migration-linter is a tool that statically analyzes Django migration files to detect potentially dangerous operations. It flags migrations that might lock tables, drop columns, or perform expensive schema changes, helping teams catch issues before deployment. This tool is ideal for teams practicing continuous delivery or working on high-availability systems. -
25
A.I.G
Full-stack AI Red Teaming platform
AI-Infra-Guard is a powerful open-source security platform from Tencent’s Zhuque Lab designed to assess the safety and resilience of AI infrastructures, codebases, and components through automated scanning and evaluation tools. It brings together AI infrastructure vulnerability scanning, MCP server risk analysis, and jailbreak evaluation into a unified workflow so that enterprises and individuals can identify critical security issues without relying on external services.
