Search Results for "Observability Open Source & DevTools"
Sort By:
Showing 30 open source projects for "Observability Open Source & DevTools"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
$300 in Free Credit Towards Top Cloud ServicesStart your project in minutes. After credits run out, 20+ products include free monthly usage. Only pay when you're ready to scale.
-
1
Arize Phoenix
Uncover insights, surface problems, monitor, and fine tune your LLM
Phoenix provides ML insights at lightning speed with zero-config observability for model drift, performance, and data quality. Phoenix is an Open Source ML Observability library designed for the Notebook. The toolset is designed to ingest model inference data for LLMs, CV, NLP and tabular datasets. It allows Data Scientists to quickly visualize their model data, monitor performance, track down issues & insights, and easily export to improve. -
2
OpenLLMetry
Open-source observability for your LLM application
The repo contains standard OpenTelemetry instrumentations for LLM providers and Vector DBs, as well as a Traceloop SDK that makes it easy to get started with OpenLLMetry, while still outputting standard OpenTelemetry data that can be connected to your observability stack. If you already have OpenTelemetry instrumented, you can just add any of our instrumentations directly. -
3
OpenLIT
OpenLIT is an open-source LLM Observability tool
OpenLIT is an OpenTelemetry-native tool designed to help developers gain insights into the performance of their LLM applications in production. It automatically collects LLM input and output metadata and monitors GPU performance for self-hosted LLMs. OpenLIT makes integrating observability into GenAI projects effortless with just a single line of code. Whether you're working with popular LLM providers such as OpenAI and HuggingFace, or leveraging vector databases like ChromaDB, OpenLIT... -
4
LangKit
An open-source toolkit for monitoring Language Learning Models (LLMs)
LangKit is an open-source text metrics toolkit for monitoring language models. It offers an array of methods for extracting relevant signals from the input and/or output text, which are compatible with the open-source data logging library whylogs. Productionizing language models, including LLMs, comes with a range of risks due to the infinite amount of input combinations, which can elicit an infinite amount of outputs. -
Earn up to 16% annual interest with Nexo.Generate interest, access liquidity without selling, and execute trades seamlessly. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
5
Pydantic Logfire
Python observability platform for tracing apps, metrics, and logs
Pydantic Logfire is an observability platform designed to help developers monitor, analyze, and understand the behavior of their applications in real time. It is built by the team behind Pydantic and follows a philosophy of combining powerful capabilities with ease of use, making it accessible to entire engineering teams. Pydantic Logfire provides deep visibility into application performance by capturing traces, metrics, and logs through an OpenTelemetry-based architecture. It is... -
6
LangCheck
Simple, Pythonic building blocks to evaluate LLM applications
Simple, Pythonic building blocks to evaluate LLM applications. -
7
HolmesGPT
CNCF Sandbox Project
HolmesGPT is an open-source AI agent designed to help DevOps and site reliability engineering teams diagnose and resolve production incidents. The system aggregates signals from observability tools such as logs, metrics, alerts, and distributed traces, then analyzes them using large language models to identify potential root causes. Rather than requiring engineers to manually correlate large volumes of monitoring data, HolmesGPT automatically synthesizes evidence and presents explanations in natural language. ... -
8
Pruna AI
Pruna is a model optimization framework built for developers
Pruna is an open-source, self-hostable AI inference engine designed to help teams deploy and manage large language models (LLMs) efficiently across private or hybrid infrastructures. Built with performance and developer ergonomics in mind, Pruna simplifies inference workflows by enabling multi-model orchestration, autoscaling, GPU resource allocation, and compatibility with popular open-source models. -
9
OpenSRE
Build your own AI SRE agents. The open source toolkit for the AI era
OpenSRE is an open-source framework designed to build AI-powered Site Reliability Engineering agents that automate incident investigation and root cause analysis across modern cloud environments. It connects to observability tools, infrastructure systems, and communication platforms to gather logs, metrics, and traces in real time. When an alert is triggered, the system autonomously analyzes correlated signals, identifies anomalies, and generates structured investigation reports with probable causes and recommended actions. ... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
Jina
Build cross-modal and multimodal applications on the cloud
Jina is a framework that empowers anyone to build cross-modal and multi-modal applications on the cloud. It uplifts a PoC into a production-ready service. Jina handles the infrastructure complexity, making advanced solution engineering and cloud-native technologies accessible to every developer. Build applications that deliver fresh insights from multiple data types such as text, image, audio, video, 3D mesh, PDF with Jina AI’s DocArray. Polyglot gateway that supports gRPC, Websockets, HTTP,... -
11
Swarms
Enterprise multi-agent orchestration framework for scalable AI apps
Swarms is an enterprise-grade multi-agent orchestration framework designed to help developers build, manage, and scale collaborative AI systems composed of multiple agents. It provides a structured infrastructure for coordinating agents in hierarchical, parallel, or sequential workflows, enabling complex task execution across distributed components. It emphasizes production readiness, offering modular architecture, high availability, and observability features suitable for large-scale... -
12
Aden Hive
Outcome driven agent development framework that evolves
Hive is an open-source agent development framework that helps developers build autonomous, reliable, self-improving AI agents by letting them describe goals in ordinary natural language instead of hand-coding detailed workflows. Rather than manually defining execution graphs, Hive’s coding agent generates the agent graph, connection code, and test cases based on your high-level objectives, enabling outcome-driven agent creation that fits real business processes. -
13
ControlFlow
Take control of your AI agents
ControlFlow is an open-source Python framework developed to help engineers design and orchestrate agentic workflows powered by large language models. The framework provides a structured approach for building AI systems by breaking complex tasks into smaller units called tasks that can be assigned to specialized AI agents. Developers can combine these tasks into flows that define how work is executed, enabling the creation of multi-step reasoning pipelines and collaborative agent systems. ... -
14
ContextForge MCP Gateway
A Model Context Protocol (MCP) Gateway & Registry
MCP Context Forge is a feature-rich gateway and registry that federates Model Context Protocol (MCP) servers and traditional REST services behind a single, governed endpoint. It exposes an MCP-compliant interface to clients while handling discovery, authentication, rate limiting, retries, and observability on the server side. The gateway scales horizontally, supports multi-cluster deployments on Kubernetes, and uses Redis for federation and caching across instances. Operators can define... -
15
NVIDIA NeMo Agent Toolkit
Library for efficiently connecting and optimizing teams of AI agents
NVIDIA NeMo Agent Toolkit is an open-source framework designed to build, optimize, and manage AI agents across different development ecosystems. It provides enterprise-grade tools for improving agent performance, reliability, and observability throughout the development lifecycle. The toolkit integrates with popular agent frameworks such as LangChain, LlamaIndex, CrewAI, Microsoft Semantic Kernel, and Google ADK. -
16
Acontext
Context data platform for building observable, self-learning AI agents
Acontext is a cloud-native context data platform designed to support the development and operation of advanced AI agents. It provides a unified system to store and manage contexts, multimodal messages, artifacts, and task workflows, enabling developers to engineer context effectively for their agent products. The platform observes agent tasks and user feedback in real time, offering robust observability into workflows and helping teams understand how agents perform over time. Acontext also... -
17
Agent Control
Centralized agent control plane for governing runtime agent behavior
Agent Control is a centralized control plane for governing AI agent behavior at runtime across different frameworks and deployment environments. It lets teams define controls once and apply them consistently to agents without rewriting the agent’s core code. The platform evaluates agent inputs and outputs against configurable policies to reduce risks such as prompt injection, unsafe responses, sensitive data exposure, and policy drift. It is designed for production environments where... -
18
Browser Harness
Self-healing browser harness that enables LLMs to complete any task
Browser Harness is a self-healing browser control system built to give language models direct and flexible access to a real Chrome browser through the Chrome DevTools Protocol. Its main philosophy is minimalism: instead of imposing a rigid framework, it exposes a very thin bridge so the agent can perform browser tasks with almost no abstraction in the way. A defining part of the project is that the agent can write or extend missing helper functions during a task, which is why the repository... -
19
Ragas
Supercharge Your LLM Application Evaluations
Objective metrics, intelligent test generation, and data-driven insights for LLM apps. Ragas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications. Say goodbye to time-consuming, subjective assessments and hello to data-driven, efficient evaluation workflows. Don't have a test dataset ready? We also do production-aligned test set generation. -
20
ACI.dev
Open platform connecting AI agents to tools via unified MCP server
ACI is an open source platform designed to enable AI agents to interact with external tools through a unified and structured interface. It focuses on simplifying tool integration by connecting hundreds of pre-built services into agentic environments, allowing developers to avoid building custom API clients and authentication flows for each service. ACI provides intent-aware tool access, meaning agents can dynamically discover and use tools based on context rather than rigid configurations.... -
21
Agent Starter Pack
Ship AI Agents to Google Cloud in minutes, not months
Agent Starter Pack is a production-focused framework that provides pre-built templates and infrastructure for rapidly developing and deploying generative AI agents on Google Cloud. It is designed to eliminate the complexity of moving from prototype to production by bundling essential components such as deployment pipelines, monitoring, security, and evaluation tools into a single package. Developers can create fully functional agent projects with a single command, generating both backend and... -
22
Claude Code Usage Monitor
Real-time Claude Code usage monitor with predictions and warnings
Claude Code Usage Monitor is a developer-focused terminal tool that provides real-time visibility into Claude Code token consumption and session behavior. The project is designed to help users avoid unexpectedly hitting usage caps by continuously tracking token burn rate, message counts, and estimated costs during active sessions. It presents analytics through a visually rich terminal interface built with modern Python tooling, making it easy to interpret usage trends at a glance. The system... -
23
BeeAI Framework
Build production-ready AI agents in both Python and Typescript
BeeAI Framework is an open-source, production-grade toolkit designed for building intelligent AI agents and complex multi-agent systems that can reason, act, and collaborate to solve real-world problems at scale. It goes beyond simple prompt-based interactions by introducing rule-based governance and constraint enforcement, enabling developers to create agents with predictable and controllable behavior while still preserving advanced reasoning capabilities. The framework supports both Python... -
24
Langroid
Harness LLMs with Multi-Agent Programming
Given the remarkable abilities of recent Large Language Models (LLMs), there is an unprecedented opportunity to build intelligent applications powered by this transformative technology. The top question for any enterprise is: how best to harness the power of LLMs for complex applications? For technical and practical reasons, building LLM-powered applications is not as simple as throwing a task at an LLM system and expecting it to do it. Effectively leveraging LLMs at scale requires a... -
25
Self-hosted AI Package
Run all your local AI together in one package
Self-hosted AI Package is an open-source Docker Compose-based starter kit that makes it easy to bootstrap a full local AI and low-code development environment with commonly used open tools, empowering developers to run LLMs and AI workflows entirely on their infrastructure. The stack typically includes Ollama for running local large language models, n8n as a low-code workflow automation platform, Supabase for database and vector storage, Open WebUI for interacting with models, Flowise for agent building, and additional services like SearXNG, Neo4j, and Langfuse for search, knowledge graphs, and observability. ...
