Search Results for "performance testing"
Sort By:
Showing 77 open source projects for "performance testing"
View related business solutions-
Build Data Resilience - Take the Assessment TodayIs your recovery strategy as strong as you think? Take this quick self-assessment to check your recovery readiness and gain tailored insights. In only 2 minutes, you'll learn where you fall on the recovery readiness scale.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Paddler
Open-source LLM load balancer and serving platform for hosting LLMs
Paddler is an open-source LLM infrastructure platform designed to deploy, manage, and scale large language models on private infrastructure. The system acts as a specialized load balancer and serving layer for language models, enabling organizations to run inference workloads without relying on external API providers. It supports running models locally through engines such as llama.cpp while distributing requests across multiple compute nodes to improve performance and reliability. The... -
2
Simple Evals
Lightweight framework for evaluating large language model performance
simple-evals is a lightweight evaluation framework developed by OpenAI for quickly testing models against small, focused benchmarks. It is designed to help researchers and developers run targeted evaluations without the complexity of large-scale pipelines. By emphasizing simplicity, the framework makes it easy to define new tasks, run evaluations, and interpret results in a reproducible way. It is particularly useful for sanity checks, exploratory research, and comparing performance across different models or configurations. ... -
3
mistral.rs
Fast, flexible LLM inference
mistral.rs is a fast and flexible LLM inference engine implemented in Rust, designed to run and serve modern language models with an emphasis on performance and practical deployment. It provides multiple entry points for developers, including a CLI for running models locally and an HTTP server that exposes an OpenAI-compatible API surface for easy integration with existing clients. The project includes hardware-aware tooling that can benchmark a system and choose sensible quantization and device-mapping strategies, helping users get strong performance without manual tuning. ... -
4
Giskard
Collaborative & Open-Source Quality Assurance for all AI models
The testing framework dedicated to ML models, from tabular to LLMs. Giskard is an open-source testing framework dedicated to ML models, from tabular models to LLMs. Testing Machine Learning applications can be tedious. Since ML models depend on data, testing scenarios depend on the domain specificities and are often infinite. At Giskard, we believe that Machine Learning needs its own testing framework. -
PRTG Catches Network Issues Before They Cause DowntimeReactive troubleshooting usually means hearing about a problem from frustrated users, not your monitoring tool. PRTG sets threshold-based alerts across devices, servers and applications, notifying your team by email, SMS or push the moment a metric crosses a set limit. That means catching a failing disk or overloaded server before it becomes an outage and getting time back from firefighting. Start a free trial and set your first alerts today.
-
5
Tianji
Evaluation suite designed to assess the performance of LLMs
Tianji is a comprehensive evaluation suite designed to assess the performance of large language models (LLMs) across multiple dimensions. It focuses on measuring general capabilities such as reasoning, knowledge, commonsense, and language understanding. Tianji provides a curated set of benchmarks and a unified framework for systematically comparing LLMs, making it useful for research and model selection. -
6
Arcade AI
Arcade Tool Development Kit (TDK), Worker, Evals, and CLI
...Core platform functionality and schemas. This repository contains the core Arcade libraries, organized as separate packages for maximum flexibility and modularity. Evaluation framework for testing tool performance. Test your MCP server's tools, resources, prompts, elicitation, and OAuth 2. MCPJam is compliant with the latest MCP specs. Connect to any MCP server. MCPJam inspector supports STDIO, SSE, and Streamable HTTP transports. -
7
Ollama Grid Search
A multi-platform desktop application to evaluate and compare LLM
...The system integrates directly with local or remote Ollama servers, enabling seamless access to models already deployed in a user’s environment. It also includes experiment logging and A/B testing capabilities, which allow users to compare outputs side by side and track performance metrics such as latency or token usage. -
8
Agent Behavior Monitoring
The open source post-building layer for agents
Agent Behavior Monitoring is an open-source framework designed to monitor, evaluate, and improve the behavior of AI agents operating in real or simulated environments. The system focuses on agent behavior monitoring by collecting interaction data and analyzing how agents perform across different scenarios and tasks. Developers can use the framework to observe agent actions in both online production environments and offline evaluation settings, making it useful for debugging and performance... -
9
Prompt flow
Build high-quality LLM apps
Prompt flow is a suite of development tools designed to streamline the end-to-end development cycle of LLM-based AI applications, from ideation, prototyping, testing, and evaluation to production deployment and monitoring. It makes prompt engineering much easier and enables you to build LLM apps with production quality. -
Earn up to 16% annual interest with Nexo.Generate interest, access liquidity without selling, and execute trades seamlessly. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
10
Nestia
NestJS Helper + AI Chatbot Development
Nestia is a high-performance toolkit and ecosystem built on top of NestJS that enhances backend development by introducing strongly typed APIs, automated SDK generation, and advanced tooling for scalable server applications. It is designed to eliminate much of the boilerplate typically associated with API development by leveraging pure TypeScript types to automatically generate validation logic, API documentation, and client SDKs. One of its defining advantages is its focus on performance... -
11
Colibrì
Run frontier MoE models on hardware you already own
...Written primarily in pure C, Colibrì has no core engine dependencies and can operate without a GPU, although CUDA, Metal, Vulkan, and other acceleration options can improve performance. A consistent command-line interface, OpenAI-compatible API, web dashboard, and desktop application support multiple model families, including GLM, DeepSeek, Kimi, Qwen, Inkling, and OLMoE. Colibrì also serves as an open research platform for testing model placement, caching, compression, speculative decoding, storage I/O, & more. -
12
AgentBench
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
...These environments require agents to interpret instructions, take actions, and adapt their strategies based on feedback from the environment. AgentBench also includes an evaluation framework that measures success rates, rewards, and task completion performance across different agent implementations. By testing models across diverse scenarios, the benchmark highlights strengths and weaknesses in reasoning, long-term planning, and tool usage. -
13
Claw Code
AI agent harness for AI coding agents
Claw Code is an open-source AI agent harness project focused on building better tools for orchestrating and managing autonomous coding agents. It originated as a clean-room reimplementation inspired by the architecture of Claude Code, aiming to replicate core concepts without using proprietary code. The project provides a Python-based foundation for experimenting with agent workflows, tool integration, and task execution pipelines. It emphasizes harness engineering—how agents are structured,... -
14
langrocks
Tools like web browser, computer access and code runner for LLMs
Langrocks is a programming language experimentation toolkit that enables developers to create, test, and optimize custom programming languages. -
15
Easy DataSet
A powerful tool for creating datasets for LLM fine-tuning
...The system includes automated question-generation capabilities, hierarchical label trees, and answer generation pipelines that use LLM APIs to produce coherent paired data with customizable templates. Beyond dataset creation, Easy-dataset also provides a built-in evaluation system with model testing and blind-test features, helping teams validate model performance using curated test sets. -
16
mlx
MLX: An array framework for Apple silicon
MlX offers a local web interface to browse, download, and run ML models via Hugging Face or local sources. It supports searching by tags or tasks, visualization of model metadata, quick inference demos, automatic setup of runtime environments, and works with PyTorch, TensorFlow, and ONNX. Ideal for researchers exploring and testing models via browser. -
17
forkd
Fork() for AI agent microVMs
...Instead of cold-booting a separate virtual machine for every worker, it forks children from a warmed parent snapshot and uses copy-on-write behavior to share the initial memory state. This makes it useful for spawning many isolated agent environments quickly during parallel research, testing, or code execution. The project is focused on runtime performance, sandbox isolation, and efficient branching from a live or prepared VM state. It is especially relevant for developers building agent infrastructure where many short-lived execution branches need to run safely. forkd is best understood as a low-level systems tool for scalable, isolated AI-agent workloads. -
18
Beehave
Behavior tree AI for Godot Engine
Beehave is a powerful AI behavior tree framework designed as an addon for the Godot game engine, enabling developers to create sophisticated and dynamic non-player character behaviors in games. It uses a node-based system that integrates directly into the Godot scene tree, allowing developers to visually design and organize complex AI logic in a structured and intuitive way. Behavior trees provide a modular approach to decision-making, making it easier to manage large and adaptive AI systems... -
19
NOFX
Open source AI trading OS for autonomous multi-model trading systems
NOFX is an open source AI-powered trading operating system designed to automate financial trading workflows using autonomous AI agents. It acts as an infrastructure layer that transforms market data into AI-driven trade decisions and execution. Instead of requiring users to manually configure machine learning models, data sources, and API integrations, the system allows AI components to perceive market conditions, select models, and perform trading actions automatically. It supports running... -
20
LLMFarm
llama and other large language models on iOS and MacOS offline
...It emphasizes modularity, allowing users to integrate different models, backends, or tools depending on their needs and hardware capabilities. LLMFarm is particularly useful for developers and researchers experimenting with local AI systems, as it lowers the barrier to entry for running and testing models without extensive setup. It also supports optimization techniques to improve performance on limited hardware, making it viable for smaller-scale deployments. -
21
GLM-4.1V
GLM-4.6V/4.5V/4.1V-Thinking, towards versatile multimodal reasoning
...Given its open-source availability under the same project repository, it provides an accessible entry point for testing multimodal reasoning and building proof-of-concept applications. -
22
Kiln
Open source platform for managing, testing, and deploying AI apps
...Kiln emphasizes reproducibility, enabling users to track changes to prompts and models while comparing outputs across different configurations. Kiln also supports systematic testing of AI systems by defining evaluation criteria and running experiments to assess performance over time. Its workflow-oriented approach helps teams move from experimentation to production by organizing assets and results in a consistent format. It is particularly useful for teams working with large language models who need visibility into how changes impact outputs and overall system quality. -
23
LangWatch
The platform for LLM evaluations and AI agent testing
LangWatch is an open-source observability and monitoring platform designed to help developers evaluate and improve applications built with large language models. The platform provides tools for tracking model interactions, analyzing prompt behavior, and identifying issues such as hallucinations, latency problems, or unexpected responses. By collecting telemetry data from AI applications, LangWatch allows developers to understand how their systems perform in real-world usage scenarios. The... -
24
AutoAgent AI
Autonomous harness engineering
...The system is particularly useful for building domain-specific agents that need continuous performance improvement. -
25
Mosec
A high-performance ML model serving framework, offers dynamic batching
Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API.
