Search Results for "benchmark testing"
Sort By:
Showing 14 open source projects for "benchmark testing"
View related business solutions-
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
AgentBench
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
...These environments require agents to interpret instructions, take actions, and adapt their strategies based on feedback from the environment. AgentBench also includes an evaluation framework that measures success rates, rewards, and task completion performance across different agent implementations. By testing models across diverse scenarios, the benchmark highlights strengths and weaknesses in reasoning, long-term planning, and tool usage. -
2
AICGSecEval
A.S.E (AICGSecEval) is a repository-level AI-generated code security
AICGSecEval is an open-source benchmark framework designed to evaluate the security of code generated by artificial intelligence systems. The project was developed to address concerns that AI-assisted programming tools may produce insecure code containing vulnerabilities such as injection flaws or unsafe logic. The framework constructs evaluation tasks based on real-world software repositories and known vulnerability cases derived from CVE records. By simulating realistic development... -
3
Hallucination Leaderboard
Leaderboard Comparing LLM Performance at Producing Hallucinations
Hallucination Leaderboard is an open research project that tracks and compares the tendency of large language models to produce hallucinated or inaccurate information when generating summaries. The project provides a standardized benchmark that evaluates different models using a dedicated hallucination detection system known as the Hallucination Evaluation Model. Each model is tested on document summarization tasks to measure how often generated responses introduce information that is not... -
4
AutoAgent AI
Autonomous harness engineering
...Instead of manually tuning prompts or workflows, developers define high-level goals in a configuration file, and the system continuously modifies its own tools, orchestration, and logic based on benchmark performance. It operates through a loop of testing, analyzing failures, and refining the agent’s configuration to maximize a scoring metric. The framework uses a single-file agent harness combined with structured tasks and evaluation suites to guide optimization. It runs inside Docker for safe execution and reproducibility. This approach shifts agent development from manual design to automated optimization. ... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
mistral.rs
Fast, flexible LLM inference
...It provides multiple entry points for developers, including a CLI for running models locally and an HTTP server that exposes an OpenAI-compatible API surface for easy integration with existing clients. The project includes hardware-aware tooling that can benchmark a system and choose sensible quantization and device-mapping strategies, helping users get strong performance without manual tuning. It also supports serving multiple models from the same server process, enabling routing or quick switching between models depending on workload needs. For user-facing testing, mistral.rs can provide a built-in web UI, and it also offers a dedicated lightweight web chat interface that supports richer interaction patterns. -
6
RecBole
A unified, comprehensive and efficient recommendation library
A unified, comprehensive and efficient recommendation library. We design general and extensible data structures to unify the formatting and usage of various recommendation datasets. We implement more than 100 commonly used recommendation algorithms and provide formatted copies of 28 recommendation datasets. We support a series of widely adopted evaluation protocols or settings for testing and comparing recommendation algorithms. RecBole is developed based on Python and PyTorch for... -
7
MiniMax-M2.5
State of the art LLM and coding model
MiniMax-M2.5 is a state-of-the-art foundation model extensively trained with reinforcement learning across hundreds of thousands of real-world environments. It delivers leading performance in coding, agentic tool use, search, and complex office workflows, achieving top benchmark scores such as 80.2% on SWE-Bench Verified and 76.3% on BrowseComp. Designed to reason efficiently and decompose tasks like an experienced architect, M2.5 plans features, structure, and system design before generating code. The model supports full-stack development across web, mobile, and desktop platforms, covering the entire lifecycle from system design to testing and code review. ... -
8
Safety-Prompts
Chinese safety prompts for evaluating and improving the safety of LLMs
Safety-Prompts is an open-source repository that provides a curated collection of prompts designed to evaluate and improve the safety behavior of large language models. The project focuses primarily on safety testing scenarios relevant to Chinese language models, though the concepts can be applied to other languages and systems. The prompts are structured to test whether models generate outputs that align with human values and safety guidelines when faced with potentially harmful or sensitive requests. Researchers and developers use the dataset to benchmark how well models avoid unsafe responses and follow alignment constraints. ... -
9
Fashion-MNIST
A MNIST-like fashion product database
...The dataset consists of 70,000 images in total, with 60,000 examples used for training and 10,000 reserved for testing. Each image has a resolution of 28 by 28 pixels and belongs to one of ten clothing classes, making it suitable for evaluating classification models. Because the dataset represents real-world objects rather than handwritten digits, it offers a more challenging benchmark for testing machine learning algorithms. -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
10
Procgen
Procedurally-Generated Game-Like Gym-Environments
...The environments are designed to run very quickly (thousands of steps per second on a single core) to facilitate large-scale experiments and make benchmarking efficient. The benchmark supports both “easy” and “hard” difficulty modes, letting researchers trade off computational cost vs challenge. The repo provides a C++ core for game logic and rendering (with support for gym/Gym3 wrappers) plus Python bindings and interactive mode for human play testing. -
11
Grade School Math
8.5K high quality grade school math problems
The grade-school-math repository (sometimes called GSM8K) is a curated dataset of 8,500 high-quality grade school math word problems intended for evaluating mathematical reasoning capabilities of language models. It is structured into 7,500 training problems and 1,000 test problems. These aren’t trivial exercises — many require multi-step reasoning, combining arithmetic operations, and handling intermediate steps (e.g. “If she sold half as many in May… how many in total?”). The problems are... -
12
CRSLab
CRSLab is an open-source toolkit
CRSLab is an open-source toolkit for building Conversational Recommender System (CRS). It is developed based on Python and PyTorch. CRSLab has the following highlights. Comprehensive benchmark models and datasets: We have integrated commonly-used 6 datasets and 18 models, including graph neural network and pre-training models such as R-GCN, BERT and GPT-2. We have preprocessed these datasets to support these models, and release for downloading. Extensive and standard evaluation protocols: We support a series of widely-adopted evaluation protocols for testing and comparing different CRS. ... -
13
Tiny
Tiny Face Detector, CVPR 2017
...The method is designed to detect tiny faces (i.e. very small-scale faces) by combining multi-scale context modeling, foveal descriptors, and scale enumeration strategies. It provides training/testing scripts, a demo (tiny_face_detector.m), model loading, evaluation on WIDER FACE, and supporting utilities (e.g. cnn_widerface_eval.m). The code depends on MatConvNet, which must be compiled (with GPU / CUDA / cuDNN support) for full performance. Pretrained model provided (ResNet101-based, plus alternatives). Demo and evaluation scripts for benchmark datasets. ... -
14
A test suite and benchmark for exact Euclidean distance transform algorithms used in Image Processing and computational geometry. It evaluates the exactness and speed of algorithms for a large number of test cases. Results can be visualized in Scilab.
- Previous
- You're on page 1
- Next
