Search Results for "benchmark testing"
Sort By:
Showing 21 open source projects for "benchmark testing"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Your monitoring isn't a stack. It's a pile. Fix that.Replace Datadog, New Relic, and Sentry without adding three more dashboards.
-
1
AgentBench
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
...These environments require agents to interpret instructions, take actions, and adapt their strategies based on feedback from the environment. AgentBench also includes an evaluation framework that measures success rates, rewards, and task completion performance across different agent implementations. By testing models across diverse scenarios, the benchmark highlights strengths and weaknesses in reasoning, long-term planning, and tool usage. -
2
AICGSecEval
A.S.E (AICGSecEval) is a repository-level AI-generated code security
AICGSecEval is an open-source benchmark framework designed to evaluate the security of code generated by artificial intelligence systems. The project was developed to address concerns that AI-assisted programming tools may produce insecure code containing vulnerabilities such as injection flaws or unsafe logic. The framework constructs evaluation tasks based on real-world software repositories and known vulnerability cases derived from CVE records. By simulating realistic development... -
3
Hallucination Leaderboard
Leaderboard Comparing LLM Performance at Producing Hallucinations
Hallucination Leaderboard is an open research project that tracks and compares the tendency of large language models to produce hallucinated or inaccurate information when generating summaries. The project provides a standardized benchmark that evaluates different models using a dedicated hallucination detection system known as the Hallucination Evaluation Model. Each model is tested on document summarization tasks to measure how often generated responses introduce information that is not... -
4
AutoAgent AI
Autonomous harness engineering
...Instead of manually tuning prompts or workflows, developers define high-level goals in a configuration file, and the system continuously modifies its own tools, orchestration, and logic based on benchmark performance. It operates through a loop of testing, analyzing failures, and refining the agent’s configuration to maximize a scoring metric. The framework uses a single-file agent harness combined with structured tasks and evaluation suites to guide optimization. It runs inside Docker for safe execution and reproducibility. This approach shifts agent development from manual design to automated optimization. ... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
5
RecBole
A unified, comprehensive and efficient recommendation library
A unified, comprehensive and efficient recommendation library. We design general and extensible data structures to unify the formatting and usage of various recommendation datasets. We implement more than 100 commonly used recommendation algorithms and provide formatted copies of 28 recommendation datasets. We support a series of widely adopted evaluation protocols or settings for testing and comparing recommendation algorithms. RecBole is developed based on Python and PyTorch for... -
6
ipbench
A benchmark suite for reliable testing of IP networks.
IPbench allows a distributed network of clients to hammer a server, collating performance figures from the clients (and optionally from the server) and then presenting them in an easly-analysable format. -
7
FuzzBench
FuzzBench - Fuzzer benchmarking as a service
FuzzBench is a large-scale, open research platform developed by Google to evaluate and benchmark fuzzers — automated software testing tools that detect vulnerabilities through randomized input generation. It provides a standardized, reproducible environment for comparing the performance and effectiveness of different fuzzing algorithms on real-world software targets. FuzzBench integrates with the OSS-Fuzz infrastructure, allowing it to run experiments on authentic open source projects and collect meaningful data on crash discovery rates, code coverage, and bug-finding efficiency. ... -
8
Crunch PNG
Insane(ly slow but wicked good) PNG image optimization
...It combines selective bit depth, color type, and color palette reduction with zopfli DEFLATE compression algorithm encoding using the pngquant and zopflipng PNG optimization tools. This approach leads to a significant file size gain relative to lossless approaches at the expense of a relatively modest decrease in image quality. Continuous benchmark testing is available in our GitHub Actions CI. Please see the benchmarks directory of this repository for details about the benchmarking approach and instructions on how to execute benchmarks locally on the reference images distributed in this repository or with your own image files. -
9
Fashion-MNIST
A MNIST-like fashion product database
...The dataset consists of 70,000 images in total, with 60,000 examples used for training and 10,000 reserved for testing. Each image has a resolution of 28 by 28 pixels and belongs to one of ten clothing classes, making it suitable for evaluating classification models. Because the dataset represents real-world objects rather than handwritten digits, it offers a more challenging benchmark for testing machine learning algorithms. -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
Grade School Math
8.5K high quality grade school math problems
The grade-school-math repository (sometimes called GSM8K) is a curated dataset of 8,500 high-quality grade school math word problems intended for evaluating mathematical reasoning capabilities of language models. It is structured into 7,500 training problems and 1,000 test problems. These aren’t trivial exercises — many require multi-step reasoning, combining arithmetic operations, and handling intermediate steps (e.g. “If she sold half as many in May… how many in total?”). The problems are... -
11
CRSLab
CRSLab is an open-source toolkit
CRSLab is an open-source toolkit for building Conversational Recommender System (CRS). It is developed based on Python and PyTorch. CRSLab has the following highlights. Comprehensive benchmark models and datasets: We have integrated commonly-used 6 datasets and 18 models, including graph neural network and pre-training models such as R-GCN, BERT and GPT-2. We have preprocessed these datasets to support these models, and release for downloading. Extensive and standard evaluation protocols: We support a series of widely-adopted evaluation protocols for testing and comparing different CRS. ... -
12
Yandex Tank
Load and performance benchmark tool
Yandex.Tank is an extensible open-source load testing tool for advanced Linux users which is especially good as a part of an automated load testing suite. Different load generators are supported. Evgeniy Mamchits' phantom is a very fast (100 000+ RPS) shooter written in C++ (default) JMeter is an extendable and widely known one. BFG is a Python-based generator that allows you to write your load scenarios in Python. Experimental Golang generator: pandora. Performance analytics backend... -
13
I3D models trained on Kinetics
Convolutional neural network model for video classification
Kinetics-I3D, developed by Google DeepMind, provides trained models and implementation code for the Inflated 3D ConvNet (I3D) architecture introduced in the paper “Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset” (CVPR 2017). The I3D model extends the 2D convolutional structure of Inception-v1 into 3D, allowing it to capture spatial and temporal information from videos for action recognition. This repository includes pretrained I3D models on the Kinetics dataset, with... -
14
RefineNet
RefineNet: Multi-Path Refinement Networks
...It implements the architecture presented in the CVPR 2017 paper RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation and its extended version published in TPAMI 2019. The framework uses multi-path refinement and improved residual pooling to achieve high-quality segmentation results across multiple benchmark datasets. It provides trained models for datasets such as PASCAL VOC 2012, Cityscapes, NYUDv2, Person_Parts, PASCAL_Context, SUNRGBD, and ADE20k, with versions based on ResNet-101 and ResNet-152 backbones. The repository supports both single-scale and multi-scale prediction, with scripts for training, testing, and evaluating segmentation performance. ... -
15
-
16
Moihack Port-Flooder
A simple TCP/UDP Port Flooder written in Python.
This is a simple Port Flooder written in Python 3.2 Use this tool to quickly stress test your network devices and measure your router's or server's load. Features are available in features section below. Moihack DoS Attack Tool was the name of the 1st version of the program. Moihack Port-Flooder is the Reloaded Version of the program with major code rewrite and changes. Code is much smaller in size now - from about 130 pure lines of codes to 35 lines only. To run it you must... -
17
Testing of the stability and quality of a network connection. This application helps users in testing the quality of a network connection and in sending feedback to the ISP.
-
18
Rugg is a hard drive and filesystem harness tool that allows you to test and benchmark drives and filesystems, by writing simple to complex scenarios that can mimic the behaviour of real-world applications.
-
19
This project is an approach to automating the testing of performance properties of complex systems.Just like functional specs for software we aim to develop an executable language for asserting performance expectations of a program.This implements Sharon
-
20
OpenLC is a set tools designed to facilitate benchmarking and stress testing of a wide variety of information servers (WEB, Email, FTP, LDAP, ...). A XML-RPC API is offered for developers interested in creating clients that query the services provid
-
21
auditpolCIS
Windows - CIS Benchmarks testing and SIEM Events Volume Optimisation
CIS Benchmark testing of Windows SIEM configuration. This is an application for testing the configuration of Windows Audit Policy settings against the CIS Benchmark recommended settings. Lots of organisations struggle with noisy Windows devices, especially if the SIEM vendor is charging by events volume. auditpolCIS points users in the direction of logging subcategories that can be disabled.
- Previous
- You're on page 1
- Next
