Search Results for "benchmark"
Sort By:
Showing 35 open source projects for "benchmark"
View related business solutions-
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
Stop Cyber Threats with VM-Series Next-Gen Firewall on AzureGain integrated visibility across all traffic in a single pass. Deploy Palo Alto Networks VM-Series to determine application identity and content while automating security policy updates via rich APIs.
-
1
SDGym
Benchmarking synthetic data generation methods
The Synthetic Data Gym (SDGym) is a benchmarking framework for modeling and generating synthetic data. Measure performance and memory usage across different synthetic data modeling techniques – classical statistics, deep learning and more! The SDGym library integrates with the Synthetic Data Vault ecosystem. You can use any of its synthesizers, datasets or metrics for benchmarking. You also customize the process to include your own work. Select any of the publicly available datasets from the... -
2
SIA
AI framework to autonomously improve the performance of any AI system
SIA is a self-improving AI framework designed to improve the performance of models or agents on benchmark tasks. It uses an iterative loop where a meta-agent creates or updates a task-specific target agent, while a feedback agent studies results and proposes improvements. The framework can refine both the harness around the task and the agent implementation itself. It is aimed at research and experimentation across tasks such as machine learning benchmarks, legal classification, code optimization, and scientific workflows. ... -
3
PyTorch Geometric
Geometric deep learning extension library for PyTorch
It consists of various methods for deep learning on graphs and other irregular structures, also known as geometric deep learning, from a variety of published papers. In addition, it consists of an easy-to-use mini-batch loader for many small and single giant graphs, a large number of common benchmark datasets (based on simple interfaces to create your own), and helpful transforms, both for learning on arbitrary graphs as well as on 3D meshes or point clouds. We have outsourced a lot of functionality of PyTorch Geometric to other packages, which needs to be additionally installed. These packages come with their own CPU and GPU kernel implementations based on C++/CUDA extensions. ... -
4
Recursive Language Models
General plug-and-play inference library for Recursive Language Models
...Within the framework, you can define custom agents, environments, policy networks, and reward structures while leveraging built-in dataset utilities, logging, and checkpointing for reproducible experiments. RLM also includes integration with popular simulation environments and benchmark suites, giving researchers a ready-made playground for algorithm comparison and performance tracking. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
Behaviour Suite Reinforcement Learning
bsuite is a collection of carefully-designed experiments
...Its main goal is to identify, measure, and analyze fundamental aspects of learning efficiency and generalization in RL algorithms. The library enables researchers to benchmark their agents on standardized tasks, facilitating reproducible and transparent comparisons across different approaches. Each experiment in bsuite is meticulously designed to capture key challenges in RL, such as exploration, credit assignment, and stability. The framework supports automated logging and analysis, generating standardized output compatible with Jupyter notebooks for streamlined evaluation. ... -
6
Pythonic Data Structures and Algorithms
Minimal examples of data structures and algorithms in Python
...For students preparing for technical interviews, self-learners brushing up on fundamentals, or developers wanting to understand algorithm internals, this repository provides ready-to-run examples, and can serve as a sandbox to experiment, benchmark, or adapt code. Because it’s in pure Python, it’s easy to read and modify, making it accessible even to those with modest programming experience. The repo helps bridge the gap between theoretical algorithm descriptions and real-world code, giving concrete, working implementations that one can study, debug, or extend. -
7
Python Outlier Detection
A Python toolbox for scalable outlier detection
...PyOD contains multiple models that also exist in scikit-learn. It is possible to train and predict with a large number of detection models in PyOD by leveraging SUOD framework. A benchmark is supplied for select algorithms to provide an overview of the implemented models. In total, 17 benchmark datasets are used for comparison, which can be downloaded at ODDS. -
8
TextDistance
Compute distance between sequences
...For main algorithms, text distance try to call known external libraries (fastest first) if available (installed in your system) and possible (this implementation can compare this type of sequences). Install text distance with extras for this feature. Textdistance use benchmark results for algorithm optimization and try to call the fastest external lib first (if possible). TextDistance show benchmarks results table for your system and saves libraries priorities into the libraries.json file in TextDistance's folder. This file will be used by text distance for calling the fastest algorithm implementation. ... -
9
RecBole
A unified, comprehensive and efficient recommendation library
A unified, comprehensive and efficient recommendation library. We design general and extensible data structures to unify the formatting and usage of various recommendation datasets. We implement more than 100 commonly used recommendation algorithms and provide formatted copies of 28 recommendation datasets. We support a series of widely adopted evaluation protocols or settings for testing and comparing recommendation algorithms. RecBole is developed based on Python and PyTorch for... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
Django Cachalot
No effort, no worry, maximum performance
...You will need a database called "cachalot" on MySQL and PostgreSQL. Additionally, on PostgreSQL, you will need to create a role called "cachalot." You can also run the benchmark, and it'll raise errors with specific instructions for how to fix it. Use cachalot for cold or modified <50 times per minutes (Most people should stick with only cachalot since you most likely won't need to scale to the point of needing cache-machine added to the bowl). -
11
PyG
Graph Neural Network Library for PyTorch
...In addition, it consists of easy-to-use mini-batch loaders for operating on many small and single giant graphs, multi GPU-support, DataPipe support, distributed graph learning via Quiver, a large number of common benchmark datasets (based on simple interfaces to create your own), the GraphGym experiment manager, and helpful transforms, both for learning on arbitrary graphs as well as on 3D meshes or point clouds. All it takes is 10-20 lines of code to get started with training a GNN model (see the next section for a quick tour). -
12
PI-Based Image Encoder / Converter
Python code able to convert / compress image to PI (3.14, π) Indexes
...Features high-performance Numba-accelerated search and a signature 'film-grain' aesthetic upon reconstruction. ZIP also include 16 MB file with 16,7 mil numbers of PI Benchmark(Single-Thread): Hardware & Environment Apple Silicon: Apple M2 (Mac mini/MacBook) x86_64 Platform: Intel Core Ultra 5 225F (Arrow Lake, 10 Cores) OS 1: Fedora 43 (GNOME) OS 2: Windows 11 Pro (23H2/24H2) Software: Python 3.14.3 + Numba JIT (latest) Results (Lower is better) Platform / OS CPU Time (Seconds) macOS (Native) Apple M2 52.151311 s (in default setup) Fedora Linux Intel Core Ultra 5 225F 58.536457 s (in default Power Management: Balanced) Windows 11 Intel Core Ultra 5 225F 59.681427 s (important! ... -
13
MMDeploy
OpenMMLab Model Deployment Framework
MMDeploy is an open-source deep learning model deployment toolset. It is a part of the OpenMMLab project. Models can be exported and run in several backends, and more will be compatible. All kinds of modules in the SDK can be extended, such as Transform for image processing, Net for Neural Network inference, Module for postprocessing and so on. Install and build your target backend. ONNX Runtime is a cross-platform inference and training accelerator compatible with many popular ML/DNN... -
14
DIG
A library for graph deep learning research
...It includes unified implementations of data interfaces, common algorithms, and evaluation metrics for several advanced tasks. Our goal is to enable researchers to easily implement and benchmark algorithms. -
15
FuzzBench
FuzzBench - Fuzzer benchmarking as a service
FuzzBench is a large-scale, open research platform developed by Google to evaluate and benchmark fuzzers — automated software testing tools that detect vulnerabilities through randomized input generation. It provides a standardized, reproducible environment for comparing the performance and effectiveness of different fuzzing algorithms on real-world software targets. FuzzBench integrates with the OSS-Fuzz infrastructure, allowing it to run experiments on authentic open source projects and collect meaningful data on crash discovery rates, code coverage, and bug-finding efficiency. ... -
16
AlphaTensor
AI discovers faster, efficient algorithms for matrix multiplication
...The repository is organized into four main components: algorithms, benchmarking, nonequivalence, and recombination. These contain implementations of the discovered matrix multiplication algorithms, tools to benchmark their real-world performance, proofs of nonequivalence among thousands of solutions, and methods for decomposing larger problems into smaller factorizations. Users can explore AlphaTensor’s discovered algorithms interactively using Colab notebooks or Python scripts. -
17
Detectron2
Next-generation platform for object detection and segmentation
Detectron2 is Facebook AI Research's next generation software system that implements state-of-the-art object detection algorithms. It is a ground-up rewrite of the previous version, Detectron, and it originates from maskrcnn-benchmark. It is powered by the PyTorch deep learning framework. Includes more features such as panoptic segmentation, Densepose, Cascade R-CNN, rotated bounding boxes, PointRend, DeepLab, etc. Can be used as a library to support different projects on top of it. We'll open source more research projects in this way. It trains much faster. ... -
18
DockStream
A Docking Wrapper to Enhance De Novo Molecular Design
DockStream is a docking wrapper providing access to a collection of ligand embedders and docking backends. Docking execution and post hoc analysis can be automated via the benchmarking and analysis workflow. The flexilibity to specifiy a large variety of docking configurations allows tailored protocols for diverse end applications. DockStream can also parallelize docking across CPU cores, increasing throughput. DockStream is integrated with the de novo design platform, REINVENT, allowing one... -
19
Awesome Graph Classification
Graph embedding, classification and representation learning papers
A collection of graph classification methods, covering embedding, deep learning, graph kernel and factorization papers with reference implementations. Relevant graph classification benchmark datasets are available. Similar collections about community detection, classification/regression tree, fraud detection, Monte Carlo tree search, and gradient boosting papers with implementations. -
20
gradslam
gradslam is an open source differentiable dense SLAM library
gradslam is an open-source framework providing differentiable building blocks for simultaneous localization and mapping (SLAM) systems. We enable the usage of dense SLAM subsystems from the comfort of PyTorch. The question of “representation” is central in the context of dense simultaneous localization and mapping (SLAM). Newer learning-based approaches have the potential to leverage data or task performance to directly inform the choice of representation. However, learning representations... -
21
Sparse Attention
"Generating Long Sequences with Sparse Transformers" examples
...It explores how modifying the self-attention mechanism with sparse patterns can reduce the quadratic scaling of standard transformers, making it possible to model much longer sequences efficiently. The repository provides implementations of sparse attention layers, training code, and evaluation scripts for benchmark datasets. It highlights both fixed and learnable sparsity patterns that trade off computational cost and model expressiveness. By enabling tractable training on longer contexts, the project opened the door to applications in large-scale text and image generation. Though archived, it remains a key reference for efficient transformer research, influencing many later architectures that aim to extend sequence length while reducing compute. -
22
Yandex Tank
Load and performance benchmark tool
Yandex.Tank is an extensible open-source load testing tool for advanced Linux users which is especially good as a part of an automated load testing suite. Different load generators are supported. Evgeniy Mamchits' phantom is a very fast (100 000+ RPS) shooter written in C++ (default) JMeter is an extendable and widely known one. BFG is a Python-based generator that allows you to write your load scenarios in Python. Experimental Golang generator: pandora. Performance analytics backend... -
23
I3D models trained on Kinetics
Convolutional neural network model for video classification
...This repository includes pretrained I3D models on the Kinetics dataset, with both RGB and optical flow input streams. The models have achieved state-of-the-art results on benchmark datasets such as UCF101 and HMDB51, and also won first place in the CVPR 2017 Charades Challenge. The project provides TensorFlow and Sonnet-based implementations, pretrained checkpoints, and example scripts for evaluating or fine-tuning models. It also offers sample data, including preprocessed video frames and optical flow arrays, to demonstrate how to run inference and visualize outputs. -
24
RefineNet
RefineNet: Multi-Path Refinement Networks
...It implements the architecture presented in the CVPR 2017 paper RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation and its extended version published in TPAMI 2019. The framework uses multi-path refinement and improved residual pooling to achieve high-quality segmentation results across multiple benchmark datasets. It provides trained models for datasets such as PASCAL VOC 2012, Cityscapes, NYUDv2, Person_Parts, PASCAL_Context, SUNRGBD, and ADE20k, with versions based on ResNet-101 and ResNet-152 backbones. The repository supports both single-scale and multi-scale prediction, with scripts for training, testing, and evaluating segmentation performance. ... -
25
coNCePTuaL
DSL for writing communication benchmarks
coNCePTuaL is a toolset for rapidly generating portable, readable, and reproducible network-performance tests. coNCePTuaL can perform the equivalent of many pages of C code with just a few mouse clicks or lines of code in a domain-specific language.
