Search Results for "benchmark" - Page 3
Sort By:
Showing 202 open source projects for "benchmark"
View related business solutions-
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
Paessler: Easy to Use With Enterprise Power. Free TrialYou shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
-
1
DFlash
Block Diffusion for Ultra-Fast Speculative Decoding
...This approach has been shown to deliver lossless acceleration on models like Qwen3-8B by combining block diffusion techniques with efficient batching, making it ideal for applications where latency and cost matter. The project includes support for multiple draft models, example integration code, and scripts to benchmark performance, and it is structured to work with popular model serving stacks like SGLang and the Hugging Face Transformers ecosystem. -
2
Step1X-Edit
A SOTA open-source image editing model
Step1X-Edit is a state-of-the-art open-source image editing model/framework that uses a multimodal large language model (LLM) together with a diffusion-based image decoder to let users edit images simply via natural-language instructions plus a reference image. You supply an existing image and a textual command — e.g. “add a ruby pendant on the girl’s neck” or “make the background a sunset over mountains” — and the model interprets the instruction, computes a latent embedding combining the... -
3
BrowserGym
A Gym environment for web task automation
BrowserGym is an open framework for web task automation research that exposes browser interaction as a Gym-style environment for training and evaluating agents. It is intended for researchers building web agents rather than for end users looking for a consumer automation product. The project provides a common environment where agents can interact with websites, execute tasks, and be evaluated against standardized benchmarks. One of its main strengths is that it bundles several important... -
4
Fara-7B
An Efficient Agentic Model for Computer Use
Fara-7B is a Microsoft initiative aimed at bringing rigor, transparency, and structured evaluation to AI systems through automated and customizable assessment frameworks. It provides stakeholders with a way to benchmark and evaluate models across dimensions such as fairness, robustness, security, privacy, and ethical considerations. Rather than relying on ad-hoc or manual review processes, FARA enables organizations to profile AI behavior using standardized tests, metrics, and reporting templates, making evaluations reproducible and comparable over time. ... -
Demo Series - Small Business Backup By VeeamWatch this on-demand demo series and learn how to protect your Microsoft 365 data with clear, simple, actionable steps that are easy to implement for businesses of all sizes.
-
5
Poetiq
Reproduction of Poetiq's record-breaking submission to the ARC-AGI-1
poetiq-arc-agi-solver is the open-source codebase from Poetiq that replicates their record-breaking submission to the challenging benchmark suite ARC-AGI (both ARC-AGI-1 and ARC-AGI-2). The project demonstrates a system that orchestrates large language models (LLMs) — like those from major providers — with carefully engineered prompting, reasoning workflows, and dynamic strategies, to tackle the abstract, logic-heavy problems in ARC-AGI. Instead of relying on a single prompt or fixed strategy, their solver dynamically adapts the reasoning path, selecting what to ask or analyze next depending on intermediate results — effectively compositing reasoning, perception, and program synthesis (or symbolic manipulation) in a loop. ... -
6
MemU
MemU is an open-source memory framework for AI companions
MemU is an agentic memory layer for LLM applications, specifically designed for AI companions. Transform your memory into an intelligent file system that automatically organizes, connects, and evolves with your memories. Simple, fast, and reliable memory infrastructure for AI applications. Powerful tools and dedicated support to scale your AI applications with confidence. Full proprietary features, commercial usage rights, and white-labeling options for your enterprise needs. SSO/RBAC... -
7
DeepSeek VL2
Mixture-of-Experts Vision-Language Models for Advanced Multimodal
DeepSeek-VL2 is DeepSeek’s vision + language multimodal model—essentially the next-gen successor to their first vision-language models. It combines image and text inputs into a unified embedding / reasoning space so that you can query with text and image jointly (e.g. “What’s going on in this scene?” or “Generate a caption appropriate to context”). The model supports both image understanding (vision tasks) and multimodal reasoning, and is likely used as a component in agent systems to... -
8
Habitat-Lab
A modular high-level library to train embodied AI agents
...Providing algorithms for single and multi-agent training (via imitation or reinforcement learning, or no learning at all as in SensePlanAct pipelines), as well as tools to benchmark their performance on the defined tasks using standard metrics. -
9
AIDE ML
AI-Driven Exploration in the Space of Code
AIDE ML is an open-source research framework designed to explore automated machine learning development through agent-based search and code optimization. The project implements the AIDE algorithm, which uses a tree-search strategy guided by large language models to iteratively generate, evaluate, and refine code. Instead of relying on manual experimentation, the agent autonomously drafts machine learning pipelines, debugs errors, and benchmarks performance against user-defined evaluation... -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
DriveLM
Driving with Graph Visual Question Answering
DriveLM is a research-oriented framework and dataset designed to explore how vision-language models can be integrated into autonomous driving systems. The project introduces a new paradigm called graph visual question answering that structures reasoning about driving scenes through interconnected tasks such as perception, prediction, planning, and motion control. Instead of treating autonomous driving as a purely sensor-driven pipeline, DriveLM frames it as a reasoning problem where models... -
11
LongWriter
Unleashing 10,000+ Word Generation from Long Context LLMs
LongWriter is an open-source framework and set of large language models designed to enable ultra-long text generation that can exceed 10,000 words while maintaining coherence and structure. Traditional large language models can process large inputs but often struggle to generate long outputs due to limitations in training data and alignment strategies. LongWriter addresses this challenge by introducing a specialized dataset and training approach that encourages models to produce longer... -
12
mini SWE-agent
The 100 line AI agent that solves GitHub issues
mini SWE-agent is a lightweight, minimalist AI-powered software engineering agent designed to autonomously solve GitHub issues and assist developers directly from the command line using large language models. Unlike more complex frameworks, it emphasizes simplicity and efficiency, consisting of roughly 100 lines of code while still achieving strong performance on benchmarks such as SWE-bench Verified, where it demonstrates competitive problem-solving capabilities. The agent operates by... -
13
TTRL
Test-Time Reinforcement Learning
TTRL is an open-source framework for test-time reinforcement learning in large language models, with a particular focus on reasoning tasks where ground-truth labels are not available during inference. The project addresses the problem of how to generate useful reward signals from unlabeled test-time data, and its central insight is that common test-time scaling practices such as majority voting can be repurposed into reward estimates for online reinforcement learning. This makes the... -
14
MiroThinker
MiroThinker is an open source deep research agent
MiroThinker is an open-source deep research AI agent designed to perform complex reasoning, information gathering, and predictive analysis tasks. The system focuses on enabling long-horizon research workflows by allowing the agent to interact repeatedly with external tools, search systems, and data sources while refining its reasoning through iterative steps. Rather than simply generating responses from a single prompt, the agent performs structured multi-step reasoning processes that... -
15
VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs)
...The toolkit provides a unified framework that allows researchers and developers to evaluate multimodal models across a wide range of datasets and standardized benchmarks with minimal setup. Instead of requiring complex data preparation pipelines or multiple repositories for each benchmark, the system enables evaluation through simple commands that automatically handle dataset loading, model inference, and metric computation. VLMEvalKit supports generation-based evaluation methods, allowing models to produce textual responses to visual inputs while measuring performance through techniques such as exact matching or language-model-assisted answer extraction. -
16
CodeGen
Open-source model for program synthesis
...The project also includes training infrastructure and model checkpoints that allow researchers to experiment with different model sizes and training configurations. Its architecture and training approach enable the models to perform competitively with proprietary coding models on benchmark tasks. -
17
Anthropic's Original Performance
Anthropic's original performance take-home, now open for you to try
Anthropic's Original Performance repository contains the publicly released version of a performance challenge originally used by Anthropic as part of their technical interview process, offering developers the opportunity to optimize and benchmark low-level code against simulated models. The project sets up a baseline performance problem where participants work to reduce simulated “clock cycles” required to run a given workload, effectively challenging them to engineer faster code under constraints. This take-home includes starter code, tests, and tools to debug performance, aiming to measure how effectively one can apply algorithmic improvements and optimizations. ... -
18
Coconut
Training Large Language Model to Reason in a Continuous Latent Space
Coconut is the official PyTorch implementation of the research paper “Training Large Language Models to Reason in a Continuous Latent Space.” The framework introduces a novel method for enhancing large language models (LLMs) with continuous latent reasoning steps, enabling them to generate and refine reasoning chains within a learned latent space rather than relying solely on discrete symbolic reasoning. It supports training across multiple reasoning paradigms—including standard... -
19
Agent S
Agent S: an open agentic framework that uses computers like a human
...Built to operate graphical user interfaces like a human, it allows AI agents to perceive screens, reason about tasks, and execute actions across macOS, Windows, and Linux systems. The latest version, Agent S3, surpasses human-level performance on the OSWorld benchmark, demonstrating state-of-the-art results in complex multi-step computer tasks. Agent S combines powerful foundation models (such as GPT-5) with grounding models like UI-TARS to translate visual inputs into precise executable actions. It supports flexible deployment via CLI, SDK, or cloud, and integrates with multiple model providers including OpenAI, Anthropic, Gemini, Azure, and Hugging Face endpoints. ... -
20
RecBole
A unified, comprehensive and efficient recommendation library
A unified, comprehensive and efficient recommendation library. We design general and extensible data structures to unify the formatting and usage of various recommendation datasets. We implement more than 100 commonly used recommendation algorithms and provide formatted copies of 28 recommendation datasets. We support a series of widely adopted evaluation protocols or settings for testing and comparing recommendation algorithms. RecBole is developed based on Python and PyTorch for... -
21
Tracking Any Point (TAP)
DeepMind model for tracking arbitrary points across videos & robotics
TAPNet is the official Google DeepMind repository for Tracking Any Point (TAP), bundling datasets, models, benchmarks, and demos for precise point tracking in videos. The project includes the TAP-Vid and TAPVid-3D benchmarks, which evaluate long-range tracking of arbitrary points in 2D and 3D across diverse real and synthetic videos. Its flagship models—TAPIR, BootsTAPIR, and the latest TAPNext—use matching plus temporal refinement or next-token style propagation to achieve state-of-the-art... -
22
PyTorch Geometric Temporal
Spatiotemporal Signal Processing with Neural Machine Learning Models
...Moreover, it comes with an easy-to-use dataset loader, train-test splitter and temporal snaphot iterator for dynamic and temporal graphs. The framework naturally provides GPU support. It also comes with a number of benchmark datasets from the epidemiological forecasting, sharing economy, energy production and web traffic management domains. Finally, you can also create your own datasets. The package interfaces well with Pytorch Lightning which allows training on CPUs, single and multiple GPUs out-of-the-box. PyTorch Geometric Temporal makes implementing Dynamic and Temporal Graph Neural Networks quite easy - see the accompanying tutorial. ... -
23
Tongyi DeepResearch
Tongyi Deep Research, the Leading Open-source Deep Research Agent
DeepResearch (Tongyi DeepResearch) is an open-source “deep research agent” developed by Alibaba’s Tongyi Lab designed for long-horizon, information-seeking tasks. It’s built to act like a research agent: synthesizing, reasoning, retrieving information via the web and documents, and backing its outputs with evidence. The model is about 30.5 billion parameters in size, though at any given token only ~3.3B parameters are active. It uses a mix of synthetic data generation, fine-tuning and... -
24
TimeMixer
Decomposable Multiscale Mixing for Time Series Forecasting
TimeMixer is a deep learning framework designed for advanced time series forecasting and analysis using a multiscale neural architecture. The model focuses on decomposing time series data into multiple temporal scales in order to capture both short-term seasonal patterns and long-term trends. Instead of relying on traditional recurrent or transformer-based architectures, TimeMixer is implemented as a fully multilayer perceptron–based model that performs temporal mixing across different... -
25
MiroFlow
Agent framework that enables tool-use agent tasks
MiroFlow is a high-performance open-source framework designed for building intelligent AI agents capable of solving complex reasoning and research tasks. The system introduces a hierarchical architecture that organizes components into control, agent, and foundation layers, allowing developers to manage agent orchestration and tool interactions in a structured manner. One of the core innovations of MiroFlow is its use of agent graphs, which enable flexible orchestration of multiple sub-agents...
