Search Results for "benchmark" - Page 3
Sort By:
Showing 177 open source projects for "benchmark"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
1
VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs)
...The toolkit provides a unified framework that allows researchers and developers to evaluate multimodal models across a wide range of datasets and standardized benchmarks with minimal setup. Instead of requiring complex data preparation pipelines or multiple repositories for each benchmark, the system enables evaluation through simple commands that automatically handle dataset loading, model inference, and metric computation. VLMEvalKit supports generation-based evaluation methods, allowing models to produce textual responses to visual inputs while measuring performance through techniques such as exact matching or language-model-assisted answer extraction. -
2
CodeGen
Open-source model for program synthesis
...The project also includes training infrastructure and model checkpoints that allow researchers to experiment with different model sizes and training configurations. Its architecture and training approach enable the models to perform competitively with proprietary coding models on benchmark tasks. -
3
Coconut
Training Large Language Model to Reason in a Continuous Latent Space
Coconut is the official PyTorch implementation of the research paper “Training Large Language Models to Reason in a Continuous Latent Space.” The framework introduces a novel method for enhancing large language models (LLMs) with continuous latent reasoning steps, enabling them to generate and refine reasoning chains within a learned latent space rather than relying solely on discrete symbolic reasoning. It supports training across multiple reasoning paradigms—including standard... -
4
Qwen-Image
Qwen-Image is a powerful image generation foundation model
Qwen-Image is a powerful 20-billion parameter foundation model designed for advanced image generation and precise editing, with a particular strength in complex text rendering across diverse languages, especially Chinese. Built on the MMDiT architecture, it achieves remarkable fidelity in integrating text seamlessly into images while preserving typographic details and layout coherence. The model excels not only in text rendering but also in a wide range of artistic styles, including... -
Stop vibe-debugging.AppSignal's MCP server hands Claude, Cursor, or Zed your real errors, traces, and the deploy that shipped them. AI writes the fix; you review the diff.
-
5
Gemma in PyTorch
The official PyTorch implementation of Google's Gemma models
...The repository demonstrates text generation pipelines, tokenizer setup, quantization paths, and adapters for low-rank or parameter-efficient fine-tuning. Example notebooks walk through instruction tuning and evaluation so teams can benchmark and iterate rapidly. The code is organized to be legible and hackable, exposing attention blocks, positional encodings, and head configurations. With standard PyTorch abstractions, it integrates easily into existing training loops, loggers, and evaluation harnesses. -
6
MemU
MemU is an open-source memory framework for AI companions
MemU is an agentic memory layer for LLM applications, specifically designed for AI companions. Transform your memory into an intelligent file system that automatically organizes, connects, and evolves with your memories. Simple, fast, and reliable memory infrastructure for AI applications. Powerful tools and dedicated support to scale your AI applications with confidence. Full proprietary features, commercial usage rights, and white-labeling options for your enterprise needs. SSO/RBAC... -
7
DeepSeek VL2
Mixture-of-Experts Vision-Language Models for Advanced Multimodal
DeepSeek-VL2 is DeepSeek’s vision + language multimodal model—essentially the next-gen successor to their first vision-language models. It combines image and text inputs into a unified embedding / reasoning space so that you can query with text and image jointly (e.g. “What’s going on in this scene?” or “Generate a caption appropriate to context”). The model supports both image understanding (vision tasks) and multimodal reasoning, and is likely used as a component in agent systems to... -
8
MiniMax-M2.1
MiniMax M2.1, a SOTA model for real-world dev & agents.
MiniMax-M2.1 is an open-source, state-of-the-art agentic language model released to democratize high-performance AI capabilities. It goes beyond a simple parameter upgrade, delivering major gains in coding, tool use, instruction following, and long-horizon planning. The model is designed to be transparent, controllable, and accessible, enabling developers to build autonomous systems without relying on closed platforms. MiniMax-M2.1 excels in real-world software engineering tasks, including... -
9
OpenOCR
An Open-Source Toolkit for General-OCR Research and Applications
OpenOCR is an open-source General OCR toolkit developed by the OCR team at Fudan University for research and real-world document processing applications. It provides a unified platform for text detection, text recognition, formula recognition, table recognition, and document parsing. Built on advanced OCR technologies such as SVTRv2 and UniRec-0.1B, OpenOCR delivers high accuracy while maintaining efficient inference performance. The toolkit supports both Chinese and English content, making... -
Our Free Plans just got better! | Auth0You asked, we delivered! Auth0 is excited to expand our Free and Paid plans to include more options so you can focus on building, deploying, and scaling applications without having to worry about your security. Auth0 now, thank yourself later.
-
10
Diplomacy Cicero
Code for Cicero, an AI agent that plays the game of Diplomacy
...The codebase is implemented primarily in Python with performance-critical components in C++ (via pybind11 bindings) and is configured to run in a high‐GPU cluster environment. Configuration is managed via protobuf files to define tasks such as self-play, benchmark agent comparisons, and RL training. The project is now archived and read-only, reflecting that it is no longer actively developed but remains publicly available for research use. -
11
TTRL
Test-Time Reinforcement Learning
TTRL is an open-source framework for test-time reinforcement learning in large language models, with a particular focus on reasoning tasks where ground-truth labels are not available during inference. The project addresses the problem of how to generate useful reward signals from unlabeled test-time data, and its central insight is that common test-time scaling practices such as majority voting can be repurposed into reward estimates for online reinforcement learning. This makes the... -
12
MiroThinker
MiroThinker is an open source deep research agent
MiroThinker is an open-source deep research AI agent designed to perform complex reasoning, information gathering, and predictive analysis tasks. The system focuses on enabling long-horizon research workflows by allowing the agent to interact repeatedly with external tools, search systems, and data sources while refining its reasoning through iterative steps. Rather than simply generating responses from a single prompt, the agent performs structured multi-step reasoning processes that... -
13
BrowserGym
A Gym environment for web task automation
BrowserGym is an open framework for web task automation research that exposes browser interaction as a Gym-style environment for training and evaluating agents. It is intended for researchers building web agents rather than for end users looking for a consumer automation product. The project provides a common environment where agents can interact with websites, execute tasks, and be evaluated against standardized benchmarks. One of its main strengths is that it bundles several important... -
14
KG-LLM-Papers
Papers integrating knowledge graphs (KGs) and large language models
...The repository functions as a continuously updated index of scholarly work that investigates how structured knowledge representations can enhance the reasoning, factual accuracy, and interpretability of language models. It includes surveys, benchmark studies, and cutting-edge research that examine topics such as knowledge graph-guided prompting, retrieval-augmented generation, reasoning over structured data, and hybrid architectures combining symbolic and neural systems. By gathering these papers into a single organized repository, the project helps researchers quickly discover relevant literature and track the evolution of the field. -
15
DFlash
Block Diffusion for Ultra-Fast Speculative Decoding
...This approach has been shown to deliver lossless acceleration on models like Qwen3-8B by combining block diffusion techniques with efficient batching, making it ideal for applications where latency and cost matter. The project includes support for multiple draft models, example integration code, and scripts to benchmark performance, and it is structured to work with popular model serving stacks like SGLang and the Hugging Face Transformers ecosystem. -
16
Anthropic's Original Performance
Anthropic's original performance take-home, now open for you to try
Anthropic's Original Performance repository contains the publicly released version of a performance challenge originally used by Anthropic as part of their technical interview process, offering developers the opportunity to optimize and benchmark low-level code against simulated models. The project sets up a baseline performance problem where participants work to reduce simulated “clock cycles” required to run a given workload, effectively challenging them to engineer faster code under constraints. This take-home includes starter code, tests, and tools to debug performance, aiming to measure how effectively one can apply algorithmic improvements and optimizations. ... -
17
RecBole
A unified, comprehensive and efficient recommendation library
A unified, comprehensive and efficient recommendation library. We design general and extensible data structures to unify the formatting and usage of various recommendation datasets. We implement more than 100 commonly used recommendation algorithms and provide formatted copies of 28 recommendation datasets. We support a series of widely adopted evaluation protocols or settings for testing and comparing recommendation algorithms. RecBole is developed based on Python and PyTorch for... -
18
kg-gen
Knowledge Graph Generation from Any Text
kg-gen is an open-source framework developed by the STAIR Lab that automatically generates knowledge graphs from unstructured text using large language models. The system is designed to transform plain text sources such as documents, articles, or conversation transcripts into structured graphs composed of entities and relationships. Instead of relying on traditional rule-based extraction techniques, KG-Gen uses language models to identify entities and their relationships, producing... -
19
ESPnet
End-to-end speech processing toolkit
ESPnet is a comprehensive end-to-end speech processing toolkit covering a wide spectrum of tasks, including automatic speech recognition (ASR), text-to-speech (TTS), speech translation (ST), speech enhancement, speaker diarization, and spoken language understanding. It uses PyTorch as its deep learning engine and adopts a Kaldi-style data processing pipeline for features, data formats, and experimental recipes. This combination allows researchers to leverage modern neural architectures while... -
20
DeepGEMM
Clean and efficient FP8 GEMM kernels with fine-grained scaling
DeepGEMM is a specialized CUDA library for efficient, high-performance general matrix multiplication (GEMM) operations, with particular focus on low-precision formats such as FP8 (and experimental support for BF16). The library is designed to work cleanly and simply, avoiding overly templated or heavily abstracted code, while still delivering performance that rivals expert-tuned libraries. It supports both standard and “grouped” GEMMs, which is useful for architectures like Mixture of... -
21
OpenFace Face Recognition
Face recognition with deep neural networks
...Accuracies from research papers have just begun to surpass human accuracies on some benchmarks. The accuracies of open source face recognition systems lag behind the state-of-the-art. See our accuracy comparisons on the famous LFW benchmark. -
22
TimeMixer
Decomposable Multiscale Mixing for Time Series Forecasting
TimeMixer is a deep learning framework designed for advanced time series forecasting and analysis using a multiscale neural architecture. The model focuses on decomposing time series data into multiple temporal scales in order to capture both short-term seasonal patterns and long-term trends. Instead of relying on traditional recurrent or transformer-based architectures, TimeMixer is implemented as a fully multilayer perceptron–based model that performs temporal mixing across different... -
23
MiroFlow
Agent framework that enables tool-use agent tasks
MiroFlow is a high-performance open-source framework designed for building intelligent AI agents capable of solving complex reasoning and research tasks. The system introduces a hierarchical architecture that organizes components into control, agent, and foundation layers, allowing developers to manage agent orchestration and tool interactions in a structured manner. One of the core innovations of MiroFlow is its use of agent graphs, which enable flexible orchestration of multiple sub-agents... -
24
AIDE ML
AI-Driven Exploration in the Space of Code
AIDE ML is an open-source research framework designed to explore automated machine learning development through agent-based search and code optimization. The project implements the AIDE algorithm, which uses a tree-search strategy guided by large language models to iteratively generate, evaluate, and refine code. Instead of relying on manual experimentation, the agent autonomously drafts machine learning pipelines, debugs errors, and benchmarks performance against user-defined evaluation... -
25
Text-to-LoRA (T2L)
Hypernetworks that adapt LLMs for specific benchmark tasks
Text-to-LoRA is a research project that introduces a method for dynamically adapting large language models using hypernetworks that generate LoRA parameters directly from textual descriptions. Instead of training a new LoRA adapter for every task or dataset, the system can produce task-specific adaptations based solely on a text description of the desired capability. This approach enables models to rapidly internalize new contextual knowledge without performing traditional fine-tuning steps....
