Search Results for "benchmarks" - Page 7
Sort By:
Showing 440 open source projects for "benchmarks"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
1
RLHF-Reward-Modeling
Recipes to train reward model for RLHF
RLHF-Reward-Modeling is an open-source research framework focused on training reward models used in reinforcement learning from human feedback for large language models. In RLHF pipelines, reward models are responsible for evaluating generated responses and assigning scores that guide the model toward outputs that better match human preferences. The repository provides training recipes and implementations for building reward and preference models using modern machine learning frameworks. It... -
2
The LLM Evaluation guidebook
Sharing both practical insights and theoretical knowledge about LLM
...The guidebook teaches developers how to design evaluation pipelines, select appropriate metrics, and interpret model performance results. It discusses multiple evaluation strategies, ranging from automated benchmarks to human evaluation and LLM-based evaluation techniques. The material also highlights the strengths and weaknesses of different evaluation methods, helping practitioners understand when and how to apply them. By organizing evaluation knowledge into structured sections, the project helps engineers and researchers build more reliable and trustworthy AI systems. -
3
Magicoder
Empowering Code Generation with OSS-Instruct
...By grounding training data in real open-source examples, Magicoder aims to reduce bias and improve the reliability of code generation results compared to models trained solely on synthetic instructions. The project includes model implementations, training resources, and evaluation benchmarks that demonstrate how the approach improves instruction-following and code synthesis capabilities. Magicoder models are intended for tasks such as programming assistance, code explanation, automated debugging, and software documentation generation. -
4
AgentBench
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
AgentBench is an open-source benchmark designed to evaluate the capabilities of large language models when used as autonomous agents. Unlike traditional language model benchmarks that focus on static text tasks, AgentBench measures how models perform in interactive environments that require planning, reasoning, and decision-making. The benchmark includes multiple environments that simulate realistic scenarios such as web interaction, database querying, and problem solving tasks. These environments require agents to interpret instructions, take actions, and adapt their strategies based on feedback from the environment. ... -
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
5
The Alignment Handbook
Robust recipes to align language models with human and AI preferences
...It provides detailed training recipes that explain how to perform tasks such as supervised fine-tuning, preference modeling, and reinforcement learning from human feedback. The handbook also includes reproducible workflows for training instruction-following models and evaluating alignment quality across different datasets and benchmarks. One of its goals is to bridge the gap between academic research on alignment methods and practical engineering implementation. -
6
FlagEmbedding
Retrieval and Retrieval-augmented LLMs
...FlagEmbedding includes a family of models known as BGE (BAAI General Embedding), which are designed to achieve strong performance across multilingual and cross-lingual retrieval benchmarks. The toolkit provides infrastructure for inference, fine-tuning, evaluation, and dataset preparation, enabling developers to train custom embedding models for specific domains or applications. It also includes reranker models that refine search results by re-evaluating candidate documents using cross-encoder architectures, improving retrieval accuracy in complex queries. -
7
Agent Reinforcement Trainer
Train multi-step agents for real-world tasks using GRPO
...ART also supports scalable training patterns, observability tools, and integration with hosted platforms like Weights & Biases, and it provides notebooks that demonstrate training on standard benchmarks and tasks. -
8
Step3-VL-10B
Multimodal model achieving SOTA performance
...Despite having only about 10 billion parameters, it delivers performance that rivals or even surpasses much larger models (10×–20× larger) on a wide range of multimodal benchmarks covering reasoning, perception, and complex tasks, positioning it as one of the most powerful models in its class. It achieves this efficiency and strong performance through unified pre-training on a massive 1.2 trillion-token multimodal corpus that jointly optimizes a language-aligned perception encoder with a powerful decoder, creating deep synergy between image processing and text understanding. -
9
Chinese-XLNet
Chinese XLNet pre-trained model
...Chinese-XLNet offers an alternative to models like BERT by emphasizing autoregressive and permutation-based learning, which can lead to performance improvements on certain benchmarks and tasks. -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
Supertonic
Lightning-fast, on-device TTS, running natively via ONNX
...It focuses on running entirely locally, eliminating the need for cloud APIs and providing low latency and strong privacy guarantees, even on constrained devices like Raspberry Pi boards and e-readers. The core model is highly compact at around 66 million parameters, yet benchmarks show it can generate speech up to 167× faster than real time on modern consumer hardware and significantly outpace popular cloud TTS APIs in throughput and real-time factor. Supertonic is designed to handle real-world text gracefully, including numbers, dates, currency symbols, abbreviations, and technical units, without requiring heavy pre-processing or custom text normalization. ... -
11
Granite Code Models
A Family of Open Foundation Models for Code Intelligence
Granite Code Models are IBM’s open-source, decoder-only models tailored for code tasks such as fixing bugs, explaining and documenting code, and modernizing codebases. Trained on code from 116 programming languages, the family targets strong performance across diverse benchmarks while remaining accessible to the community. The repository introduces the model lineup, intended uses, and evaluation highlights, and it complements IBM’s broader Granite initiative spanning multiple modalities. IBM’s research blog details the motivation for opening these models and points developers to downloads, papers, and hosting options. ... -
12
MetaCLIP
ICLR2024 Spotlight: curation/training code, metadata, distribution
MetaCLIP is a research codebase that extends the CLIP framework into a meta-learning / continual learning regime, aiming to adapt CLIP-style models to new tasks or domains efficiently. The goal is to preserve CLIP’s strong zero-shot transfer capability while enabling fast adaptation to domain shifts or novel class sets with minimal data and without catastrophic forgetting. The repository provides training logic, adaptation strategies (e.g. prompt tuning, adapter modules), and evaluation... -
13
DLRM
An implementation of a deep learning recommendation model (DLRM)
...The implementation is optimized for performance at scale, supporting multi-GPU and multi-node execution, quantization, embedding partitioning, and pipelined I/O to feed huge embeddings efficiently. It includes data loaders for standard benchmarks (like Criteo), training scripts, evaluation tools, and capabilities like mixed precision, gradient compression, and memory fusion to maximize throughput. -
14
ThreatMapper
Open source cloud native security observability platform
...ThreatMapper carries on the good 'shift left' security practices that you already employ in your development pipelines. It continues to monitor running applications against emerging software vulnerabilities and monitors the host and cloud configuration against industry-expert benchmarks. -
15
LINKERD
Ultralight, security-first service mesh for Kubernetes
Enterprise power without enterprise complexity. Linkerd adds security, observability, and reliability to any Kubernetes cluster. 100% open source, CNCF graduated, and written in Rust. Instantly add latency-aware load balancing, request retries, timeouts, and blue-green deploys to keep your applications resilient. Incredibly small and blazing fast Linkerd2-proxy micro-proxy written in Rust for security and performance. Self-contained control plane, incrementally deployable data plane, and... -
16
Jawn
Jawn is for parsing jay-sawn (JSON)
...Jawn was designed to parse JSON into an AST as quickly as possible. Currently, Jawn is competitive with the fastest Java JSON libraries (GSON and Jackson) and in the author's benchmarks, it often wins. It seems to be faster than any other Scala parser that exists (as of July 2014). -
17
node-rate-limiter-flexible
Count and limit requests by key with atomic increments
...Combine limiters, block key for some duration, delay actions, manage failover with insurance options, configure smart key blocking in memory and many others. Average request takes 0.7ms in Cluster and 2.5ms in a Distributed application. See benchmarks. It provides a unified API for all limiters. Whenever your application grows, it is ready. Prepare your limiters in minutes. No matter which node package you prefer: redis or ioredis, sequelize/typeorm or knex, memcached, native driver or mongoose. It works with all of them. -
18
ASP.NET Minimal APIs Made Easy
A light-weight REST API development framework for ASP.Net 6
It nudges you towards the REPR Design Pattern (Request-Endpoint-Response) for convenient & maintainable endpoint creation with virtually no boilerplate. Performance is on par with Minimal APIs. It's faster, uses less memory and does around 45k more requests per second than a MVC Controller in our benchmarks. Convenient business logic validation & error responses. Easy access to environment & configuration settings. Supports policy/permission/role/claim based security. In-process pub/sub event notifications (with auto discovery). Declarative security policy building (within each endpoint). Easy Server-Sent-Events for real-time data streaming. ... -
19
HunyuanOCR
OCR expert VLM powered by Hunyuan's native multimodal architecture
HunyuanOCR is an open-source, end-to-end OCR (optical character recognition) Vision-Language Model (VLM) developed by Tencent‑Hunyuan. It’s designed to unify the entire OCR pipeline, detection, recognition, layout parsing, information extraction, translation, and even subtitle or structured output generation, into a single model inference instead of a cascade of separate tools. Despite being fairly lightweight (about 1 billion parameters), it delivers state-of-the-art performance across a... -
20
Poetiq
Reproduction of Poetiq's record-breaking submission to the ARC-AGI-1
poetiq-arc-agi-solver is the open-source codebase from Poetiq that replicates their record-breaking submission to the challenging benchmark suite ARC-AGI (both ARC-AGI-1 and ARC-AGI-2). The project demonstrates a system that orchestrates large language models (LLMs) — like those from major providers — with carefully engineered prompting, reasoning workflows, and dynamic strategies, to tackle the abstract, logic-heavy problems in ARC-AGI. Instead of relying on a single prompt or fixed... -
21
ESPnet
End-to-end speech processing toolkit
...This combination allows researchers to leverage modern neural architectures while still benefiting from the robust data preparation practices developed in the speech community. ESPnet provides many ready-to-run recipes for popular academic benchmarks, making it straightforward to reproduce published results or serve as baselines for new research. The toolkit also hosts numerous pretrained models and example configs, ranging from Transformer and Conformer architectures to various attention-based encoder-decoder models. -
22
PokeeResearch-7B
Pokee Deep Research Model Open Source Repo
...It is built to operate end-to-end: planning a research strategy, gathering sources, reasoning over conflicting claims, and writing a grounded response. The repository includes evaluation results on multi-step QA and research benchmarks, illustrating how web-time context boosts accuracy. Because the system is modular, you can swap the search component, reader, or policy to fit private deployments or different data domains. It’s aimed at developers who want a transparent, hackable research agent they can run locally or wire into existing workflows. -
23
vJEPA-2
PyTorch code and models for VJEPA2 self-supervised learning from video
VJEPA2 is a next-generation self-supervised learning framework for video that extends the “predict in representation space” idea from i-JEPA to the temporal domain. Instead of reconstructing pixels, it predicts the missing high-level embeddings of masked space-time regions using a context encoder and a slowly updated target encoder. This objective encourages the model to learn semantics, motion, and long-range structure without the shortcuts that pixel-level losses can invite. The... -
24
CogVLM2
GPT4V-level open-source multi-modal model based on Llama3-8B
CogVLM2 is the second generation of the CogVLM vision-language model series, developed by ZhipuAI and released in 2024. Built on Meta-Llama-3-8B-Instruct, CogVLM2 significantly improves over its predecessor by providing stronger performance across multimodal benchmarks such as TextVQA, DocVQA, and ChartQA, while introducing extended context length support of up to 8K tokens and high-resolution image input up to 1344×1344. The series includes models for both image understanding and video understanding, with CogVLM2-Video supporting up to 1-minute videos by analyzing keyframes. It supports bilingual interaction (Chinese and English) and has open-source versions optimized for dialogue and video comprehension. ... -
25
XiangShan
Open-source high-performance RISC-V processor
XiangShan is an open-source, high-performance RISC-V processor project that implements out-of-order superscalar cores using Chisel for hardware construction. The design targets modern performance goals—deep pipelines, speculative execution, multi-issue decode/execute, and sophisticated branch prediction—while remaining synthesizable for ASIC flows and portable to FPGAs for research. A modular microarchitecture separates frontend, backend, and memory subsystems with coherent caches and...
