Search Results for "benchmarks" - Page 4
Sort By:
Showing 440 open source projects for "benchmarks"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Pi Agent
AI agent toolkit: coding agent CLI, unified LLM API, TUI & web UI
...The framework is built around extensibility, allowing developers to create custom tools, workflows, and autonomous coding experiences. Pi also encourages the sharing of real-world AI coding sessions to improve agent performance through practical usage data instead of synthetic benchmarks. With its modular architecture, active open-source community, and support for advanced agent capabilities, Pi provides a comprehensive foundation for building next-generation AI development tools and autonomous c -
2
fastjson
A fast JSON parser/generator for Java
Fastjson is a Java library that can be used to quickly convert Java Objects into their JSON representation or convert JSON strings to their equivalent Java object. Based on benchmarks it provides the best performance in both server side and Android client, and can work with arbitrary Java objects, even those pre-existing objects of which you do not have the source code. Fastjson provides a simple conversion method to turn Java objects to JSON and vice versa, with extensive support of Java Generics and arbitrarily complex objects. -
3
DeepSeek-V3.2-Exp
An experimental version of DeepSeek model
...According to the authors, they aligned the training setup of V3.2-Exp with V3.1-Terminus so that benchmark results remain largely comparable, even though the internal attention mechanism changes. In public evaluations across a variety of reasoning, code, and question-answering benchmarks (e.g. MMLU, LiveCodeBench, AIME, Codeforces, etc.), V3.2-Exp shows performance very close to or in some cases matching that of V3.1-Terminus. The repository includes tools and kernels to support the new sparse architecture—for instance, CUDA kernels, logit indexers, and open-source modules like FlashMLA and DeepGEMM are invoked for performance. -
4
pgvecto.rs
Vector database plugin for Postgres, written in Rust
pgvecto.rs is a Postgres extension that provides vector similarity search functions. It is written in Rust and based on pgrx. It is currently under heavy development, please take care when using it in production. pgvecto.rs is a Postgres extension, which means that you can use it directly within your existing database. This makes it easy to integrate into your existing workflows and applications. pgvecto.rs supports filtering. You can set conditions when searching or retrieving points. This... -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
5
Codeflash
Optimize your code automatically with AI
Codeflash is a general-purpose optimizer for Python that uses advanced large language models (LLMs) to automatically generate, test, and benchmark multiple optimization ideas, then creates merge-ready pull requests with the best improvements for your code. Optimize an entire existing codebase by running codeflash --all. Automate optimizing all future code you will write by installing Codeflash as a GitHub action. Optimize a Python workflow python myscript.py end-to-end by running codeflash... -
6
FastHTTP
Fast HTTP package for Go
FastHTTP is the fastest and reliable HTTP implementation for Go. FastHTTP is tuned for high performance, with zero memory allocations in hot paths. In benchmarks, both FastHTTP server and client have shown to be up to 10 times faster than those of net/http. However, FastHTTP does not provide an API identical to net/http, but it is better to write FastHTTP request handlers by hand so as to make use of all its advantages. One of its key advantages is that unlike net/http it does not limit many optimization opportunities, such as reusing existing objects per request and avoiding unnecessary memory allocations. -
7
SAM 2
The repository provides code for running inference with SAM 2
...SAM2 comes with pretrained weights and easy-to-use APIs, enabling developers and researchers to integrate promptable segmentation into annotation tools, vision pipelines, or downstream tasks. The project also includes scripts and notebooks to compare SAM2 against SAM on edge cases, benchmarks showing improvements, and evaluation suites to measure mask quality metrics like IoU and boundary error. -
8
LLMSurvey
A Survey of Large Language Models
...The repository organizes hundreds of research papers into thematic sections that reflect the main areas of LLM research, including model architectures, training strategies, evaluation benchmarks, alignment techniques, and downstream applications. By structuring the literature in a navigable format, LLMSurvey allows researchers and practitioners to quickly explore important publications in the field without manually searching through multiple databases. -
9
Simple Evals
Lightweight framework for evaluating large language model performance
simple-evals is a lightweight evaluation framework developed by OpenAI for quickly testing models against small, focused benchmarks. It is designed to help researchers and developers run targeted evaluations without the complexity of large-scale pipelines. By emphasizing simplicity, the framework makes it easy to define new tasks, run evaluations, and interpret results in a reproducible way. It is particularly useful for sanity checks, exploratory research, and comparing performance across different models or configurations. ... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
chi
Lightweight, idiomatic & composable router to build Go HTTP services
...Its design chiefly considers project structure, maintainability, standard http handlers (stdlib-only), developer productivity, and deconstructing a large system into many small parts. It is very lightweight and fast (according to benchmarks), yet is still able to include several useful subpackages (middleware, render and docgen). -
11
Kimi K2.5
Moonshot's most powerful AI model
...Designed for agentic workflows, it features an Agent Swarm mechanism that decomposes complex problems into coordinated sub-agents executing in parallel. With a 256K context length and MoonViT vision encoder, the model excels across reasoning, coding, long-context comprehension, image, and video benchmarks. Kimi K2.5 is available via Moonshot’s API (OpenAI/Anthropic-compatible) and supports deployment through vLLM, SGLang, and KTransformers. -
12
Zvec
A lightweight, lightning-fast, in-process vector database
...Zvec excels at approximate nearest neighbor search and retrieval tasks that power features like semantic search, recommendation systems, and retrieval-augmented generation (RAG) setups. Its performance benchmarks show it achieving high queries-per-second and fast index build times compared to similar tools. Because it runs in-process, developers can embed it in native apps, microservices, or edge computing scenarios where traditional server-based vector databases might be overkill. -
13
Kimi k1.5
Scaling Reinforcement Learning with LLMs
Kimi-k1.5 is an advanced open-source multimodal large-language model project that explores scaling reinforcement learning with long-context chains of thought, achieving performance that rivals or surpasses state-of-the-art models on benchmarks like LiveCodeBench, AIME, and MATH-500. The project emphasizes a simplistic yet powerful framework where the context window scales up to 128k tokens, enabling reasoning that resembles planning, reflection, and correction over a much longer sequence of data than typical models. By using techniques like partial rollouts to improve training efficiency and applying sophisticated policy optimization methods, the developers demonstrate that strong ability can emerge without relying on complex solutions like Monte Carlo tree search or value functions. ... -
14
CuPy
A NumPy-compatible array library accelerated by CUDA
...It consists of cupy.ndarray, a core multi-dimensional array class and many functions on it. CuPy offers GPU accelerated computing with Python, using CUDA-related libraries to fully utilize the GPU architecture. According to benchmarks, it can even speed up some operations by more than 100X. CuPy is highly compatible with NumPy, serving as a drop-in replacement in most cases. CuPy is very easy to install through pip or through precompiled binary packages called wheels for recommended environments. It also makes writing a custom CUDA kernel very easy, requiring only a small code snippet of C++. -
15
OpenOCR
An Open-Source Toolkit for General-OCR Research and Applications
...OpenOCR includes training, evaluation, fine-tuning, and deployment tools, allowing users to customize models for specific OCR tasks. Its comprehensive ecosystem bridges academic research and industrial applications through reproducible benchmarks and commercial-grade OCR solutions. -
16
Fara-7B
An Efficient Agentic Model for Computer Use
Fara-7B is a Microsoft initiative aimed at bringing rigor, transparency, and structured evaluation to AI systems through automated and customizable assessment frameworks. It provides stakeholders with a way to benchmark and evaluate models across dimensions such as fairness, robustness, security, privacy, and ethical considerations. Rather than relying on ad-hoc or manual review processes, FARA enables organizations to profile AI behavior using standardized tests, metrics, and reporting... -
17
OWL
Optimized Workforce Learning for General Multi-Agent Assistance
...Unlike single-agent systems, it treats task completion as a collaborative workforce where agents take on specialized roles (planning, execution, analysis) and coordinate via a modular multi-agent architecture that supports flexible teamwork across domains. OWL delivers state-of-the-art performance on benchmarks like GAIA and emphasizes real-time decision-making, web automation, rich search integration, document parsing, and multi-tool workflows, making it suitable for tasks ranging from information retrieval to interactive automation. -
18
Qwen3-Coder
Qwen3-Coder is the code version of Qwen3
...Its flagship version, Qwen3-Coder-480B-A35B-Instruct, features a massive 480 billion-parameter Mixture-of-Experts architecture with 35 billion active parameters, delivering top-tier performance on coding and agentic tasks. This model sets new state-of-the-art benchmarks among open models for agentic coding, browser-use, and tool-use, matching performance comparable to leading models like Claude Sonnet. Qwen3-Coder supports an exceptionally long context window of 256,000 tokens, extendable to 1 million tokens using Yarn, enabling repository-scale code understanding and generation. It is capable of handling 358 programming languages, from common to niche, making it versatile for a wide range of development environments. ... -
19
SIA
AI framework to autonomously improve the performance of any AI system
...The framework can refine both the harness around the task and the agent implementation itself. It is aimed at research and experimentation across tasks such as machine learning benchmarks, legal classification, code optimization, and scientific workflows. It includes built-in tasks, a command-line runner, and a visual dashboard for following generations as they evolve. It also lets users define custom providers, profiles, seed agents, and task directories without changing the core code. -
20
Atropos
Language Model Reinforcement Learning Environments frameworks
...Designed as a scalable ecosystem of environment microservices, Atropos allows researchers and developers to collect, evaluate, and manage trajectories (sequences of actions and outcomes) generated by LLMs across a variety of tasks—from static dataset benchmarks to dynamic interactive games and real-world scenario environments. It provides foundational tooling for asynchronous RL loops where environment services communicate with trainers and inference engines, enabling complex workflow orchestration in distributed and parallel setups. This framework facilitates experimentation with RLHF (Reinforcement Learning from Human Feedback), RLAIF, or multi-turn training approaches by abstracting environment logic, scoring, and logging into reusable components. -
21
graphql-hooks
Minimal hooks-first GraphQL client
...We believe graphql-hooks is a great choice as a hooks-first GraphQL client due to its concise API and package size. In terms of performance, this is more of a grey area as we have no official benchmarks yet. If you need a client that offers more customization such as advanced cache configuration, then apollo-hooks may work out to be a good choice for your project if bundle size is not an issue. -
22
pydantic
Data parsing and validation using Python type hints
Data validation and settings management using Python type hinting. Fast and extensible, pydantic plays nicely with your linters/IDE/brain. Define how data should be in pure, canonical Python 3.6+; validate it with pydantic. id is of type int; the annotation-only declaration tells pydantic that this field is required. Strings, bytes or floats will be coerced to ints if possible; otherwise an exception will be raised. name is inferred as a string from the provided default; because it has a... -
23
MuJoCo Playground
An open source library for GPU-accelerated robot learning
...It unifies a range of control, locomotion, and manipulation tasks into a consistent and scalable framework optimized for JAX and Warp backends. The project includes classic control benchmarks from dm_control, advanced quadruped and bipedal locomotion systems, and dexterous as well as non-prehensile manipulation setups. It also offers optional vision-based training capabilities through integration with Madrona-MJX, allowing researchers to train policies directly from image input on GPUs. MuJoCo Playground supports both the MJX JAX implementation and the Warp physics engine, enabling flexible use across research pipelines. ... -
24
ImageReward
[NeurIPS 2023] ImageReward: Learning and Evaluating Human Preferences
ImageReward is the first general-purpose human preference reward model (RM) designed for evaluating text-to-image generation, introduced alongside the NeurIPS 2023 paper ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation. Trained on 137k expert-annotated image pairs, ImageReward significantly outperforms existing scoring methods like CLIP, Aesthetic, and BLIP in capturing human visual preferences. It is provided as a Python package (image-reward) that enables... -
25
Hyperf
A coroutine framework that focuses on hyperspeed and flexibility
Hyperf is an extremely performant and flexible PHP CLI framework, powered by a state-of-the-art coroutine server and a large number of battle-tested components. Aside from decisively beating PHP-FPM frameworks in benchmarks, Hyperf is unique in its focus on flexibility and composition. Hyperf ships with an AOP-enabling (aspect-oriented programming) dependency injector to ensure components and classes are pluggable and meta-programmable. All of Hyperf's core components strictly follow PSR standards and can be used in other frameworks. Although there are many new PHP frameworks, we still haven't found a framework that matches an elegant design with ultra-high performance, nor have we found a framework that paves the way for PHP microservices. ...
