Search Results for "benchmark" - Page 5
Sort By:
Showing 177 open source projects for "benchmark"
View related business solutions-
$300 Free Credits for Your Google Cloud ProjectsLaunch your next project with $300 in free Google Cloud credits—no strings attached. Test, build, and deploy without risk. Use your credits across the entire Google Cloud platform to find what works best for your needs. After your credits are used, continue with always-free tier services. Only pay when you're ready to scale. Sign up in minutes and start exploring.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
MLPerf
Reference implementations of MLPerf™ training benchmarks
...Together, we create the reference implementations, rules, policies, and procedures to benchmark a wide variety of AI workloads. -
2
InternVL
A Pioneering Open-Source Alternative to GPT-4o
InternVL is a large-scale multimodal foundation model designed to integrate computer vision and language understanding within a unified architecture. The project focuses on scaling vision models and aligning them with large language models so that they can perform tasks involving both visual and textual information. InternVL is trained on massive collections of image-text data, enabling it to learn representations that capture both visual patterns and semantic meaning. The model supports a... -
3
DeepSeek VL
Towards Real-World Vision-Language Understanding
DeepSeek-VL is DeepSeek’s initial vision-language model that anchors their multimodal stack. It enables understanding and generation across visual and textual modalities—meaning it can process an image + a prompt, answer questions about images, caption, classify, or reason about visuals in context. The model is likely used internally as the visual encoder backbone for agent use cases, to ground perception in downstream tasks (e.g. answering questions about a screenshot). The repository... -
4
Evals
Evals is a framework for evaluating LLMs and LLM systems
The openai/evals repository is a framework and registry for evaluating large language models and systems built with LLMs. It’s designed to let you define “evals” (evaluation tasks) in a structured way and run them against different models or agents, with the ability to score, compare, and analyze results. The framework supports templated YAML eval definitions, solver-based evaluations, custom metrics, and composition of multi-step evaluations. It includes utilities and APIs to plug in... -
Stop Cyber Threats with VM-Series Next-Gen Firewall on AzureGain integrated visibility across all traffic in a single pass. Deploy Palo Alto Networks VM-Series to determine application identity and content while automating security policy updates via rich APIs.
-
5
AI File Sorter
Local AI file organization with categorization and rename suggestions
AI File Sorter is a cross-platform desktop application that uses AI (local LLMs run on your computer) to organize files and suggest meaningful file names based on real content, not just filenames or extensions. The app can analyze images locally and propose descriptive rename suggestions (for example, IMG_2048.jpg → clouds_over_lake.jpg). It can also analyze document text to improve categorization and renaming. Supported formats include PDF, DOCX, XLSX, PPTX, ODT, ODS, ODP, and common... -
6
OWL
Optimized Workforce Learning for General Multi-Agent Assistance
Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation. OWL (Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation) is an advanced framework designed to enhance multi-agent collaboration, improving task automation across various domains. By utilizing dynamic agent interactions, OWL aims to streamline and optimize complex workflows, making AI collaboration more natural, efficient, and adaptable. It is built on... -
7
GLM-4-32B-0414
Open Multilingual Multimodal Chat LMs
GLM-4-32B-0414 is a powerful open-source large language model featuring 32 billion parameters, designed to deliver performance comparable to leading models like OpenAI’s GPT series. It supports multilingual and multimodal chat capabilities with an extensive 32K token context length, making it ideal for dialogue, reasoning, and complex task completion. The model is pre-trained on 15 trillion tokens of high-quality data, including substantial synthetic reasoning datasets, and further enhanced... -
8
DeepSeek LLM
DeepSeek LLM: Let there be answers
The DeepSeek-LLM repository hosts the code, model files, evaluations, and documentation for DeepSeek’s LLM series (notably the 67B Chat variant). Its tagline is “Let there be answers.” The repo includes an “evaluation” folder (with results like math benchmark scores) and code artifacts (e.g. pre-commit config) that support model development and deployment. According to the evaluation files, DeepSeek LLM 67B Chat achieves strong performance on math benchmarks under both chain-of-thought (CoT) and tool-assisted reasoning modes. The model is trained from scratch, reportedly on a vast multilingual + code + reasoning dataset, and competes with other open or open-weight models. ... -
9
Foolbox
Python toolbox to create adversarial examples
Foolbox: Fast adversarial attacks to benchmark the robustness of machine learning models in PyTorch, TensorFlow, and JAX. Foolbox 3 is built on top of EagerPy and runs natively in PyTorch, TensorFlow, and JAX. Foolbox provides a large collection of state-of-the-art gradient-based and decision-based adversarial attacks. Catch bugs before running your code thanks to extensive type annotations in Foolbox. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
10
Safety-Prompts
Chinese safety prompts for evaluating and improving the safety of LLMs
...The prompts are structured to test whether models generate outputs that align with human values and safety guidelines when faced with potentially harmful or sensitive requests. Researchers and developers use the dataset to benchmark how well models avoid unsafe responses and follow alignment constraints. The repository also serves as a training resource for improving model alignment by providing examples of prompts that require safe reasoning and appropriate refusal behavior. In addition to evaluation prompts, the project references related tools and benchmarks for assessing model safety across different contexts. -
11
BIG-bench
Beyond the Imitation Game collaborative benchmark for measuring
BIG-bench (Beyond the Imitation Game Benchmark) is a large, collaborative benchmark suite designed to probe the capabilities and limitations of large language models across hundreds of diverse tasks. Rather than focusing on a single metric or domain, it aggregates many hand-authored tasks that test reasoning, commonsense, math, linguistics, ethics, and creativity. Tasks are intentionally heterogeneous: some are multiple-choice with exact scoring, others are free-form generation judged by model-based or human evaluation. ... -
12
SSD in PyTorch 1.0
High quality, fast, modular reference implementation of SSD in PyTorch
This repository implements SSD (Single Shot MultiBox Detector). The implementation is heavily influenced by the projects ssd.pytorch, pytorch-ssd and maskrcnn-benchmark. This repository aims to be the code base for research based on SSD. Multi-GPU training and inference: We use DistributedDataParallel, you can train or test with arbitrary GPU(s), the training schema will change accordingly. Add your own modules without pain. We abstract backbone, Detector, BoxHead, BoxPredictor, etc. You can replace every component with your own code without changing the code base. ... -
13
GPT-2 Output Dataset
Dataset of GPT-2 outputs for research in detection, biases, and more
...The repository provides scripts and metadata for working with the dataset, with the goal of supporting research in areas like detection, evaluation of text coherence, and analysis of generative models. While no active development is expected, the dataset remains a useful benchmark for tasks involving text classification, style analysis, and generative model evaluation. -
14
MMDeploy
OpenMMLab Model Deployment Framework
MMDeploy is an open-source deep learning model deployment toolset. It is a part of the OpenMMLab project. Models can be exported and run in several backends, and more will be compatible. All kinds of modules in the SDK can be extended, such as Transform for image processing, Net for Neural Network inference, Module for postprocessing and so on. Install and build your target backend. ONNX Runtime is a cross-platform inference and training accelerator compatible with many popular ML/DNN... -
15
MMAction2
OpenMMLab's Next Generation Video Understanding Toolbox and Benchmark
OpenMMLab's next generation video understanding toolbox and benchmark. MMAction2 is an open-source toolbox for video understanding based on PyTorch. It is a part of the OpenMMLab project. Modular design: We decompose a video understanding framework into different components. One can easily construct a customized video understanding framework by combining different modules. Support four major video understanding tasks: MMAction2 implements various algorithms for multiple video understanding tasks, including action recognition, action localization, Spatio-temporal action detection, and skeleton-based action detection. ... -
16
xTuring
Easily build, customize and control your own LLMs
xTuring is an open-source AI personalization software. xTuring makes it easy to build and control LLMs by providing a simple interface to personalize LLMs to your own data and application. xTuring provides fast, efficient and simple fine-tuning of LLMs, such as LLaMA, GPT-J, Galactica, and more. By providing an easy-to-use interface for fine-tuning LLMs to your own data and application, xTuring makes it simple to build, customize and control LLMs. The entire process can be done inside your... -
17
MMClassification
OpenMMLab Image Classification Toolbox and Benchmark
...MMClassification is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedback. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to re-implement existing methods and develop their own new classifiers. MMClassification mainly uses python files as configs. The design of our configuration file system integrates modularity and inheritance, facilitating users to conduct various experiments. -
18
GLM-130B
GLM-130B: An Open Bilingual Pre-Trained Model (ICLR 2023)
...Built on the SwissArmyTransformer (SAT) framework and compatible with DeepSpeed and FasterTransformer, it supports high-speed inference (up to 2.5× faster) and reproducible evaluation across 30+ benchmark tasks. -
19
ReAct Prompting
Synergizing Reasoning and Acting in Language Models
ReAct is an open-source research project that demonstrates a prompting and reasoning framework designed to improve the problem-solving capabilities of large language models. The project implements the methodology described in the research paper “ReAct: Synergizing Reasoning and Acting in Language Models,” which combines reasoning traces with action-based interactions. Instead of generating answers in a single step, models using the ReAct approach produce intermediate reasoning steps and... -
20
OGB
Benchmark datasets, data loaders, and evaluators for graph machine
The Open Graph Benchmark (OGB) is a collection of realistic, large-scale, and diverse benchmark datasets for machine learning on graphs. OGB datasets are automatically downloaded, processed, and split using the OGB Data Loader. The model performance can be evaluated using the OGB Evaluator in a unified manner. OGB is a community-driven initiative in active development. -
21
Chameleon LLM
Codes for "Chameleon: Plug-and-Play Compositional Reasoning
...By integrating various tools such as vision models, web search engines, Python functions, and rule-based modules, Chameleon delivers more accurate, up-to-date, and precise responses, making it a game-changer in the natural language processing landscape. With GPT-4 at its core, Chameleon has showcased exceptional improvements in accuracy on benchmark tasks, outperforming competitors and setting new industry standards. -
22
DIG
A library for graph deep learning research
...It includes unified implementations of data interfaces, common algorithms, and evaluation metrics for several advanced tasks. Our goal is to enable researchers to easily implement and benchmark algorithms. -
23
VnCoreNLP
A Vietnamese natural language processing toolkit
VnCoreNLP is a Java-based natural language processing toolkit tailored for Vietnamese. It offers a fast and accurate pipeline for essential NLP tasks, facilitating research and application development in Vietnamese language processing. -
24
Merlion
A Machine Learning Framework for Time Series Intelligence
...It supports various time series learning tasks, including forecasting, anomaly detection, and change point detection for both univariate and multivariate time series. This library aims to provide engineers and researchers a one-stop solution to rapidly develop models for their specific time series needs, and benchmark them across multiple time series datasets. -
25
BCI
BCI: Breast Cancer Immunohistochemical Image Generation
...The routine evaluation of HER2 is conducted with immunohistochemical techniques (IHC), which is very expensive. Therefore, for the first time, we propose a breast cancer immunohistochemical (BCI) benchmark attempting to synthesize IHC data directly with the paired hematoxylin and eosin (HE) stained images.
