Search Results for "metrics" - Page 5
Sort By:
Showing 240 open source projects for "metrics"
View related business solutions-
Demo Series - Small Business Backup By VeeamWatch this on-demand demo series and learn how to protect your Microsoft 365 data with clear, simple, actionable steps that are easy to implement for businesses of all sizes.
-
Earn up to 16% annual interest with Nexo.Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
-
1
Atropos
Language Model Reinforcement Learning Environments frameworks
Atropos is a comprehensive open-source framework for reinforcement learning (RL) environments tailored specifically to work with large language models (LLMs). Designed as a scalable ecosystem of environment microservices, Atropos allows researchers and developers to collect, evaluate, and manage trajectories (sequences of actions and outcomes) generated by LLMs across a variety of tasks—from static dataset benchmarks to dynamic interactive games and real-world scenario environments. It... -
2
Nevergrad
A Python toolbox for performing gradient-free optimization
...Nevergrad supports parallelization, budget scheduling, and multiple cost/resource constraints, allowing it to scale to nontrivial optimization problems. It includes visualization tools and diagnostic metrics to compare strategy performance, track parameter evolution, and detect stagnation. -
3
HumanEval
Code for the paper "Evaluating Large Language Models Trained on Code"
human-eval is a benchmark dataset and evaluation framework created by OpenAI for measuring the ability of language models to generate correct code. It consists of hand-written programming problems with unit tests, designed to assess functional correctness rather than superficial metrics like text similarity. Each task includes a natural language prompt and a function signature, requiring the model to generate an implementation that passes all provided tests. The benchmark has become a standard for evaluating code generation models, including those in the Codex and GPT families. Researchers can use the dataset to run reproducible comparisons across models and track improvements in functional code synthesis. ... -
4
ML for Beginners
12 weeks, 26 lessons, 52 quizzes, classic Machine Learning for all
...Organized as a multi-week course, it mixes short lectures with labs in notebooks so learners practice regression, classification, clustering, and recommendation techniques on real datasets. Each lesson aims to connect the algorithm to a relatable scenario, reinforcing intuition before diving into parameters, metrics, and trade-offs. The repository includes quizzes, solutions, and instructor materials to make the content usable in classrooms or self-study. It emphasizes ethical considerations and model evaluation—accuracy is not the only metric—so students learn to validate and communicate results responsibly. By the end, participants can build end-to-end ML experiments, interpret outputs, and iterate with confidence rather than just copying code. -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
PyBroker
Algorithmic Trading in Python with Machine Learning
Are you looking to enhance your trading strategies with the power of Python and machine learning? Then you need to check out PyBroker! This Python framework is designed for developing algorithmic trading strategies, with a focus on strategies that use machine learning. With PyBroker, you can easily create and fine-tune trading rules, build powerful models, and gain valuable insights into your strategy’s performance. -
6
Mosec
A high-performance ML model serving framework, offers dynamic batching
Mosec is a high-performance and flexible model-serving framework for building ML model-enabled backend and microservices. It bridges the gap between any machine learning models you just trained and the efficient online service API. -
7
whylogs
The open standard for data logging
...They capture key statistical properties of data, such as the distribution (far beyond simple mean, median, and standard deviation measures), the number of missing values, and a wide range of configurable custom metrics. By capturing these summary statistics, we are able to accurately represent the data and enable all of the use cases described in the introduction. -
8
Arize Phoenix
Uncover insights, surface problems, monitor, and fine tune your LLM
Phoenix provides ML insights at lightning speed with zero-config observability for model drift, performance, and data quality. Phoenix is an Open Source ML Observability library designed for the Notebook. The toolset is designed to ingest model inference data for LLMs, CV, NLP and tabular datasets. It allows Data Scientists to quickly visualize their model data, monitor performance, track down issues & insights, and easily export to improve. Deep Learning Models (CV, LLM, and Generative)... -
9
PyTorch Forecasting
Time series forecasting with PyTorch
PyTorch Forecasting aims to ease state-of-the-art time series forecasting with neural networks for both real-world cases and research alike. The goal is to provide a high-level API with maximum flexibility for professionals and reasonable defaults for beginners. A time series dataset class that abstracts handling variable transformations, missing values, randomized subsampling, multiple history lengths, etc. A base model class that provides basic training of time series models along with... -
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
10
OpenSRE
Build your own AI SRE agents. The open source toolkit for the AI era
OpenSRE is an open-source framework designed to build AI-powered Site Reliability Engineering agents that automate incident investigation and root cause analysis across modern cloud environments. It connects to observability tools, infrastructure systems, and communication platforms to gather logs, metrics, and traces in real time. When an alert is triggered, the system autonomously analyzes correlated signals, identifies anomalies, and generates structured investigation reports with probable causes and recommended actions. Its multi-agent architecture allows parallel reasoning across systems, mimicking how experienced SRE teams debug complex issues. ... -
11
Kaggle Solutions
Collection of Kaggle Solutions and Ideas
...The repository acts as a knowledge base for competitive machine learning by collecting solution write-ups, discussion threads, code notebooks, and tutorial resources shared by top Kaggle participants. Each competition entry typically includes information about the dataset, evaluation metrics, modeling strategies, and techniques used by high-ranking competitors. The repository also highlights important machine learning concepts such as feature engineering, cross-validation strategies, ensemble modeling, and post-processing methods commonly used in winning solutions. Because the content is organized by competition categories such as computer vision, natural language processing, tabular data, and time-series forecasting, users can explore techniques relevant to specific problem types. -
12
Weibo Crawler
Python crawler for collecting and downloading Sina Weibo user data
weibo-crawler is a Python-based data collection tool designed to retrieve information from Sina Weibo user accounts. It automates the process of gathering posts, user profile details, and engagement metrics from one or more target accounts. weibo-crawler can extract comprehensive information about users, including profile attributes such as nickname, follower count, following count, and account metadata. It also captures detailed data about each post, including the content, publishing time, topics, mentions, likes, reposts, and comments. ... -
13
Hallucination Leaderboard
Leaderboard Comparing LLM Performance at Producing Hallucinations
...The results are published as a leaderboard that allows researchers and developers to compare model reliability and factual consistency. By focusing on hallucination rates rather than traditional metrics such as accuracy or fluency, the benchmark highlights an important aspect of AI system safety and trustworthiness. The leaderboard is regularly updated as new models are released and evaluation methods evolve. -
14
AgentBench
A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)
AgentBench is an open-source benchmark designed to evaluate the capabilities of large language models when used as autonomous agents. Unlike traditional language model benchmarks that focus on static text tasks, AgentBench measures how models perform in interactive environments that require planning, reasoning, and decision-making. The benchmark includes multiple environments that simulate realistic scenarios such as web interaction, database querying, and problem solving tasks. These... -
15
Tunix
A JAX-native LLM Post-Training Library
...Examples and reference configs demonstrate end-to-end runs for common model families, helping teams reproduce baselines before customizing. Tunix also leans into research ergonomics: logging, checkpointing, and metrics are built in, and the code is written to be hackable rather than monolithic. Overall it aims to shorten the path from an off-the-shelf base model to a well-aligned, task-ready model using scalable JAX primitives. -
16
MetaCLIP
ICLR2024 Spotlight: curation/training code, metadata, distribution
...The repository provides training logic, adaptation strategies (e.g. prompt tuning, adapter modules), and evaluation across base and target domains to measure how well the model retains its general knowledge while specializing as needed. It includes utilities to fine-tune vision-language embeddings, compute prompt or adapter updates, and benchmark across transfer and retention metrics. MetaCLIP is especially suited for real-world settings where a model must continuously incorporate new visual categories or domains over time. -
17
Pearl
A Production-ready Reinforcement Learning AI Agent Library
...Tutorials demonstrate end-to-end workflows on OpenAI Gym tasks and contextual-bandit setups derived from tabular datasets, emphasizing reproducibility and clear baselines. Pearl’s design favors clarity and deployability: metrics, logging, and evaluation harnesses are integrated so you can monitor learning, compare agents, and catch regressions. -
18
Argilla
The open-source data curation platform for LLMs
Argilla is a production-ready framework for building and improving datasets for NLP projects. Deploy your own Argilla Server on Spaces with a few clicks. Use embeddings to find the most similar records with the UI. This feature uses vector search combined with traditional search (keyword and filter based). Argilla is free, open-source, and 100% compatible with major NLP libraries (Hugging Face transformers, spaCy, Stanford Stanza, Flair, etc.). In fact, you can use and combine your preferred... -
19
ClearML
Streamline your ML workflow
ClearML is an open source platform that automates and simplifies developing and managing machine learning solutions for thousands of data science teams all over the world. It is designed as an end-to-end MLOps suite allowing you to focus on developing your ML code & automation, while ClearML ensures your work is reproducible and scalable. The ClearML Python Package for integrating ClearML into your existing scripts by adding just two lines of code, and optionally extending your experiments... -
20
Data Version Control
Git-based data version control for machine learning workflows
DVC (Data Version Control) is an open source tool designed to bring version control principles to machine learning and data science workflows. It enables developers and data scientists to track datasets, machine learning models, and experiment results in a way that integrates with existing Git repositories. Instead of storing large datasets directly in Git, DVC keeps lightweight metadata in the repository while storing the actual data in external storage systems. This approach allows teams... -
21
Acontext
Context data platform for building observable, self-learning AI agents
Acontext is a cloud-native context data platform designed to support the development and operation of advanced AI agents. It provides a unified system to store and manage contexts, multimodal messages, artifacts, and task workflows, enabling developers to engineer context effectively for their agent products. The platform observes agent tasks and user feedback in real time, offering robust observability into workflows and helping teams understand how agents perform over time. Acontext also... -
22
Memobase
Fast backend for long-term AI user memory via structured profiles
...Instead of relying purely on traditional embedding-based retrieval or RAG systems, Memobase uses profile and timeline structures to deliver memory that reflects user context efficiently and meaningfully. The system focuses on three principal performance metrics: high search performance, reduced large language model (LLM) costs through batch processing techniques, and low latency with minimal SQL operations. Memobase supports integration with existing LLM workflows via APIs and SDKs (including Python, Node, and Go), making it easy to adopt within diverse application stacks. -
23
openbench
Provider-agnostic, open-source evaluation infrastructure
openbench is an open-source, provider-agnostic evaluation infrastructure designed to run standardized, reproducible benchmarks on large language models (LLMs), enabling fair comparison across different model providers. It bundles dozens of evaluation suites — covering knowledge, reasoning, math, code, science, reading comprehension, long-context recall, graph reasoning, and more — so users don’t need to assemble disparate datasets themselves. With a simple CLI interface (e.g. bench eval... -
24
WavTokenizer
SOTA discrete acoustic codec models with 40/75 tokens per second
...Its architecture incorporates a broader vector-quantization space, extended contextual windows, and improved attention networks, combined with multi-scale discriminators and inverse Fourier transform blocks to enhance waveform reconstruction. Extensive experiments show that WavTokenizer matches or surpasses previous neural codecs across speech, music, and general audio on both objective metrics and subjective listening tests. -
25
Agentex
Open source codebase for Scale Agentex
...The design encourages clean separation between experiment configuration and code, which makes sharing results or re-running baselines straightforward. Teams use it to progress from prototypes to production-ready agent behaviors by iterating on prompts, adding tools, and validating improvements with consistent metrics.
