Search Results for "metrics"
Sort By:
Showing 240 open source projects for "metrics"
View related business solutions-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
FFmpeg Quality Metrics
Calculate quality metrics with FFmpeg (SSIM, PSNR, VMAF, VIF)
FFmpeg Quality Metrics is a Python-based tool that evaluates video quality by calculating objective metrics using FFmpeg. It supports widely used metrics such as PSNR, SSIM, VIF, MSAD, and VMAF, enabling detailed comparison between reference and distorted video files. The tool outputs both per-frame data and aggregated statistics like averages and standard deviation, making it useful for research, encoding optimization, and benchmarking. -
2
TorchMetrics
Machine learning metrics for distributed, scalable PyTorch application
TorchMetrics is a collection of 80+ PyTorch metrics implementations and an easy-to-use API to create custom metrics. Your data will always be placed on the same device as your metrics. You can log Metric objects directly in Lightning to reduce even more boilerplate. The module-based metrics contain internal metric states (similar to the parameters of the PyTorch module) that automate accumulation and synchronization across devices! -
3
MetricFlow
MetricFlow allows you to define, build, and maintain metrics in code
MetricFlow is an open-source semantic layer engine designed to help organizations define, manage, and query business metrics in a consistent, governed way. It works alongside a data stack—typically built with dbt—and allows you to express metrics as YAML‐based definitions tied to semantic models and dimension tables, rather than embedding logic ad-hoc across many dashboards or scripts. When a user or tool requests a metric (e.g., “monthly revenue by region”), MetricFlow generates optimized, warehouse-specific SQL to compute that metric, handling joins, filters, time grains, offsets, and other complexities under the hood. ... -
4
TorchMetrics AI
Machine learning metrics for distributed, scalable PyTorch application
TorchMetrics is a collection of 100+ PyTorch metrics implementations and an easy-to-use API to create custom metrics. -
Paessler: Easy to Use With Enterprise Power. Free TrialYou shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
-
5
Pydantic Logfire
Python observability platform for tracing apps, metrics, and logs
...It is built by the team behind Pydantic and follows a philosophy of combining powerful capabilities with ease of use, making it accessible to entire engineering teams. Pydantic Logfire provides deep visibility into application performance by capturing traces, metrics, and logs through an OpenTelemetry-based architecture. It is particularly strong in Python environments, offering detailed insights into Python objects, event loops, database queries, and validation flows. Logfire also integrates closely with Pydantic models, enabling developers to inspect and analyze how data moves through validation layers. ... -
6
django-prometheus
Export Django monitoring metrics for Prometheus.io
Export Django monitoring metrics for Prometheus.io. This library provides Prometheus metrics for Django-related operations. Prometheus uses Histogram based grouping for monitoring latencies. You can define custom buckets for latency, adding more buckets decreases performance but increases accuracy. SQLite, MySQL, and PostgreSQL databases can be monitored. Just replace the ENGINE property of your database, replacing django.db.backends with django_prometheus.db.backends. ... -
7
Simple Evals
Lightweight framework for evaluating large language model performance
...It is particularly useful for sanity checks, exploratory research, and comparing performance across different models or configurations. The project provides clear structures for defining datasets, metrics, and evaluation logic, while staying minimal enough to adapt for custom use cases. With its straightforward design, simple-evals is well-suited for rapid iteration and for teams that want to integrate evaluation into their model development workflows. -
8
OpenJarvis
Personal AI, On Personal Devices
...Developed as part of the Intelligence Per Watt research initiative, it focuses on improving the efficiency and practicality of on-device AI systems. The framework provides shared primitives for building local-first agents, along with evaluation tools that measure performance using metrics such as energy consumption, latency, cost, and accuracy. OpenJarvis integrates with local inference engines like Ollama, vLLM, SGLang, and llama.cpp to run language models directly on personal hardware. It also includes a learning loop that allows models to improve over time using locally generated interaction traces. By prioritizing local execution and efficiency, OpenJarvis aims to provide a foundation for privacy-preserving personal AI assistants. -
9
PyTorch Ignite
Library to help with training and evaluating neural networks
...Extremely simple engine and event system. Out-of-the-box metrics to easily evaluate models. Built-in handlers to compose training pipeline, save artifacts and log parameters and metrics. -
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
10
mosaicml composer
Supercharge Your Model Training
composer is a deep learning training framework built on PyTorch and designed to make large-scale model training more efficient, scalable, and customizable. At the center of the project is a highly optimized Trainer abstraction that simplifies the management of training loops, parallelization, metrics, logging, and data loading. The framework is intended for modern workloads that may span anything from a single GPU to very large distributed training environments, which makes it suitable for both experimentation and production-scale development. It includes built-in support for distributed training strategies such as Fully Sharded Data Parallelism and standard Distributed Data Parallel execution, helping teams scale models without having to assemble as much infrastructure by hand. -
11
Logfire MCP
The Logfire MCP Server is here
The Logfire MCP Server is a Model Context Protocol server that allows AI applications to access OpenTelemetry traces and metrics sent to Logfire. It enables retrieval and analysis of telemetry data, enhancing debugging and observability workflows. -
12
LangCheck
Simple, Pythonic building blocks to evaluate LLM applications
Simple, Pythonic building blocks to evaluate LLM applications. -
13
Ragas
Supercharge Your LLM Application Evaluations
Objective metrics, intelligent test generation, and data-driven insights for LLM apps. Ragas is your ultimate toolkit for evaluating and optimizing Large Language Model (LLM) applications. Say goodbye to time-consuming, subjective assessments and hello to data-driven, efficient evaluation workflows. Don't have a test dataset ready? We also do production-aligned test set generation. -
14
DeepEval is a simple-to-use, open-source LLM evaluation framework, for evaluating and testing large-language model systems. It is similar to Pytest but specialized for unit testing LLM outputs. DeepEval incorporates the latest research to evaluate LLM outputs based on metrics such as G-Eval, hallucination, answer relevancy, RAGAS, etc., which uses LLMs and various other NLP models that run locally on your machine for evaluation. Whether your application is implemented via RAG or fine-tuning, LangChain, or LlamaIndex, DeepEval has you covered. With it, you can easily determine the optimal hyperparameters to improve your RAG pipeline, prevent prompt drifting, or even transition from OpenAI to hosting your own Llama2 with confidence.
-
15
DefectDojo
DefectDojo is a DevSecOps and vulnerability management tool
...DefectDojo has bi-directional integration with JIRA to manage vulnerabilities in developer's natural backlogs. DefectDojo has smart features that learn over time and can automatically tune results. DefectDojo has everything you need to track projects, personnel, metrics, and tasks. DefectDojo allows you to focus on what you actually enjoy doing, rather than reports and metrics. -
16
AICGSecEval
A.S.E (AICGSecEval) is a repository-level AI-generated code security
...AICGSecEval combines static and dynamic evaluation techniques to analyze generated code for vulnerabilities and functional correctness. The framework includes datasets, test cases, and evaluation metrics that measure how AI programming tools perform across multiple programming languages and vulnerability categories. -
17
AIDE ML
AI-Driven Exploration in the Space of Code
...Instead of relying on manual experimentation, the agent autonomously drafts machine learning pipelines, debugs errors, and benchmarks performance against user-defined evaluation metrics. The system repeatedly improves its generated code by exploring different implementation paths and selecting the best-performing solutions. AIDE ML is packaged as a Python toolkit with built-in utilities such as command-line tools, configuration presets, and visualization interfaces that allow researchers to observe how the search process evolves. ... -
18
Flower
Real-time monitor and web admin for Celery distributed task queue
Flower is an open-source web application that provides real-time monitoring and administrative control for Celery distributed task queues. It exposes detailed visibility into worker status, task execution history, and queue metrics through an interactive web dashboard. Developers and operators can remotely manage workers by restarting instances, adjusting pool sizes, revoking tasks, or applying rate limits without direct server access. Flower also supports broker monitoring and integrates with authentication providers and Prometheus for metrics export, making it suitable for production environments. ... -
19
Uncertainty Baselines
High-quality implementations of standard and SOTA methods
...Each baseline emphasizes reproducibility: fixed seeds, standard splits, and strong metrics such as calibration error, AUROC for OOD, and accuracy under shift. -
20
Vedana
Open source multi-agent RAG over a knowledge graph
...The system lets agents navigate data step by step through Cypher queries, vector search, document lookup, and source verification. Its architecture combines a knowledge graph, pgvector-based embeddings, incremental ETL, and a backoffice interface for chat, metrics, prompt tuning, and data loading. It also includes JIMS, a framework for persistent conversational agents with typed events and pluggable pipelines. Overall, Vedana is useful for teams that need reliable answers from real data, especially when relationships, counts, rules, and source-backed reasoning matter. -
21
Grafana
Leading open-source visualization and observability platform
...With support for 100+ data source plugins—including Prometheus, Loki, Elasticsearch, InfluxDB, SQL/NoSQL databases, and OpenTelemetry—Grafana helps teams correlate metrics, logs, and traces across applications and infrastructure. Users can build interactive dashboards with rich visualizations, template variables, and reusable panels to monitor systems and troubleshoot issues in real time. Grafana includes capabilities such as ad hoc data exploration, alerting, annotations, and flexible query support. ... -
22
autoresearch-macos
AI agents running research on single-GPU nanochat training
autoresearch-macos is a macOS-focused adaptation of autonomous research loop systems inspired by the autoresearch paradigm, enabling AI agents to iteratively improve machine learning models through self-directed experimentation. The system follows a structured loop in which an agent modifies a training script, executes a fixed-duration experiment, evaluates performance metrics, and decides whether to keep or revert changes. It is designed to operate efficiently within macOS environments, making it accessible for developers working outside traditional high-performance GPU clusters. The project typically includes components such as data preparation scripts, a training loop, and an instruction file that guides the agent’s behavior. ... -
23
GDScript Toolkit
Independent set of GDScript tools - parser, linter and formatter
...At the moment it provides a parser that produces a parse tree for debugging and educational purposes. A linter that performs a static analysis according to some predefined configuration. A formatter that formats the code according to some predefined rules. A code metrics calculator which calculates the cyclomatic complexity of functions and classes. To install this project you need python3 and pip. Regardless of the target version, installation is done by pip3 command and for stable releases, it downloads the package from PyPI. -
24
Evaluate
A library for easily evaluating machine learning models and datasets
Evaluate is a library that makes evaluating and comparing models and reporting their performance easier and more standardized. -
25
TikTok-ViewBot
ViewBot using requests updated 2025
...It is often used to study rate limits, signature schemes, request patterns, and the fragility of naïve automation. Because it touches on automation against a third-party service, responsible use and adherence to platform terms are emphasized; using such tools to manipulate metrics or violate policies is unethical and can be unlawful. From a security research perspective, understanding these patterns helps both defenders and platform engineers improve abuse detection. For developers, the repo also serves as a cautionary example of how brittle unofficial integrations can be and why resilient, compliant APIs matter.
