Search Results for "scoring"
Sort By:
Showing 41 open source projects for "scoring"
View related business solutions-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
$300 Free Credits to Build on Google CloudStart your next project with $300 in free Google Cloud credit. Spin up VMs, run containers, query petabytes in BigQuery, or build agents with Gemini Enterprise Agent Platform. Once your credits are used, keep building with 20+ always-free tier products including Compute Engine, Cloud Storage, GKE, and Cloud Run functions. No commitment required—just sign up and start building.
-
1
ImageReward
[NeurIPS 2023] ImageReward: Learning and Evaluating Human Preferences
ImageReward is the first general-purpose human preference reward model (RM) designed for evaluating text-to-image generation, introduced alongside the NeurIPS 2023 paper ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation. Trained on 137k expert-annotated image pairs, ImageReward significantly outperforms existing scoring methods like CLIP, Aesthetic, and BLIP in capturing human visual preferences. It is provided as a Python package (image-reward) that enables quick scoring of generated images against textual prompts, with APIs for ranking, scoring, and filtering outputs. Beyond evaluation, ImageReward supports Reward Feedback Learning (ReFL), a method for directly fine-tuning diffusion models such as Stable Diffusion using human-preference feedback, leading to demonstrable improvements in image quality. -
2
NGBoost
Natural Gradient Boosting for Probabilistic Prediction
ngboost is a Python library that implements Natural Gradient Boosting, as described in "NGBoost: Natural Gradient Boosting for Probabilistic Prediction". It is built on top of Scikit-Learn and is designed to be scalable and modular with respect to the choice of proper scoring rule, distribution, and base learner. A didactic introduction to the methodology underlying NGBoost is available in this slide deck. -
3
Prometheus-Eval
Evaluate your LLM's response with Prometheus and GPT4
...It also provides training data and utilities for fine-tuning evaluator models so they can assess outputs according to custom scoring rubrics such as helpfulness, accuracy, or style. -
4
TreeQuest
A Tree Search Library with Flexible API for LLM Inference-Time Scaling
TreeQuest, developed by SakanaAI, is a versatile Python library implementing adaptive tree search algorithms—such as AB‑MCTS—for enhancing inference-time performance of large language models (LLMs). It allows developers to define custom state-generation and scoring functions (e.g., via LLMs), and then efficiently explores possible answer trees during runtime. With support for multi-LLM collaboration, checkpointing, and mixed policies, TreeQuest enables smarter, trial‑and‑error question answering by leveraging both breadth (multiple attempts) and depth (iterative refinement) strategies to find better outputs dynamically -
Cut Data Warehouse Costs by 54%BigQuery delivers 54% lower TCO with exabyte scale and flexible pricing. Free migration tools handle the SQL translation automatically.
-
5
AI Marketing Skills
Open-source AI marketing skills for Claude Code
AI Marketing Skills is a comprehensive open-source framework designed to transform AI agents into fully operational marketing and sales systems by equipping them with structured, reusable “skills” that automate real business workflows. Instead of simple prompts, the project provides complete operational modules that include scripts, scoring systems, and decision-making logic, allowing AI tools like Claude Code to execute complex marketing tasks end-to-end. The system is organized into multiple domains such as growth experimentation, sales pipeline generation, content production, outbound marketing, SEO optimization, and financial analysis, effectively covering the entire revenue lifecycle of a business. ... -
6
AutoAgent AI
Autonomous harness engineering
...Instead of manually tuning prompts or workflows, developers define high-level goals in a configuration file, and the system continuously modifies its own tools, orchestration, and logic based on benchmark performance. It operates through a loop of testing, analyzing failures, and refining the agent’s configuration to maximize a scoring metric. The framework uses a single-file agent harness combined with structured tasks and evaluation suites to guide optimization. It runs inside Docker for safe execution and reproducibility. This approach shifts agent development from manual design to automated optimization. The system is particularly useful for building domain-specific agents that need continuous performance improvement. -
7
WebGLM
An Efficient Web-enhanced Question Answering System
...WebGLM introduces several components that coordinate this process, including a retrieval module that selects relevant web documents, a generator that produces answers, and a scoring system that evaluates the quality of generated responses. The architecture aims to improve the reliability and usefulness of AI systems that answer questions about current or external knowledge sources. -
8
MLflow
Open source platform for the machine learning lifecycle
MLflow is a platform to streamline machine learning development, including tracking experiments, packaging code into reproducible runs, and sharing and deploying models. MLflow offers a set of lightweight APIs that can be used with any existing machine learning application or library (TensorFlow, PyTorch, XGBoost, etc), wherever you currently run ML code (e.g. in notebooks, standalone applications or the cloud). -
9
Agent Zero
Agent Zero AI framework
...There is a lot of freedom in this framework. You can instruct your agents to regularly report back to superiors asking for permission to continue. You can instruct them to use point-scoring systems when deciding when to delegate subtasks. Superiors can double-check subordinates' results and disputes. -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
Agent Behavior Monitoring
The open source post-building layer for agents
Agent Behavior Monitoring is an open-source framework designed to monitor, evaluate, and improve the behavior of AI agents operating in real or simulated environments. The system focuses on agent behavior monitoring by collecting interaction data and analyzing how agents perform across different scenarios and tasks. Developers can use the framework to observe agent actions in both online production environments and offline evaluation settings, making it useful for debugging and performance... -
11
Three.js Object Sculptor
Codex plugin that turns attached object images into code-only
Three.js Object Sculptor is a Codex plugin that converts a reference image into a procedural Three.js object written entirely in code. It first evaluates whether the image is suitable and produces an ObjectSculptSpec describing geometry, materials, lighting, hierarchy, pivots, and quality targets. Codex then follows staged passes from blockout and structural work through surface detail, interaction design, and optimization. Generated models include meaningful sockets and anchors for... -
12
MemPalace
The highest-scoring AI memory system ever benchmarked
MemPalace is an open-source AI memory system designed to solve one of the most persistent limitations of large language models: the loss of context between sessions. Instead of relying on summarization or selective extraction like most memory tools, it takes a radically different approach by storing conversations in their entirety and making them retrievable through structured organization and semantic search. The system is inspired by the classical “memory palace” mnemonic technique,... -
13
Marvin
A batteries-included library for building AI-powered software
...These functions differ from conventional ones in that they don’t rely on source code, but instead generate their outputs on-demand through AI. With AI functions, you don't have to write complex code for tasks like extracting entities from web pages, scoring sentiment, or categorizing items in your database. Just describe your needs, call the function, and you're done. AI functions work with native data types, so you can seamlessly integrate them into any codebase and chain them into sophisticated pipelines. In addition to AI functions, Marvin also introduces more flexible bots. Bots are highly capable AI assistants that can be given specific instructions and personalities or roles. -
14
Atropos
Language Model Reinforcement Learning Environments frameworks
...This framework facilitates experimentation with RLHF (Reinforcement Learning from Human Feedback), RLAIF, or multi-turn training approaches by abstracting environment logic, scoring, and logging into reusable components. -
15
zhengxi-views
Zheng Xi (Efonda Fund Manager) Investment Research Agent Skill
zhengxi-views is a traceable investment research Agent Skill centered on the public views of Zheng Xi, a fund manager at E Fund. It is built to reduce unsupported AI answers by grounding responses in original public statements, fund reports, interviews, and documented methodology. The project organizes a corpus of Zheng Xi’s views from 2012 to 2026, then connects those materials to an extracted investment framework. It also includes real fund data snapshots for managed funds and broader fund... -
16
Hiring Agent
AI agent to evaluate and score resumes
Hiring Agent is an AI-powered resume evaluation pipeline for screening technical candidates. It reads a resume PDF and converts the content into Markdown-like text. It then uses a local or hosted language model to extract structured candidate information into sectioned JSON. The system can enrich that resume data with GitHub profile and repository signals when a profile is available. After the data is collected, it produces an explainable evaluation with category scores, supporting evidence,... -
17
VideoRAG
"VideoRAG: Chat with Your Videos
VideoRAG is a retrieval-augmented generation (RAG) framework tailored for video content that enables AI systems to answer questions, summarize, and reason over long videos by combining visual embeddings with contextual search. The system works by first breaking video into clips, extracting visual and audio-textual features, and indexing them into embeddings, then using an LLM with a retriever to pull relevant segments on demand. When a user query is received, VideoRAG locates semantically... -
18
Lingua-Py
The most accurate natural language detection library for Python
Its task is simple: It tells you which language some text is written in. This is very useful as a preprocessing step for linguistic data in natural language processing applications such as text classification and spell checking. Other use cases, for instance, might include routing e-mails to the right geographically located customer service department, based on the e-mails' languages. Language detection is often done as part of large machine learning frameworks or natural language processing... -
19
uqlm
Uncertainty Quantification for Language Models, is a Python package
UQLM is a Python library developed to detect hallucinations and quantify uncertainty in the outputs of large language models. The system implements a variety of uncertainty quantification techniques that assign confidence scores to model responses. These scores help developers determine how likely a generated answer is to contain errors or fabricated information. The library includes both black-box and white-box approaches to uncertainty estimation. Black-box methods evaluate model outputs... -
20
ASSERT
Requirement-driven evaluation harness for AI agents and LLM
ASSERT is a requirement-driven evaluation harness for AI agents and LLM applications. It turns natural-language specifications, policies, product requirements, and launch criteria into structured tests that can be reviewed, executed, scored, and improved. The pipeline derives behavior categories, generates single-turn and multi-turn test cases, runs them against a target system, and uses an LLM judge to score conversations against the stated policies. It can evaluate hosted models, custom... -
21
Cheat on Content
Workflow that turns every post into a calibrated experiment
Cheat on Content is an AI-assisted workflow for creators who want to make content performance measurable instead of relying on instinct alone. It turns every post into a structured experiment by asking creators to score ideas, make blind predictions, publish, review results after a defined time window, and evolve their own content rubric. Rather than generating posts for the creator, it focuses on sharpening judgment and helping users understand why certain content performs better. The... -
22
SEO Machine
A specialized Claude Code workspace for creating long-form
SEO Machine is an AI-powered content production system built as a structured workspace for generating long-form, SEO-optimized blog content through automated workflows. It integrates research, writing, analysis, and optimization into a single pipeline, allowing users to produce high-quality articles tailored to search engine performance. The system uses specialized commands and agents to perform tasks such as keyword research, competitor analysis, content drafting, and optimization. It... -
23
VLMEvalKit
Open-source evaluation toolkit of large multi-modality models (LMMs)
VLMEvalKit is an open-source evaluation toolkit designed for benchmarking large vision-language models that combine visual understanding with natural language reasoning. The toolkit provides a unified framework that allows researchers and developers to evaluate multimodal models across a wide range of datasets and standardized benchmarks with minimal setup. Instead of requiring complex data preparation pipelines or multiple repositories for each benchmark, the system enables evaluation... -
24
Qwen3-VL-Embedding
Multimodal embedding and reranking models built on Qwen3-VL
Qwen3-VL-Embedding (with its companion Qwen3-VL-Reranker) is a state-of-the-art multimodal embedding and reranking model suite built on the open-sourced Qwen3-VL foundation, developed to handle diverse inputs including text, images, screenshots, and videos. The core embedding model maps such inputs into semantically rich vectors in a unified representation space, enabling similarity search, clustering, and cross-modal retrieval. The reranking model then precisely scores relevance between a... -
25
CLIP
CLIP, Predict the most relevant text snippet given an image
CLIP (Contrastive Language-Image Pretraining) is a neural model that links images and text in a shared embedding space, allowing zero-shot image classification, similarity search, and multimodal alignment. It was trained on large sets of (image, caption) pairs using a contrastive objective: images and their matching text are pulled together in embedding space, while mismatches are pushed apart. Once trained, you can give it any text labels and ask it to pick which label best matches a given...