Search Results for "solver"

...It’s designed to let you define “evals” (evaluation tasks) in a structured way and run them against different models or agents, with the ability to score, compare, and analyze results. The framework supports templated YAML eval definitions, solver-based evaluations, custom metrics, and composition of multi-step evaluations. It includes utilities and APIs to plug in completion functions, manage prompts, wrap retries or error handling, and register new evaluation types. It also maintains a growing registry of standard benchmarks or “evals” that users can reuse (for example, tasks measuring reasoning, factual accuracy, or chain-of-thought capabilities). ...