evalship
evalship helps engineering teams test and improve LLM-powered software. At setup, your coding agent reviews every scorer, judge and prompt in your evals and finds their flaws, such as a judge that scores 1.0 when its call fails, each with fix instructions. On every pull request, evalship compares results with the default branch, reports the evals a change really broke and the likely cause, sets flaky evals aside, and flags AI calls no eval exercises. Its check can block a merge that breaks an eval. Optimize has your coding agent try prompt and model changes on your machine, such as a cheaper model or clearer instructions, and evalship says which win beyond chance and by how much in pass rate, cost and time. Evals run with your own keys, in your own GitHub Actions or on your machine. evalship reads results from pytest, Vitest, Jest, promptfoo, Langfuse, Braintrust and LangSmith, with no SDK to add. Public repositories are free; paid plans start at $19 a month.
Learn more about evalship
BiG EVAL
The BiG EVAL solution platform provides powerful software tools needed to assure and improve data quality during the whole lifecycle of information. BiG EVAL's data quality management and data testing software tools are based on the BiG EVAL platform - a comprehensive code base aimed for high performance and high flexibility data validation. All features provided were built by practical experience based on the cooperation with our customers. Assuring a high data quality during the whole life cycle of your data is a crucial part of your data governance and is very important to get the most business value out of your data. This is where the automation solution BiG EVAL DQM comes in and supports you in all tasks regarding data quality management. Ongoing quality checks validate your enterprise data continuously, provide a quality metric and supports you in solving the quality issues. BiG EVAL DTA lets you automate testing tasks in your data oriented project.
Learn more about BiG EVAL
asqav
asqav is an AI governance and security platform designed to make AI agents audit-ready by providing real-time monitoring, enforcement, and verifiable proof of every action taken by an agent. It introduces a lightweight SDK that allows developers to integrate governance directly into their agents in just a few lines of code, enabling continuous oversight across the full lifecycle of AI operations. It includes behavioral monitoring to detect issues such as drift, rate limits, and scope violations, along with advanced threat detection that identifies prompt injections, exposure of sensitive data, toxic outputs, and other risks. It enforces policy through configurable “policy gates,” which apply per-agent rules, preflight checks, and dynamic approvals before actions are executed, ensuring that agents operate within defined boundaries. asqav also provides automated incident response capabilities, including the ability to suspend, quarantine, or escalate risky agents.
Learn more about asqav
F5 AI Guardrails
F5 AI Guardrails is a runtime AI security solution designed to protect AI models, applications, agents, and connected data throughout deployment and operation. The platform helps organizations defend against adversarial threats such as prompt injection, jailbreak attacks, harmful outputs, and unauthorized AI behavior. It provides real-time monitoring and enforcement of security policies to prevent data leakage, compliance violations, and misuse of AI systems. Organizations can implement predefined guardrails or create customized policies tailored to specific business requirements and AI use cases. The platform also delivers observability, auditing, and governance capabilities that help organizations maintain visibility into AI interactions and regulatory compliance. By combining threat protection, data security, and AI governance, F5 AI Guardrails helps enterprises operate AI systems more safely and responsibly.
Learn more about F5 AI Guardrails