Search Results for "test coverage"

...It follows an iterative workflow where generated tests are executed, validated, and refined to ensure they contribute meaningful coverage improvements. Internally, Qodo Cover uses a modular architecture that includes components for prompt generation, AI interaction, coverage analysis, and test validation. It supports scanning entire repositories to automatically detect test files, gather relevant context, and extend test suites accordingly.

CodiumAI Cover Agent aims to help efficiently increasing code coverage, by automatically generating qualified tests to enhance existing test suites.

ASSERT is a requirement-driven evaluation harness for AI agents and LLM applications. It turns natural-language specifications, policies, product requirements, and launch criteria into structured tests that can be reviewed, executed, scored, and improved. The pipeline derives behavior categories, generates single-turn and multi-turn test cases, runs them against a target system, and uses an LLM judge to score conversations against the stated policies. It can evaluate hosted models, custom...

...Its scanning workflow uses a dual-LLM architecture in which one model acts as the target being tested and another acts as a controller that evaluates whether an attack succeeded. The repository emphasizes broad coverage, including test rules for prompt stealing, jailbreaks, harmful content generation, hate-related outputs, social bias, and distraction attacks. It also supports multiple providers such as OpenAI, Anthropic, Google, xAI, and open-source models through Ollama, making it flexible for both commercial and local deployments.

Reliable Fidelity and Diversity Metrics for Generative Models (ICML 2020). Devising indicative evaluation metrics for the image generation task remains an open problem. The most widely used metric for measuring the similarity between real and generated images has been the Fréchet Inception Distance (FID) score. Because it does not differentiate the fidelity and diversity aspects of the generated images, recent papers have introduced variants of precision and recall metrics to diagnose those...