Download
Ensure high-quality LLM outputs with automatic evals. Use a representative sample of user inputs to reduce subjectivity when tuning prompts. Use built-in metrics, LLM-graded evals, or define your own custom metrics. Compare prompts and model outputs side-by-side, or integrate the library into your existing test/CI workflow. Use OpenAI, Anthropic, and open-source models like Llama and Vicuna, or integrate custom API providers for any LLM API.
Features
- Create a list of test cases
- Set up evaluation metrics
- Select the best prompt & model
- Use a representative sample of user inputs to reduce subjectivity when tuning prompts
- Use built-in metrics, LLM-graded evals, or define your own custom metrics
- Compare prompts and model outputs side-by-side, or integrate the library into your existing test/CI workflow
Project Samples
Categories
Large Language Models (LLM)License
MIT LicenseFollow promptfoo
Other Useful Business Software
Ship Agents Faster
Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of promptfoo!