Download
Ensure high-quality LLM outputs with automatic evals. Use a representative sample of user inputs to reduce subjectivity when tuning prompts. Use built-in metrics, LLM-graded evals, or define your own custom metrics. Compare prompts and model outputs side-by-side, or integrate the library into your existing test/CI workflow. Use OpenAI, Anthropic, and open-source models like Llama and Vicuna, or integrate custom API providers for any LLM API.
Features
- Create a list of test cases
- Set up evaluation metrics
- Select the best prompt & model
- Use a representative sample of user inputs to reduce subjectivity when tuning prompts
- Use built-in metrics, LLM-graded evals, or define your own custom metrics
- Compare prompts and model outputs side-by-side, or integrate the library into your existing test/CI workflow
Project Samples
Categories
Large Language Models (LLM)License
MIT LicenseFollow promptfoo
Other Useful Business Software
Secure File Transfer for Windows with Cerberus by Redwood
Cerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of promptfoo!
