promptfoo Files

Evaluate and compare LLM outputs, catch regressions, improve prompts

This is an exact mirror of the promptfoo project, hosted at https://github.com/promptfoo/promptfoo. SourceForge is not affiliated with promptfoo.

The interactive file manager requires Javascript. Please enable it or use sftp or scp.
You may still browse the files here.

Name	Modified	Size	InfoDownloads / Week
Parent folder
0.124.1 source code.tar.gz	2026-10-08	183.1 MB	0
0.124.1 source code.zip	2026-10-08	186.2 MB	1
README.md	2026-10-08	4.5 kB	0
Totals: 3 Items		369.3 MB	1

0.124.1 (2026-10-08)

Features

assertions: record cost without a threshold (#11451) (037f1f6)
providers: add OpenAI Decisions support (#11454) (2dbf3ec)
providers: expose catalog text-token pricing (#11452) (3ecce2e)

Bug Fixes

assertions: preserve async expression grading (#11453) (1df3eb8)
assertions: preserve keyword-property grading (#11450) (a2da804)
assertions: preserve regex operands after yield (#11463) (4b4151a)
code-scan: mint OIDC after CLI installation (#11467) (9a404c8)
deps: bump the npm_and_yarn group across 1 directory with 3 updates (#11441) (84f7aaa)
deps: patch Sharp and Codex SDK security vulnerabilities (#11456) (0c78299)
eval: respect #gid in Google Sheets URLs (#11460) (8950bce)
handle assertion comments and dataset limits (#11435) (a65fe81)
preserve Cloud errors and check npm aliases (#11447) (8321730)
providers: expose incomplete Responses status (#11465) (49ba1ce)
providers: retain prepared custom environment (#11442) (4415bcb)
redteam: report missing target output as errors (#11445) (a2e6681)
redteam: sample dataset plugin rows uniformly (#11461) (e9ba85c)
redteam: stop Meta on invalid JSON requests (#11466) (d56d58c)
redteam: sync Codex paths and client cleanup (#11462) (99b2b09)
util: preserve complete JSON objects (#11468) (bde0dbe)

Performance Improvements

eval: index extracted tests when filtering results (#11464) (ee1e987)

Source: README.md, updated 2026-10-08

Other Useful Business Software

Ship Agents Faster Icon

Ship Agents Faster

Transform your applications and workflows into powerful agentic systems at global scale.

Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.

Start Free

$300 Free Credits to Build on Google Cloud Icon

$300 Free Credits to Build on Google Cloud

New customers can spin up VMs, build with AI, and query data at no cost.

Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.

Start Free

Ship Agents Faster

Transform your applications and workflows into powerful agentic systems at global scale.

Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.

Start Free

Recommended Projects

DeepEval
DeepEval is a simple-to-use, open-source LLM evaluation framework, for evaluating and testing large-language model systems. It is similar to Pytest but specialized for unit testing LLM outputs. DeepEval incorporates the latest research to evaluate LLM outputs based on metrics such as G-Eval, hallucination, answer relevancy, RAGAS, etc., which uses LLMs and various other NLP models that run locally on your machine for evaluation. Whether your application is implemented via RAG or fine-tuning, LangChain, or LlamaIndex, DeepEval has you covered. With it, you can easily determine the optimal hyperparameters to improve your RAG pipeline, prevent prompt drifting, or even transition from OpenAI to hosting your own Llama2 with confidence.
In-The-Wild Jailbreak Prompts on LLMs
A dataset consists of 15,140 ChatGPT prompts from Reddit
Prompt Master
A Claude skill that writes the accurate prompts for any AI tool
Arch
Arch is an intelligent prompt gateway. Engineered with (fast) LLMs
DevoxxGenie
DevoxxGenie is a plugin for IntelliJ IDEA that uses local LLM's