promptfoo Files

0.122.1 (2026-08-26)

Features

• eval: distinguish scan footprint from incurred token usage (#10497) (8df8ef0)
• providers: add Gemini 3.7 Flash and Flash-Lite (#10389) (bc6b189)
• redteam: account for multi-turn attacker and grading tokens (#10407) (33cc8bc)
• redteam: report generation, target, attacker, and grading tokens (#10408) (0891e4e)
• tracing: add test-case roots and universal target spans (#10328) (334586e)
• tracing: align spans with OpenTelemetry GenAI conventions (#10361) (e6f44f8)
• tracing: fetch external spans from Grafana Tempo (#10327) (1b445b0)
• tracing: fetch external traces from Braintrust (#10329) (f372318)
• tracing: fetch external traces from Langfuse (#10372) (f7809f6)
• tracing: harden trace span persistence (#10326) (d9d0e82)
• tracing: instrument grading and red-team execution (#10362) (a53ba32)
• tracing: integrate OpenAI Agents and Vercel SDK telemetry (#10364) (8f89002)
• tracing: standardize tool execution across providers (#10363) (b54ccbd)
• tracing: support Claude and Codex subprocess telemetry (#10365) (84c518f)

Bug Fixes

• app: clarify red team grading actions (#10366) (84a5cf2)
• assertions: avoid regex backtracking in GLEU and SQL parsing (#10373) (57243cf)
• assertions: correct inverse JSON failure messages (#9824) (d1d8de3)
• assertions: reject out-of-range trace-span-duration percentile (#10089) (49c0f6d)
• assertions: score tokenless GLEU inputs as zero (#9850) (cc8c0c6)
• deps: align Chevrotain CST generator with parser (#10360) (6ae80cd)
• deps: block install scripts in Docker builds (#10330) (c4454c4)
• deps: bundle the runtime cache dependency (#10302) (b5f8ea3)
• deps: enforce npm release age during lockfile maintenance (#10396) (42c1076)
• deps: hold jsdom below the release our Node floor cannot install (#10351) (2dcb128)
• deps: hold tanstack table below incompatible v9 (#10388) (fded938)
• deps: keep Renovate on the supported npm major (#10393) (f5034c1)
• deps: limit pnpm install scripts and new releases (#10331) (0dbf6c8)
• deps: patch vulnerable transitive js-yaml releases (#10356) (2626fd0)
• deps: prevent incompatible Chevrotain parser updates (#10306) (77c08d5)
• deps: prevent incompatible chevrotain-allstar upgrades (#10399) (5357d06)
• deps: restore standard cache dependency resolution (#10314) (ca65846)
• deps: update anthropic packages (#10426) (ef4cd6b)
• deps: update anthropic packages (#10471) (ed5bae3)
• deps: update dependency @apidevtools/json-schema-ref-parser to v16 (#10474) (7600c55)
• deps: update dependency @hono/node-server to v2.1.0 (#10385) (af026d0)
• deps: update dependency @hono/node-server to v2.1.1 (#10472) (24ef223)
• deps: update dependency chalk to v6 (#10309) (1c30e18)
• deps: update dependency gcp-metadata to v9 (#10350) (16cb1f3)
• deps: update dependency js-yaml to v5.2.3 (#10354) (fc36f28)
• deps: update dependency js-yaml to v5.3.0 (#10473) (e419aff)
• deps: update dependency openai to v7 (#10318) (9a5f84b)
• providers: apply path-traversal guard to all callback loaders (#9472) (87a5a42)
• providers: avoid crash on empty choices in snowflake and openrouter (#10418) (c149fcf)
• providers: count cached and thinking tokens for Claude on Bedrock (#10492) (0170037)
• providers: count Claude Agent SDK subagent tokens (#10455) (5cd3baa)
• providers: future-proof Claude sampling without alias collisions (#10359) (9a1e965)
• providers: harden coding-agent tool policies (#9044) (7759d6f)
• providers: honor Claude Agent SDK cancellation (#10459) (a488cb4)
• providers: isolate Anthropic credentials and Meta Responses (#10110) (f24966a)
• providers: normalize OpenRouter URLs and validate SDK ranges (#10433) (a0b9c34)
• providers: preserve Claude SDK task tools (#10489) (ac71008)
• providers: preserve scheduled Claude task results (#10233) (b1999a0)
• providers: raise max_tokens above a manual thinking budget on every Claude route (#10491) (73e0e1f)
• providers: redact AWS SigV4 credentials in sanitized output (#10482) (eeebd72)
• providers: resolve the anthropic shorthand for uncatalogued models (#10490) (3c77783)
• providers: update GPT model costs (#10319) (9988fcb)
• providers: validate coding-agent targets and cache keys (#10111) (2003fcc)
• redteam: account for all scan-time model calls (#10479) (9b11bd8)
• redteam: account for remote strategy generation usage (#10445) (02c9209)
• redteam: count unmetered generation requests (#10429) (b74d5e0)
• redteam: preserve cached generation and retry accounting (#10507) (4868649)
• redteam: preserve cumulative grading token usage (#10406) (be75b25)
• redteam: prevent cached and duplicate token charges (#10417) (4ba76df)
• redteam: separate Mischievous User token usage (#10431) (f30f67d)
• scheduler: parse RFC 3339 rate-limit reset headers (#10322) (12f0322)
• tracing: capture effective OpenAI completion request settings (#10368) (176215c)
• tracing: select meaningful model and tool spans for red teaming (#10358) (be44f83)
• webui: stop the media bulk download when the page unmounts (#10487) (2309f65)