promptfoo Files

0.121.4 (2026-04-10)

Features

• allow per-test opt-out of defaultTest assertions (5e5959e)
• codex: expand Codex SDK eval controls and docs (#8433) (80c3f7f)
• eval: group serial grading by provider (#8509) (d289602)
• examples: add traced openai agents python sdk example (#8354) (6870717)
• http: support structured multipart requests (#8533) (5bac47c)
• japan fiea plugin (#8316) (f330ab3)
• matchers: expose grading provider metadata in GradingResult (#8330) (03cbac6)
• openai: enable chatgpt login via codex (#8327) (5a9cb96)
• providers: add Gemma 4 provider support (#8454) (b0667ed)
• providers: add missing Anthropic SDK features and fix apiKeyRequired bug (#8351) (3060847)
• providers: expand OpenClaw support (#8589) (93f29ec)
• providers: support Codex local images and harden SDK (929790b)
• providers: support multimodal openai agents input (#8397) (4065844)
• redteam: add coding-agent redteam plugins (85a0cfe)
• redteam: add FDA medical plugins (#8456) (30e4ac3)
• redteam: add next coding-agent plugins (5ab3ba4)
• redteam: add teen safety plugins (#8308) (61aa057)
• redteam: enforce max chars per message (#8428) (9af7b7d)
• redteam: use Codex login for default text graders (#8493) (9a6b61b)

Bug Fixes

• app: clarify attack success rate label (#8386) (d088eac)
• app: clarify attack success rate label (#8387) (7482eff)
• app: keep select-all checkbox visible (#8549) (02f9064)
• assertions: apply weights to named scores in assertion results (#8206) (01da019)
• assertions: handle inverse flag in finish-reason handler (#8556) (7155f28)
• assertions: normalize javascript function assertion results (#8377) (a7fafba)
• assertions: normalize negated script assertions (#8485) (fe62243)
• assertions: preserve nested metric weights (#8558) (f3c173d)
• assertions: resolve search-rubric web providers (#8557) (af751aa)
• assertions: skip simulated-user as implicit grader (#8429) (9bc5fdf)
• auth: honour org flag in login command (#8430) (8bdb15d)
• bedrock: preserve zero-valued inference config (#8272) (1cd397e)
• browser: make stealth plugin install explicit (#8434) (aa72a34)
• cache: cache repeated evals by repeat index (#8480) (fbd59a6)
• cli: enforce exact Node engine range (#8380) (1f8e3b4)
• cli: serialize source-map support initialization (#8401) (d4a9d4a)
• code-scan: avoid npm before env for MCP npx (#8515) (7d2eacd)
• code-scan: retry scanner MCP request timeouts (#8545) (4312797)
• deps: pin claude-agent-sdk lockfile to 0.2.87 (#8400) (cbfb91a)
• deps: update dependency @anthropic-ai/sdk to ^0.82.0 (#8510) (cbc96df)
• deps: update dependency @modelcontextprotocol/sdk to ^1.28.0 (416f3cd)
• deps: update dependency @modelcontextprotocol/sdk to ^1.28.0 (5a8a732)
• deps: update dependency @modelcontextprotocol/sdk to ^1.29.0 (#8501) (8821383)
• deps: update dependency @modelcontextprotocol/sdk to ^1.29.0 (#8502) (84db2aa)
• deps: update dependency @openai/agents to ^0.8.1 (#8371) (3f2234c)
• deps: update dependency @openai/agents to ^0.8.1 (#8372) (aecdb11)
• deps: update dependency @openai/agents to ^0.8.2 (#8500) (2b2ce6c)
• deps: update dependency @opentelemetry/exporter-trace-otlp-http to ^0.214.0 (#8379) (462f1e5)
• deps: update dependency ai to ^6.0.138 (#8403) (855e6b3)
• deps: update dependency ai to ^6.0.138 (#8410) (4aa56cf)
• deps: update dependency openai to ^6.33.0 (#8388) (19e04ce)
• deps: update dependency proxy-agent to v8 (#8503) (52953a6)
• deps: update dependency undici to ^7.24.5 (#8411) (3d8a24d)
• deps: update example dependencies (#8367) (977a656)
• deps: update IBM Cloud SDK Core (#8584) (07aaf97)
• deps: update openai packages (#8368) (e5b842a)
• elevenlabs: Disable implicit retries for POST calls (#8541) (34f819a)
• elevenlabs: preserve explicit zero retries (#8358) (493ef88)
• eval: fix bugs and performance issues in evaluator orchestrator (#8481) (36a2788)
• G-Eval wrong scoring for negative criteria (#8259) (e7bba6e)
• handle generated redteam exports during eval (#8301) (ef3f67f)
• harden object guards in comparison checks (#8408) (6e81cc9)
• image should preserve aspect ratio in results table (#8279) (e96d630)
• mcp: preserve falsy values and bound truncation (#8423) (384249e)
• mcp: stabilize withTimeout timeout handling (#8399) (df7d8a9)
• openai: Avoid invalid realtime zero output tokens (#8543) (9cde9b4)
• openai: preserve agents zero token usage (#8283) (6354f4e)
• openai: preserve assistant temperature zero (#8271) (311c0a4)
• parse rate-limit reset headers (#8341) (985ab28)
• preserve configured plugin config in strategy preview (#8326) (e20458e)
• prevent SSTI in conversation-relevance assertion (#8258) (49cbe41)
• prompts: skip recursive rendering for undefined vars (#8394) (8f14e8d)
• providers: align omitDefaults handling across OpenAI and Azure (#8332) (edb5db6)
• providers: deduplicate token refresh retries (#8421) (b78ea03)
• providers: include azure moderation config in cache key (#8349) (c528c5a)
• providers: preserve explicit zero values in Mistral, AI21, and Azure Assistant config (#8555) (7612368)
• providers: prevent max_tokens leaking via passthrough in multi-provider eval (#8449) (b07ca39)
• providers: support systemInstruction for Vertex Claude models (#8522) (ff1cf68)
• providers: support transitive TypeScript provider imports (#8445) (1bb0f39)
• providers: use apiKeyEnvar in error messages across OpenAI providers (#8180) (7008087)
• providers: use rendered prompt in simulated user (#8363) (1f1fa86)
• redteam: avoid nested buttons in transform dialog (#8520) (50c65b4)
• redteam: clarify remote generation guidance (#8252) (45870ab)
• redteam: fix --env-file arg parsing and test flakiness (#8420) (6a920c5)
• redteam: guard missing cross-session leak metadata (#8238) (9924bae)
• redteam: honour generate concurrency config (#8463) (e626ea9)
• redteam: normalize code editor imports (#8306) (057f9e0)
• redteam: pass full conversation history to crescendo grader (#8239) (01619a8)
• redteam: preserve goat zero max turns (#8294) (3077acf)
• redteam: preserve voice crescendo zero max turns (#8310) (5d38b28)
• redteam: preserve zero prompt injection sample (#8291) (16b3221)
• redteam: preserve zero-valued crescendo config (#8273) (9ec66b5)
• redteam: preserve zero-valued custom config (#8274) (a76058f)
• redteam: preserve zero-valued voice crescendo config (#8270) (7d36044)
• redteam: remove dead stores and preserve metadata (#8407) (28d3383)
• redteam: skip unblocking feature checks when disabled (#8266) (c6bdf39)
• redteam: support policy multi-input generation (#8320) (be14251)
• refresh prompt listings after prompt metadata updates (#8120) (85694e9)
• replace undeclared tiny-invariant imports in app (#8309) (f85704f)
• replicate: cache string responses (#8353) (e96fc94)
• respect review language in example test generation (#8334) (3163583)
• sagemaker: preserve zero maxTokens (#8286) (75603d3)
• util: preserve shared references in safeJsonStringify (#8342) (987e8ad)
• watsonx: preserve zero max_new_tokens (#8288) (b489160)
• webui: clear output link feedback timers (#8559) (aeac87d)

Reverts

• feat(redteam): add next coding-agent plugins (#8585) (720d677)