promptfoo Files

What's Changed

New Features

• feat: add support for OpenAI deep research models by @mldangelo in https://github.com/promptfoo/promptfoo/pull/4661
• feat: add mcp server by @MrFlounder in https://github.com/promptfoo/promptfoo/pull/4595
• feat: add support for connecting to existing Chrome browser sessions by @mldangelo in https://github.com/promptfoo/promptfoo/pull/5069

Bug Fixes

• fix(providers): Preserve text formatting when no images present for Google provider by @trojanowski in https://github.com/promptfoo/promptfoo/pull/5058
• fix: progress bars incrementing beyond their maximum values by @mldangelo in https://github.com/promptfoo/promptfoo/pull/5049
• fix: resolve Jest test failures and open handles by @mldangelo in https://github.com/promptfoo/promptfoo/pull/5052
• fix: skip validation for defaultTest to allow partial test case properties by @mldangelo in https://github.com/promptfoo/promptfoo/pull/4732
• fix: Nested config field for custom target json by @faizanminhas in https://github.com/promptfoo/promptfoo/pull/5076
• fix(simba): fix simba host by @sklein12 in https://github.com/promptfoo/promptfoo/pull/5092
• fix: handle multi-line prompts in parseGeneratedPrompts for testGenerationInstructions by @MrFlounder in https://github.com/promptfoo/promptfoo/pull/5093

Chores

• chore: refactor token tracking utilities, track all tokens by @mldangelo in https://github.com/promptfoo/promptfoo/pull/4897
• chore: add new fields to eval_ran telemetry by @mldangelo in https://github.com/promptfoo/promptfoo/pull/4638
• chore(redteam): improve redteam plugin error messaging by @mldangelo in https://github.com/promptfoo/promptfoo/pull/4330
• test: CoverBot: Added tests for model audit and prompt management UI components (src/app) by @use-tusk[bot] in https://github.com/promptfoo/promptfoo/pull/5087
• chore: bump version 0.117.4 by @MrFlounder in https://github.com/promptfoo/promptfoo/pull/5094

Docs

• docs(site): add AI red teaming for first-timers blog post by @mldangelo in https://github.com/promptfoo/promptfoo/pull/5017
• docs(blog): defcon and blackhat info by @vsauter in https://github.com/promptfoo/promptfoo/pull/5050
• docs: clarifiy derivedMetrics documentation by @mldangelo in https://github.com/promptfoo/promptfoo/pull/5068
• docs: update defcon posting by @mldangelo in https://github.com/promptfoo/promptfoo/pull/5070
• docs: update defcon posting by @mldangelo in https://github.com/promptfoo/promptfoo/pull/5071
• docs: switch to likert preview image by @typpo in https://github.com/promptfoo/promptfoo/pull/5083

New Contributors

• @trojanowski made their first contribution in https://github.com/promptfoo/promptfoo/pull/5058

Full Changelog: https://github.com/promptfoo/promptfoo/compare/0.117.3...0.117.4