Download Latest Version 0.125.1 source code.zip (186.9 MB) Google Add to Preferred Sources
Home / 0.124.0
Name Modified Size InfoDownloads / Week
Parent folder
0.124.0 source code.tar.gz 2026-10-06 183.0 MB
1 1 weekly downloads
0.124.0 source code.zip 2026-10-06 186.1 MB
1 1 weekly downloads
README.md 2026-10-06 38.7 kB
1 1 weekly downloads
Totals: 3 Items   369.2 MB 3

0.124.0 (2026-10-06)

⚠ BREAKING CHANGES

  • providers: remove hosted ChatKit provider (#11434)
  • providers: make WatsonX SDKs opt-in (#11359)
  • deps: make Langfuse SDK opt-in (#11351)
  • deps: make Slack SDK opt-in (#11349)
  • providers: make OpenAI Agents an explicit install (#11177)
  • providers: make Codex Security SDK opt-in (#11325)
  • providers: make browser SDKs explicit installs (#11163)
  • deps: make local Transformers opt-in (#11119)

Features

Bug Fixes

  • anthropic: bump OAuth Claude Code user-agent to 2.1.285 (#11323) (29bddde)
  • app: share cloud configuration requests across consumers (#11193) (ef7923c)
  • assertions: accept grader result shapes earlier releases recorded (#11391) (3233b37)
  • assertions: await meteor handler so missing natural dep grades as failure (#11312) (6b00513)
  • assertions: clamp inverse RAG scores to the documented 0-1 range (#11082) (eee76e2)
  • assertions: honor the not- prefix for model-graded-closedqa and factuality (#11185) (8a1d6a7)
  • assertions: ignore blank BLEU references (#11278) (eee7e74)
  • assertions: ignore grader prose in context-relevance scoring (#10251) (016aad3)
  • assertions: keep custom reason when negated script assertions fail (#11246) (0f11b51)
  • assertions: log why select-best grading failed (#11396) (941f738)
  • assertions: validate BLEU weights before empty score (#9858) (0251010)
  • assertions: validate custom grading results (#11079) (f34fe4e)
  • cli: allow a literal comma in --filter-metadata values (#11397) (d206bb3)
  • cli: correct Cloud organization guidance (#11078) (de8a542)
  • cli: fail the exit code when a target error stops the eval (#11400) (84fbc8e)
  • cli: handle asynchronous browser launch errors (#11115) (11a4ce2)
  • cli: keep Cloud login in the token organization (#11026) (b85c2f1)
  • cli: preserve Cloud team selection (#11018) (3379c9e)
  • cli: silence internal decompression warning (#11346) (c0017f6)
  • code-scan: include large unknown text files (#11129) (3d65dce)
  • code-scan: launch filesystem MCP on Windows (#11244) (9759f34)
  • code-scan: pin MCP launch and await shutdown (#11285) (b0c124f)
  • code-scan: share diff range parsing (#11204) (9a55724)
  • config: accept numeric and boolean env values (#11386) (20b1ffe)
  • config: expand globs in single-string tests (#11097) (874f44a)
  • config: isolate evaluation context and preserve replay state (#10893) (8d5e81e)
  • config: preserve imported test file path resolution (#11283) (78a76bd)
  • config: restore relative labels for file prompts (#11384) (29a2e4f)
  • config: restore relative vars file paths (#11380) (c4863c9)
  • constrain provider and shell inputs (#11318) (b93b2e7)
  • csv: keep explicit thresholds for assertion types without a CSV default (#11337) (7ab9e6e)
  • deps-dev: bump brace-expansion from 1.1.18 to 1.1.21 in the npm_and_yarn group across 1 directory (#11316) (74eda9a)
  • deps-dev: bump http-cache-semantics from 4.2.0 to 4.3.0 in the npm_and_yarn group across 1 directory (#11439) (1287cfd)
  • deps: converge Azure and IBM SDK patches (#11164) (cdb544d)
  • deps: migrate simple-git to 4.0.2 (#11428) (7504383)
  • deps: patch serialize-javascript escaping (#11332) (ae1e7d5)
  • deps: pin ansi-tokenize to 0.3.0 (#10990) (a89e3cc)
  • deps: preserve terminal OpenAI streaming events (#11143) (d405281)
  • deps: replace vulnerable archive and image parsers (#11106) (71b8b5a)
  • deps: require csv-parse 7.0.3 (#11372) (61666d5)
  • deps: restore compatible watcher and proxy versions (#11120) (8386dc4)
  • deps: update anthropic packages (#11009) (a33f261)
  • deps: update anthropic packages (#11073) (9d24937)
  • deps: update anthropic packages (#11264) (d6f0419)
  • deps: update Claude Agent SDK to v0.3.278 (#11292) (1de8a1e)
  • deps: update dependency @anthropic-ai/claude-agent-sdk to v0.3.276 (#11256) (33bb323)
  • deps: update dependency ai to ^6.0.286 (#11343) (0e1e076)
  • deps: update dependency js-yaml to v5.4.2 (#11038) (d59f045)
  • deps: update dependency zod to v4.6.1 (#11000) (dd144d5)
  • deps: update DOMPurify to 3.4.16 (#11326) (eb5f2e1)
  • deps: update Drizzle ORM and Kit (#11344) (0640453)
  • deps: update Engine.IO client to 6.6.7 (#11423) (70575c4)
  • deps: update engine.io to v6.6.11 (#11403) (78783ea)
  • deps: update example dependencies (#11418) (26492a8)
  • deps: update fast-uri to v3.1.8 (#11279) (5ba35f7)
  • deps: update Google Sheets to 18.0.1 (#11389) (ef45a94)
  • deps: update Hono to 4.13.9 (#11417) (e73b6dd)
  • deps: update hono to v4.13.8 (#11257) (76bdf58)
  • deps: update ibm-cloud-sdk-core to v5.6.1 (#10986) (17b5a0a)
  • deps: update ip-address to v10.7.1 (#11064) (7110bef)
  • deps: update lru-cache to v11.5.3 (#11287) (fa59117)
  • deps: update MCP SDK to 1.32.0 for redirect security fixes (#11412) (07346fe)
  • deps: update OpenCode SDK to 1.18.32 (#11413) (863271e)
  • deps: update opencode sdk to v1.18.31 (#11252) (1c6df33)
  • deps: update schema ref parser to 16.0.3 (#11328) (b86456d)
  • deps: update Socket.IO client to 4.8.4 (#11424) (8d66ecf)
  • deps: update Socket.IO server to 4.8.4 (#11427) (f202c5c)
  • deps: update type definitions (#11010) (8174e60)
  • env: honor --env-file and config env for prompt separator and assertion concurrency (#11059) (851641e)
  • eval: cap HuggingFace dataset pages at 100 rows (#11421) (f5738e3)
  • eval: keep HuggingFace rows when a prefetch fails (#11419) (13f82c3)
  • eval: sanitize invalid XML characters in JUnit reports (#10994) (5502626)
  • eval: validate the personas response array (#11297) (7f73160)
  • examples: align LiteLLM prompts and setup (#11147) (36bee67)
  • examples: complete minimal LangGraph Python setup (#11155) (63df964)
  • examples: refresh CrewAI async execution (#11142) (c219d9c)
  • examples: refresh Google ADK integration (#11152) (98ebda0)
  • examples: refresh Python provider dependencies (#11108) (5493966)
  • examples: repair F-score metrics and refresh dependencies (#11114) (e6bcec9)
  • examples: simplify and verify browser demos (#11124) (edaebce)
  • examples: simplify Docker sandbox dependencies (#11125) (9876eda)
  • examples: use supported E2B sandbox controls (#11146) (5035fee)
  • harden agent examples and gateway errors (#11096) (fe63b90)
  • harden AI finding edge cases (#10997) (757ff56)
  • isolate Foundry clients and preserve transforms (#11236) (51a1585)
  • isolate HTTP signing and validate webhook scores (#10976) (27c866b)
  • output: preserve JUnit sanitized name identities (#11033) (d833727)
  • package: support CommonJS API consumers (#10765) (23a90ab)
  • preserve log context and validate media paths (#11379) (aae7bff)
  • preserve log messages and provider responses (#11407) (0e911c9)
  • preserve path stats, provider metadata, and video text (#11363) (a16d12b)
  • prompts: preserve descriptor config through file loading (#11219) (730969c)
  • propagate filesystem blob deletion failures (#11179) (e7283b8)
  • providers: accept Ollama tool calls back in a multi-turn conversation (#10980) (27ad4b9)
  • providers: address review findings from #10635 and #10632 (#10637) (d1aa582)
  • providers: align Anthropic tools and output (#10821) (f74b2f4)
  • providers: apply Claude sampling rules on Vertex and Bedrock (#11284) (c753817)
  • providers: avoid duplicate Azure fallback warnings (#11029) (e5d3386)
  • providers: bound Azure Foundry tool loops and share credential setup (#11030) (56b1058)
  • providers: control Foundry execution and cancellation (#11086) (68ce0f7)
  • providers: correct explicit Mantle endpoint advice (#11408) (b5505e5)
  • providers: deduplicate Google tool declarations (#10982) (b3f6587)
  • providers: degrade gracefully on malformed Ollama responses (#10984) (d536238)
  • providers: fill recent model catalog gaps (#11350) (f5c9b85)
  • providers: forward OpenCode env and trace context (#10606) (1db5b5e)
  • providers: honor Bedrock region overrides (#11052) (2b0d41a)
  • providers: honor bustCache in the Ollama completion provider (#10979) (c0ba816)
  • providers: honor per-call cache overrides (#10917) (56e361b)
  • providers: keep colons in model names from provider paths (#11216) (b0c0773)
  • providers: keep distinct MCP servers distinct for Claude Agent SDK (#10841) (87434e5)
  • providers: keep non-ASCII words in ElevenLabs WER (#11406) (a76630e)
  • providers: launch npm-installed Codex app-server on Windows (#11282) (3e8220f)
  • providers: normalize missing Ollama tool arguments (#11005) (74d8f90)
  • providers: normalize Vercel finish reasons (#11076) (103b24f)
  • providers: pass custom system prompts to older Claude Agent SDKs (#11392) (9cc10fe)
  • providers: preserve Foundry response state and usage (#11081) (fdb5e64)
  • providers: prevent n8n webhook replay (#11240) (06b44fb)
  • providers: prevent scheduler replay of stateful n8n calls (#11229) (8b09087)
  • providers: reconcile model catalogs, routing and costs (#9792) (3e6ad35)
  • providers: recover and share Azure authentication (#11080) (f66a47c)
  • providers: reject withdrawn bedrock models on all geos (#10825) (ada9cbe)
  • providers: release resources while preserving reuse (#11432) (bbd7795)
  • providers: repair provider config bugs found in the docs audit (#10989) (455b702)
  • providers: report per-call Claude usage (#11266) (3ac0d9e)
  • providers: respect cache policy for ElevenLabs TTS (#11194) (3b5ca95)
  • providers: restore file env override precedence (#11364) (65585a7)
  • providers: scope prepared cloud provider resolution (#11440) (86e69ca)
  • providers: show the global install command for opt-in SDKs (#11394) (23da761)
  • providers: simplify Vercel response handling (#11077) (67cdd30)
  • providers: stop capping GPT-6 Chat with OPENAI_MAX_TOKENS (#11393) (a799bbc)
  • providers: treat Bedrock GPT-6 Sol and Luna as reasoning models (#11040) (e576652)
  • providers: treat null Ollama tool arguments as empty (#11023) (ccf5d68)
  • providers: validate OpenAI Agents token counts (#11019) (a1cb8af)
  • providers: warn on partial Azure service principal config (#11003) (90df930)
  • redteam: honor explicit redteamProvider in iterative, crescendo and iterative:tree (#11004) (4edf6b8)
  • redteam: preserve zero sampling values (#11238) (eace58c)
  • redteam: refresh LangChain example dependencies and token reporting (#11140) (0933e02)
  • redteam: reject invalid VLSU concurrency (#11313) (6df001e)
  • redteam: report the turn the grader flagged (#11032) (30ca142)
  • redteam: simplify API demo Python dependencies (#11137) (9991218)
  • remove bias from array sampling (#11307) (2c628e6)
  • remove unused legacy media sidecar writes (#11178) (d3653fd)
  • storage: keep the legacy media index read-only (#11226) (b698a4a)
  • storage: retry Windows blob publication contention (#11341) (1c6f570)
  • vertex: preserve zero Claude sampling parameters (#11243) (588e089)
  • webui: count uniform custom integer scores in histogram (#11309) (017f4c8)
  • webui: keep downloaded eval configs importable (#11387) (185f292)
  • webui: reset the lifetime of replacement toasts (#11197) (0a0c136)
  • webui: share safe editor highlighting fallbacks (#11207) (0910b3a)
  • webui: use per-column named metric totals in column headers (#11075) (9770bd6)

Performance Improvements

Miscellaneous Chores

Code Refactoring

Source: README.md, updated 2026-10-06