Switchyard v0.3.0

Switchyard 0.3.0 adds reusable runner integrations, Advisor Gate and Composite routing, Auto presets, and updated Rust and Python embedding APIs. It also fixes provider translation and tool-loop behavior and removes the legacy Python server and coding-agent launchers.

This is a breaking upgrade for library and Python CLI users. Review the migration notes below before upgrading.

Full [changelog](https://github.com/NVIDIA-NeMo/Switchyard/blob/v0.3.0/CHANGELOG.md#030 · Changes since v0.2.0

Routing and integrations

• Advisor Gate lets an executor model work with a stronger advisor for review, approval, and redo decisions, with session review budgets and stall checkpoints.
• Composite routing combines an LLM classifier with Stage Router. Auto presets provide capable and efficient Stage configurations.
• The new switchyard-runner crate loads TOML deployments and runs routing in-process. NeMo Relay and LiteLLM integrations use the native routing path.
• Sub-agent routing, per-target system prompts and reasoning effort, and per-client request deadlines provide more control over agent traffic.
• DeepSWE v1.1 and NeMo Gym/LiteLLM examples document reproducible evaluation setups. Their results apply to the documented configurations, not every deployment.

Library and Python APIs

• Python requires 3.10 or newer; the core package has no runtime dependencies.
• Python streaming exposes Step.CallModel and Step.Done, with response dictionaries and async iterators.
• Routing outcomes report ordered selected_model_ids, bounded decision metadata, and evidence through Rust, Python, logs, and OpenTelemetry.
• The decision endpoint supports routing without executing the selected answer model.

Compatibility and correctness

• Provider translation preserves tool allowlists, parallel-call limits, native tool history, reasoning signatures, citations, and terminal Responses output. Unsupported cross-format native output fails explicitly.
• Streaming fixes retain late response IDs, fragmented tool names, empty tool inputs, and tool-use stop reasons.
• Responses continuations return to the provider that holds their state. Cross-format continuations retain conversation history in process memory; records are lost on restart.
• Media and tool-result translation preserves supported image, audio, document, and file payloads.
• Credential redaction and header handling protect request trust boundaries. Failed or abandoned streams count as errors, and usage reporting preserves translated cache-write and thinking tokens.

Migration and removals

• The Python switchyard command, switchyard serve, coding-agent wrappers, YAML route bundles, legacy server stack, and compatibility bindings are removed. Connect clients to the standalone switchyard-server with native TOML configuration.
• Package extras [server], [all], [cli], [tracing], and [affinity-redis] are removed. The Python package declares no optional dependency extras.
• classify_trigger replaces session_affinity; use classify_trigger = "new_session" for the previous session-affinity behavior.
• Rust and Python embedding callers must adopt the updated step and routing-outcome APIs. Rust struct literals may also need the new Response.upstream_headers and Metadata.subagent_identity_unsupported fields.

Install

After the tag-triggered package publishing workflow completes, install the Python bindings with:

:::bash
pip install nemo-switchyard==0.3.0

Install the standalone server separately:

:::bash
cargo install --locked switchyard-server --version 0.3.0

See the getting-started guide for configuration and source builds.

Limitations

The standalone server remains a demo/evaluation component. Release validation covers Ubuntu 24.04 on Linux x86_64; other platforms are outside that scope.

The Relay integration requires >=0.8.0, <1.0.0. Relay 0.8.x and 0.9.0 have a documented native-plugin upstream-error propagation issue. Plugin bundles are published separately; see the Relay integration guide.

Prefill Router and the LiteLLM example remain experimental. Prefill Router does not include a supported checkpoint, exporter, or model assets. See the routing overview and LiteLLM guide.