#102 feat(llm): provider fallback chain with a TUI editor (Closes #70)

[Anonymous]

Originally created by: sosidudku1

Problem

When the active LLM provider is unavailable (429, 5xx, network), a turn fails even if a second provider is configured. The two existing retry layers recover a request on the same provider; nothing switches to a different one.

Change

A cross-provider circuit breaker (src/llm/fallback/) layered above the per-provider retry budget, wrapping the llmComplete / llmCompleteStream seams. It fails over through an ordered list of configured provider ids, primary first. Timer-free: every decision is computed lazily from the wall clock at each turn boundary (no setInterval). The seams are built by src/runtime/llm-fallback-seam.ts (createFallbackCompleter / createFallbackStreamer) and wired in bootstrap. Plus a Fallback pane on the LLM tab to view and edit the chain.

Config (llm.fallback)

"fallback": {
  "chain": ["openrouter-gpt", "groq-llama"],  // ordered provider ids; each must be configured
  "appendLocal": true,                          // default: append the llama-server provider to the tail
  "failureThreshold": 3,                        // consecutive non-immediate failures before switching
  "cooldownMs": [30000, 60000, 300000],         // escalating ladder; last entry caps
  "probeThrottleMs": 300000,                    // min gap between primary probes
  "failureWindowMs": 86400000                   // no-error window that resets the counter + ladder
}

chain defaults to [activeTextProvider]; the active text provider is always hoisted to the head, so a hot-swap re-primes the chain. Unknown ids rejected at parse time.

UI (Fallback pane)

Fourth pane on the LLM tab (left/right after Local/Cloud/External). Lists the effective chain, active head tagged, appended-local tagged. Keys: j/k cursor, </> reorder, a add link, d remove, l toggle appendLocal. Edits persist to config.json via the sole writer setFallbackChainInConfig (re-validated with parseLlmFallbackConfig; hand-set timing knobs preserved; the synthesised local link is never written into the stored chain), effective on the next turn. The pane re-mirrors on a provider hot-swap so the head follows the active provider. Live status mirrors the last provider_switched event only; no invented cooldown countdown, because the runtime breaker instance is not exposed to the TUI.

Reset policy (per provider id)

shouldAdvance(err) decides which failures advance (transport/model yes; grammar/tool/cancelled never). Immediate signals (429/408/5xx/network-null) switch on the first failure; others at failureThreshold. Sticky after switching (dead primary not re-picked every turn). Escalating cooldown 30s to 60s to 300s (cap), reset after failureWindowMs with no failure. When on an override and the primary's cooldown elapsed plus probe throttle passed, one turn probes the primary; success clears the override and emits a one-shot "switched back". At most one provider_switched notice per state transition.

Testing

Engine: circuit-breaker transitions, should-advance taxonomy, sticky/probe, cooldown ladder, appendLocal, cross-transport request+response through the real seam factories (llm-fallback-seam.test.ts, where deleting either servedTransport stamp turns it red) and end-to-end through AgentLoop + step-executor (unary and streaming). UI: chain view/order, clamped edits, config persistence including the appendLocal round-trip (no local-link doubling), key routing, provider_switched mirroring, re-mirror on provider hot-swap, empty states. tsc clean; full suite green apart from the same pre-existing floaters as origin/main.

Closes [#70].

[Anonymous]

Ticket changed by: sosidudku1

• status: open --> closed