#70 Support a fallback chain of models/providers when the primary is unavailable

[Anonymous]

Originally created by: sosidudku1

Summary

There is no fallback model today. If the configured provider or model is unreachable (endpoint down, rate limited, out of credit), the run stops and the user has to reconfigure manually.

User quote (Discord)

"it has no fallback model"

"also all fallback models"

The request is explicitly a chain, not a single backup: try the primary, then the second, then the third.

Proposal

Let the user define an ordered list of providers/models. On a retryable failure (connection error, 429, 5xx, model-not-found), the agent moves to the next entry and reports the switch in the UI rather than failing the turn.

Open questions:

• Should a local model be the last resort by default, so the agent always has something to fall back to?
• Should the chain reset to the primary on the next turn, or stay on the working provider for the session?
• Which error classes should trigger a switch vs. a plain retry? The failure taxonomy in src/llm/reliability/ already separates transport, model and grammar failures, which is the natural place to draw that line.

Why it matters

This is a local-first agent that users increasingly point at cloud endpoints. Cloud endpoints fail more often than local ones, and a mid-task failure with no fallback wastes the whole run.

Related

[#66] (health probe rejects non-llama.cpp runners) is a different failure mode but overlaps in outcome: the user ends up with a configured endpoint the agent will not use.

[Anonymous]

Ticket changed by: sosidudku1

• status: open --> closed