#73 fix(llm): validate the /health body and identify OpenAI-compatible runners

[Anonymous]

Originally created by: sosidudku1

Closes [#65], closes [#66].

What actually happened in [#65]

KoboldCpp serves its web UI with HTTP 200 on every path, including /health. The external-URL probe treated any 200 as success, so it passed falsely, the chat route switched onto a server the llama.cpp client cannot drive, and the next call hung while the busy flag ate the keyboard. That is the permanent freeze the reporter described for http://localhost:5001. With /v1 appended the downstream call failed fast instead, which is why that variant unfroze after seconds. (The earlier theory in the issue thread blamed probe retries; the probe on this path already ran a single 8s attempt, the hang lived after the false pass.)

Change

1. A 200 on /health is no longer enough. The body must carry llama.cpp's JSON shape (a status string). KoboldCpp's HTML now fails the probe instead of hijacking the route.
2. A loading llama.cpp is recognized as llama.cpp. Real builds answer /health with 503 while the model loads: new builds send {"error":{"code":503,"message":"Loading model..."}} with no status field, old builds {"status":"loading model"}. Both now report kind: "llama-loading", and the save path tells the operator the server is still loading the model and to save the URL again in a minute, instead of steering a genuine llama.cpp user toward the cloud-provider flow.
3. A second probe identifies OpenAI-compatible runners. When /health answers with a 200 carrying a non-llama body or with a 404, {base}/v1/models is asked; a JSON answer with a data array marks the result kind: "openai-compat" (KoboldCpp, LM Studio, vLLM all match). The probe is skipped when nothing answered at all (connection refused, timeout) and when the server already identified itself as a loading llama.cpp, so those paths no longer spend extra seconds on a question with a known answer.
4. No retries on a deterministic answer. A 200 with a non-llama body cannot change between attempts, so the retry loop exits after the first one instead of spending the whole backoff budget on it.
5. The save path talks. probing <url> is emitted immediately (an 8s silent probe reads as a freeze), and an openai-compat answer produces an actionable message: add the server as a cloud provider, base URL included, which is the path that actually works for these runners today.

Testing

llama-server-health.test.ts (the module had no tests before): real llama /health accepted, both 503 loading shapes reported as llama-loading with the /v1/models probe skipped, KoboldCpp-style HTML 200 rejected with a clear error and no retries, the /v1/models fallback identifying an OpenAI-compatible runner, connection refused reporting unknown without the fallback probe, and retry short-circuiting on first success.

All probe consumers pass unchanged: src/llm, src/tui/llm-health, src/http, src/runtime (359 tests). tsc clean. Full suite shows the same pre-existing failures as main.

[Anonymous]

Ticket changed by: Ooooze

• status: open --> closed