#66 Feature request: Please add support to other LLM runners

[Anonymous]

Originally created by: Gamer-Hacker123

I have tried to use KoboldCpp(an LLM runner just like Ollama and Llama.cpp), but the verification returned a bad health status (a non working endpoint) although it was working perfectly fine in my end.

In other words: Atomic Agent supports only native LLama.cpp endpoint, and other endpoints are not working in it even if they were running.

Important:

The port of KoboldCpp endpoint is : 5001, or exactly: localhost:5001 .
Hope this is understandable. And hope this project grows in the best way you want 💖

[Anonymous]

Originally posted by: sosidudku1

Thanks for the detailed report and the screenshot, this is a real bug on our side. The external-server check probes /health, which is llama.cpp specific: KoboldCpp does not serve that path, so a perfectly working server gets marked unreachable. The fix is to fall back to an OpenAI-compatible probe (/v1/models) when /health is absent, which would cover KoboldCpp and most other runners exposing an OpenAI-compatible API. Keeping this open as a confirmed bug.

Meanwhile, if KoboldCpp exposes /v1 you can point Atomic at it as an openai-compatible cloud provider (base URL http://localhost:5001) and it should work today; the local-server route is the one that trips on the probe.

[Anonymous]

Ticket changed by: Ooooze

• status: open --> closed