Menu

#122 Add Support for llama.cpp Server Mode

open
nobody
None
2025-11-25
2025-11-25
Anonymous
No

Originally created by: haiphucnguyen

Summary

Add a provider that communicates with the llama.cpp HTTP server (llama-server). This server exposes a simple API for running local LLM inference. Supporting it would allow Askimo to connect directly to standalone llama.cpp instances on macOS, Linux, Windows, and ARM devices.

Proposed Implementation

  • Add a new provider: LlamaCppServerProvider.

  • Default base URL: http://localhost:8080.

  • Allow user to override the endpoint.

Discussion


Log in to post a comment.