TurboFieldfare Files

TurboFieldfare 0.2 adds an experimental OpenAI-compatible server for using Gemma 4 26B-A4B with local API clients on Apple Silicon.

OpenAI-compatible local server

• Serves GET /health, GET /v1/models, and POST /v1/chat/completions.
• Returns standard JSON responses or streams tokens with Server-Sent Events.
• Accepts system, developer, user, assistant, and tool messages for multi-turn conversations.
• Reuses one verified KV prefix for compatible conversation continuations and reports reused tokens through usage.prompt_tokens_details.cached_tokens.
• Returns OpenAI-style function calls for the client to authorize and execute.
• Queues requests with bounded memory, recovers cleanly from cancellation, and handles SIGINT and SIGTERM with graceful teardown.

The server binds only to 127.0.0.1. It has no authentication or TLS, so do not expose it through a proxy or tunnel. Tool execution remains under the client's permission policy.

Start the server

:::bash
swift build -c release --product TurboFieldfareServer
.build/release/TurboFieldfareServer \
  --model scratch/gemma4.gturbo \
  --port 8080 \
  --max-context 16384

Connect OpenAI-compatible clients to http://127.0.0.1:8080/v1. See the server guide for OpenAI Python and OpenCode examples, prompt caching, tool loops, and the supported request fields.

Current scope

The server supports one local model, one choice, and the Chat Completions API. It does not support the Responses API, legacy Completions, embeddings, multimodal input, structured output, batching, log probabilities, or remote model switching. The Mac app UI remains single-turn.

Full changelog: https://github.com/drumih/turbo-fieldfare/compare/0.1.1...0.2

Implemented in #13.