Search Results for "mona server mac"

...Instead of loading the entire model, it keeps the shared core and KV cache in RAM while streaming only the routed experts required for each token from SSD. This approach reduces active memory use to roughly 2 GB while the installed model occupies about 14.3 GB of storage. The project includes a native Mac application, command-line tools, a streaming installer, a Swift library, and an experimental OpenAI-compatible local server. Quantized weights, custom Metal kernels, chunked prefill, and a bounded expert cache improve efficiency. The installer downloads and repacks model ranges without staging a second full checkpoint. Its current scope is text-only Gemma inference on macOS 26 and Apple Silicon hardware.