BaseRT 0.2.2

Muse Glimmer, and native GGUF k-quants. BaseRT now runs Muse Glimmer 30B — text and vision — on Apple Silicon, and reads GGUF k-quantized weights (Q4_K / Q5_K / Q6_K) directly, without a dequantize-and-repack step. Short prompts prefill 1.4–1.55× faster on M5.

Added

Muse Glimmer

• Muse Glimmer 30B, text and vision, including the perception tower — so the model answers about images, not just text.
• Pre-converted bundles at basecompute/Muse-Glimmer-30B:

bash basert pull basecompute/Muse-Glimmer-30B # 21 GB, recommended basert pull basecompute/Muse-Glimmer-30B:q4k-17gb # 18 GB, smaller

Both are the model's own published k-quants, repackaged — the super-block bytes are copied through unchanged, so the weights are bit-identical to the upstream GGUFs rather than a re-quantization of them. Both carry the vision tower; there is no separate projector file to manage.

• Images use the model's own <|patch|> placeholder:

bash basert complete basecompute/Muse-Glimmer-30B --chat \ --image photo.png --prompt "<|patch|>Describe this image."

Native GGUF k-quants

• Q4_K / Q5_K / Q6_K run natively — prefill, decode and embedding lookup all read the quantized weights directly. Q2_K and Q3_K are supported as well.
• --kquant-passthrough copies GGUF super-blocks into a .base bundle verbatim, so the weights stay bit-identical to the source. Re-quantizing an already-quantized model compounds its error; copying the blocks through introduces none, so this is not a quality trade and needs no override flag.
• --mmproj folds a companion mmproj-*.gguf perception tower into the same bundle. Without it a bundle converted from GGUF is text-only, because the tower ships as a separate file. A bundle built this way is interchangeable with one converted from the original safetensors.

Changed

• Faster short-prompt prefill on M5 — 1.4–1.55× measured on M5 Max. Short prompts previously fell back to a slower path than the hardware supports; they now use the same accelerated one longer prompts already did. Long prefill is unchanged, because it was already there.
• Faster k-quant prefill — 3.4×, from running k-quantized weights on the accelerated path rather than the general one.
• Wider accelerated coverage for the 5-, 2- and 3-bit quantizations, alongside the 4-, 6- and 8-bit ones.
• Decode speed is unchanged. At these model shapes it is limited by memory bandwidth rather than compute, so the changes above do not move it, and the decode-side ideas tried for this release measured slower and were dropped.

Fixed

• M3 Ultra was identified as a base M3, so it was tuned for a far smaller chip than it is. Reachable only when the system does not report a GPU core count, but it would have affected every future per-chip decision.
• Language bindings could corrupt memory. The model config is returned by value, and the Python, Rust and Node definitions of that struct had fallen behind the C header — a mirror even four bytes short lets the engine write past the caller's allocation. All four bindings are back in step, and a test now holds them there.
• Models converted from GGUF ignored the end-of-turn token and kept generating past the end of a reply.
• baseRT_format_chat is available without a chat-capable build — it no longer depends on the chat layer being present.

Notes

• k-quant support is Apple Silicon (Metal) only in this release.
• Folding a perception tower from a companion mmproj GGUF is exact for still images, which is what these models take today. Such a file does not carry everything a video-frame path would need.