BaseRT 0.2.1

Speech-to-text. BaseRT now runs the full OpenAI Whisper family — all twelve variants from tiny to large-v3-turbo, including the English-only builds — as .base bundles on both Apple Silicon (Metal) and NVIDIA GB10 (CUDA), served through an OpenAI-compatible transcription API with live streaming.

Added

Whisper speech-to-text

• All twelve Whisper variants convert to .base and run end-to-end: tiny, base, small, medium, large, large-v2, large-v3, large-v3-turbo, and the .en English-only builds. Special-token IDs are read from each model's own tokenizer rather than inferred from vocabulary size, so large-v3 and large-v3-turbo (100 languages, 128 mel bins) map correctly.
• Quantized bundles — whisper-q8 and whisper-q4 profiles quantize the linear projections while the convolutional front end, positional and token embeddings, norms, and biases stay f16. Q8 meets the same word-error-rate parity gates as f16 against the reference implementation.
• CUDA support — Whisper runs on NVIDIA GB10 with transcripts byte-identical to the Metal path.
• POST /v1/audio/transcriptions and POST /v1/audio/translations, OpenAI-compatible, including verbose_json with per-segment avg_logprob, no_speech_prob, compression_ratio and temperature, plus the detected language.
• Streaming transcription — stream: true emits segments from the decode loop as each window completes, in the documented transcript.text.delta / transcript.text.done wire format, so official OpenAI SDKs parse it directly. Time-to-first-event is one window's decode rather than the whole file.
• Decode parity with the reference implementation — task=translate, language auto-detection (language: "auto"), initial_prompt with condition-on-previous-text, and the temperature-fallback ladder. Word-level timestamps remain future work.

C API

• baseRT_load_model_ex with BaseRTLoadOptions — per-load KV width, paged-KV, batch size, prefix cache, prefill chunk and paged-weight mode, replacing the process-wide setters. The options struct is ABI-guarded by struct_size.
• Transcription accessors: baseRT_set_task, baseRT_set_initial_prompt, baseRT_set_condition_on_previous_text, per-segment results, detected language, and source-audio duration.
• Batched logits: baseRT_batch_logits_stride, baseRT_argmax_logits_row, baseRT_sample_logits_row, baseRT_logits_row_logprobs, baseRT_mask_logits_row.
• baseRT_capabilities reports what the running backend supports.

All of the above are bound in the Python, Node, Swift and Rust bindings.

Changed

• Faster shared-prefix serving — concurrent requests that share a prompt prefix now detect and reuse it through a cascade dispatch path.
• Faster MoE decode on CUDA — fused SiLU gate/up for 4-bit experts.
• Metal GEMM accuracy — f32 accumulation on the paths where f16 accumulation was measurably lossy.

Fixed

• context_length_exceeded reported a capped token count. The number came from a fixed tokenizer buffer, so any over-long prompt reported the same figure no matter how far over it was — a client could not trim to a number that was itself the ceiling. It now reports the real count.
• Image requests under-reported usage and never returned finish_reason: "length": the first token produced during image prefill was streamed to the client but not counted, so a truncated answer looked complete.
• Gemma 4 audio produced unusable embeddings. The channel-last convolution indexed its weights as [out, kh, kw, in] where the bundle stores [out, in, kh, kw]. For a single input channel the two are identical, so the first subsampling convolution was correct and masked the fault while the second convolved with transposed weights.
• Repetition penalty silently did nothing on some Apple GPUs — a sampling kernel read a scratch buffer that was never bound, which is undefined behaviour and therefore hardware-dependent.
• A KV-cache shift kernel had a data race — an in-place shift whose source and destination overlap, spread across threads, so results depended on scheduling order.
• Rate limiting could grow without bound. Idle buckets expired on time alone, so a source rotating addresses added an entry per request and nothing became evictable within the window. Bucket count is now capped.
• Absolute --media-dir paths are honoured as written and must resolve inside the configured root, instead of being re-anchored under it — which made a legitimate absolute path fail as an image-decoding error.
• Rejections now appear in the access log, so an operator can tell a refused request from a hung one.
• Second images and audio parts in a chat request are rejected with a clear 400 instead of a 500 or, for audio, a confident answer about audio the model never received.
• Image-decoding failures no longer echo the server's resolved filesystem path back to the caller.
• basert inspect reported 0 tensors for working vision and audio towers: it counted name prefixes that converted bundles do not use.
• Chat-template cache eviction and a Qwen3 tool round-trip fault.

Notes

• BASERT_VERSION now advertises the patch component, and the release pipeline verifies the tag, the compiled-in version, and every package manifest agree before publishing.