Search Results for "cpu disk memory monitor"

Kimi K3 in C is a portable C99 inference engine built to run the 2.78-trillion-parameter Kimi K3 model on CPUs without BLAS, machine-learning frameworks, or GPUs. It demonstrates inference from a roughly 1.56 TB checkpoint with measured memory use as low as 8.24 GB. The runtime streams model trunk layers and routed experts from disk instead of keeping all weights resident in memory. Memory presets balance pinned layers and an expert LRU cache for laptops, desktops, workstations, and servers....

whisper.cpp is a lightweight, C/C++ reimplementation of OpenAI’s Whisper automatic speech recognition (ASR) model—designed for efficient, standalone transcription without external dependencies. The entire high-level implementation of the model is contained in whisper.h and whisper.cpp. The rest of the code is part of the ggml machine learning library. The command downloads the base.en model converted to custom ggml format and runs the inference on all .wav samples in the folder samples....

WASTE is an embeddable C inference engine for running extremely large mixture-of-experts models when the weights exceed available RAM. It keeps the shared model trunk in memory and streams only the experts selected for each token from fast NVMe storage. A bounded cache reuses recently needed experts, while lookahead routing begins disk reads before the next layer requires them. Its main target is the full 2.78-trillion-parameter Kimi K3 model, including multimodal image input. The engine has no third-party runtime dependency on its CPU inference path and exposes both a CLI and C library. ...