Search Results for "nvme"

Colibrì is an open-source inference engine designed to run frontier-scale mixture-of-experts (MoE) models on consumer and heterogeneous hardware. It treats VRAM, system RAM, and NVMe storage as a unified memory hierarchy, dynamically placing and streaming model weights where they can be accessed most efficiently. This architecture enables users to run models ranging from 7 billion to 2.8 trillion parameters without requiring the entire model to fit in expensive GPU memory. Written primarily in pure C, Colibrì has no core engine dependencies and can operate without a GPU, although CUDA, Metal, Vulkan, and other acceleration options can improve performance. ...