Search Results for "ram linux"

Colibri is a compact inference engine designed to run the 744-billion-parameter GLM-5.2 mixture-of-experts model on consumer hardware. It keeps the dense portion of the quantized model in memory while streaming routed experts from a large disk-based store as they are needed. The runtime is implemented in pure C, requires no Python or BLAS during inference, and can operate without a GPU. Compressed attention caches, expert caching, optional hot tiers, and speculative decoding reduce memory...

PicoLM is an open-source inference framework designed to run large language models on extremely constrained hardware environments such as inexpensive single-board computers and embedded systems. The project focuses on enabling efficient local inference by optimizing memory usage, computation, and system dependencies so that relatively large models can operate on devices with minimal RAM. It is written primarily in C and designed with a minimalist architecture that removes unnecessary...

WASTE is an embeddable C inference engine for running extremely large mixture-of-experts models when the weights exceed available RAM. It keeps the shared model trunk in memory and streams only the experts selected for each token from fast NVMe storage. A bounded cache reuses recently needed experts, while lookahead routing begins disk reads before the next layer requires them. Its main target is the full 2.78-trillion-parameter Kimi K3 model, including multimodal image input. The engine has...

Run a fast ChatGPT-like model locally on your device. This combines the LLaMA foundation model with an open reproduction of Stanford Alpaca a fine-tuning of the base model to obey instructions (akin to the RLHF used to train ChatGPT) and a set of modifications to llama.cpp to add a chat interface. Download the zip file corresponding to your operating system from the latest release. The weights are based on the published fine-tunes from alpaca-lora, converted back into a PyTorch checkpoint...

This is YOLO-v3 and v2 for Windows and Linux. YOLO (You only look once) is a state-of-the-art, real-time object detection system of Darknet, an open source neural network framework in C. YOLO is extremely fast and accurate. It uses a single neural network to divide a full image into regions, and then predicts bounding boxes and probabilities for each region. This project is a fork of the original Darknet project.

[ Please read https://sourceforge.net/p/darkbot/news/2014/01/darkbots-revitalization/ ] Darkbot is a portable IRC chat robot written in the C language that can be taught responses to user inquiries, and even have conversations with them. Darkbot was originally created by Jason Hamilton as an aid for help channels on Intenet Relay Chat.