Search Results for "ternary"

Bonsai 27B is a repository for downloading, configuring, and running PrismML’s highly compressed Bonsai language models on local hardware. It supports the 1-bit Bonsai and higher-quality Ternary-Bonsai families in 1.7B, 4B, 8B, and 27B sizes. The models can run on macOS, Linux, and Windows through CPU, Metal, CUDA, Vulkan, ROCm, llama.cpp, or MLX backends. Its 27B models process text, images, screenshots, and PDFs while supporting reasoning and long-context conversations. They also provide OpenAI-compatible tool calling and optional MCP server integration for agentic workflows. ...

bitnet.cpp is the official open-source inference framework and ecosystem designed to enable ultra-efficient execution of 1-bit large language models (LLMs), which quantize most model parameters to ternary values (-1, 0, +1) while maintaining competitive performance with full-precision counterparts. At its core is bitnet.cpp, a highly optimized C++ backend that supports fast, low-memory inference on both CPUs and GPUs, enabling models such as BitNet b1.58 to run without requiring enormous compute infrastructure. The project’s focus on extreme quantization dramatically reduces memory footprint and energy consumption compared with traditional 16-bit or 32-bit LLMs, making it practical to deploy advanced language understanding and generation models on everyday machines. ...

MatMul-Free LM is an experimental implementation of a large language model architecture designed to eliminate traditional matrix multiplication operations used in transformer networks. Since matrix multiplication is one of the most computationally expensive components of modern language models, the project explores alternative computational strategies that reduce hardware requirements while maintaining comparable performance. The architecture relies on quantization-aware training and...