Open Source C Large Language Models (LLM)

Run, create, and share large language models (LLMs). Get up and running with large language models, locally. Run Llama 2 and other models on macOS. Customize and create your own.

The llama.cpp project enables the inference of Meta's LLaMA model (and other models) in pure C/C++ without requiring a Python runtime. It is designed for efficient and fast model execution, offering easy integration for applications needing LLM-based capabilities. The repository focuses on providing a highly optimized and portable implementation for running large language models directly within C/C++ environments.

Run a fast ChatGPT-like model locally on your device. This combines the LLaMA foundation model with an open reproduction of Stanford Alpaca a fine-tuning of the base model to obey instructions (akin to the RLHF used to train ChatGPT) and a set of modifications to llama.cpp to add a chat interface. Download the zip file corresponding to your operating system from the latest release. The weights are based on the published fine-tunes from alpaca-lora, converted back into a PyTorch checkpoint with a modified script and then quantized with llama.cpp the regular way.

llama2.c is a minimalist, end-to-end LLM toolkit that lets you train a Llama-2–style model in PyTorch and run inference with a single ~700-line C program (run.c). The project emphasizes simplicity and education: the Llama-2 architecture is hard-coded, there are no external C dependencies, and you can see the full forward pass plainly in C. Despite the tiny footprint, it’s “full-stack”: you can train small models (e.g., 15M/42M/110M params on TinyStories) and then sample tokens directly from the C runtime at interactive speeds on a laptop. You can also export and run Meta’s Llama-2 models (currently practical up to 7B due to fp32 inference and memory limits), plus try chat/Code Llama variants with proper tokenizers. A quantized int8 path (runq.c) reduces checkpoint size (e.g., 26GB→6.7GB for 7B) and speeds up inference (e.g., ~3× vs fp32 in author’s notes), with modest quality tradeoffs.

Python bindings for the Transformer models implemented in C/C++ using GGML library.