Showing 25 open source projects for "llama.cpp python"

View related business solutions
  • Fully Managed MySQL, PostgreSQL, and SQL Server Icon
    Fully Managed MySQL, PostgreSQL, and SQL Server

    Automatic backups, patching, replication, and failover. Focus on your app, not your database.

    Cloud SQL handles your database ops end to end, so you can focus on your app.
    Start Free
  • Secure File Transfer for Windows with Cerberus by Redwood Icon
    Secure File Transfer for Windows with Cerberus by Redwood

    Protect and share files over FTP/S, SFTP, HTTPS and SCP with the #1 rated Windows file transfer server.

    Cerberus supports unlimited users and connections on a single IP, with built-in encryption, 2FA, and a browser-based web client — all deployable in under 15 minutes with a 25-day free trial.
    Download for Free Today
  • 1
    llama.cpp

    llama.cpp

    Port of Facebook's LLaMA model in C/C++

    The llama.cpp project enables the inference of Meta's LLaMA model (and other models) in pure C/C++ without requiring a Python runtime. It is designed for efficient and fast model execution, offering easy integration for applications needing LLM-based capabilities. The repository focuses on providing a highly optimized and portable implementation for running large language models directly within C/C++ environments.
    5.0 out of 5 stars 1 Review
    Downloads: 117 This Week
    Last Update:
    See Project
  • 2
    llama.cpp Python Bindings

    llama.cpp Python Bindings

    Python bindings for llama.cpp

    llama-cpp-python provides Python bindings for llama.cpp, enabling the integration of LLaMA (Large Language Model Meta AI) language models into Python applications. This facilitates the use of LLaMA's capabilities in natural language processing tasks within Python environments.
    Downloads: 13 This Week
    Last Update:
    See Project
  • 3
    GPT4All

    GPT4All

    Run Local LLMs on Any Device. Open-source

    ...The software provides a simple, user-friendly application that can be downloaded and run on various platforms, including Windows, macOS, and Ubuntu, without requiring specialized hardware. It integrates with the llama.cpp implementation and supports multiple LLMs, allowing users to interact with AI models privately. This project also supports Python integrations for easy automation and customization. GPT4All is ideal for individuals and businesses seeking private, offline access to powerful LLMs.
    5.0 out of 5 stars 1 Review
    Downloads: 86 This Week
    Last Update:
    See Project
  • 4
    OpenJarvis

    OpenJarvis

    Personal AI, On Personal Devices

    OpenJarvis is an open-source framework designed to build personal AI agents that run primarily on local devices rather than relying on cloud infrastructure. Developed as part of the Intelligence Per Watt research initiative, it focuses on improving the efficiency and practicality of on-device AI systems. The framework provides shared primitives for building local-first agents, along with evaluation tools that measure performance using metrics such as energy consumption, latency, cost, and...
    Downloads: 290 This Week
    Last Update:
    See Project
  • Custom VMs From 1 to 96 vCPUs With 99.95% Uptime Icon
    Custom VMs From 1 to 96 vCPUs With 99.95% Uptime

    General-purpose, compute-optimized, or GPU/TPU-accelerated. Built to your exact specs.

    Live migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
    Start Free
  • 5
    OuteTTS

    OuteTTS

    Interface for OuteTTS models

    ...It provides a high-level Interface API that wraps model configuration, speaker handling, and audio generation so you can focus on integrating speech into your application rather than wiring up low-level engines. The project supports multiple backends including llama.cpp (Python bindings and server), Hugging Face Transformers, ExLlamaV2, VLLM and a JavaScript interface via Transformers.js, allowing it to run on CPUs, NVIDIA CUDA GPUs, AMD ROCm, Vulkan-capable GPUs, and Apple Metal. It also includes a notion of speaker profiles: you can create a speaker from a short audio sample, save it as JSON, and reuse it for consistent voice identity across generations and sessions. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 6
    mac code

    mac code

    Claude Code, but it runs on your Mac for free

    mac code is a local AI coding agent designed to run large language models directly on Apple Silicon machines without relying on cloud services, effectively transforming a Mac into a self-contained AI development environment. The project focuses on enabling models that traditionally exceed available RAM to run efficiently by streaming model weights from SSD storage, thereby overcoming hardware limitations through innovative memory management techniques. It operates as a CLI-based assistant...
    Downloads: 2 This Week
    Last Update:
    See Project
  • 7
    Harbor LLM

    Harbor LLM

    Run a full local LLM stack with one command using Docker

    Harbor is an open source, containerized toolkit designed to simplify running local large language model (LLM) environments. It combines a CLI and companion app to launch backends, frontends, and supporting services with minimal setup. With a single command, users can start preconfigured tools like Ollama and Open WebUI, enabling chat, workflows, and integrations immediately. Harbor supports multiple inference engines, including llama.cpp and vLLM, and connects them seamlessly to user...
    Downloads: 2 This Week
    Last Update:
    See Project
  • 8
    CoPaw

    CoPaw

    Your Personal AI Assistant; easy to install, deploy on local or coud

    CoPaw is a personal AI assistant designed to run on your own machine or in the cloud, giving you full control over memory, models, and data. Built by the AgentScope team, it connects to multiple chat platforms—including DingTalk, Feishu, QQ, Discord, iMessage, and more—through a single unified assistant. CoPaw supports both cloud-based LLM providers and fully local models such as llama.cpp, MLX, and Ollama, allowing you to operate without API keys if preferred. It includes a browser-based...
    5.0 out of 5 stars 1 Review
    Downloads: 5 This Week
    Last Update:
    See Project
  • 9
    Bonsai 27B

    Bonsai 27B

    Run Bonsai (1-bit) and Ternary-Bonsai language models locally

    Bonsai 27B is a repository for downloading, configuring, and running PrismML’s highly compressed Bonsai language models on local hardware. It supports the 1-bit Bonsai and higher-quality Ternary-Bonsai families in 1.7B, 4B, 8B, and 27B sizes. The models can run on macOS, Linux, and Windows through CPU, Metal, CUDA, Vulkan, ROCm, llama.cpp, or MLX backends. Its 27B models process text, images, screenshots, and PDFs while supporting reasoning and long-context conversations. They also provide...
    Downloads: 29 This Week
    Last Update:
    See Project
  • $300 Free Credits to Build on Google Cloud Icon
    $300 Free Credits to Build on Google Cloud

    New customers can spin up VMs, build with AI, and query data at no cost.

    Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
    Start Free
  • 10
    Outlines

    Outlines

    Structured Outputs

    Outlines is a Python library for making language models generate predictable, structured outputs. Instead of repairing malformed responses afterward, it constrains generation to match a requested type or format. Developers can require predefined choices, primitive Python types, Pydantic models, JSON schemas, regular expressions, function signatures, or custom grammars.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 11
    Hy-MT2
    Hy-MT2 is a family of fast-thinking multilingual translation models built for complex real-world translation scenarios. It includes 1.8B, 7B, and 30B-A3B model sizes, giving users options for lightweight deployment, stronger general performance, or MoE-based capacity. The models support translation across 33 languages and are designed to follow detailed translation instructions in multiple languages. They can handle tasks involving terminology, style, personalization, delimiters, structured...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 12
    Qwen3

    Qwen3

    Qwen3 is the large language model series developed by Qwen team

    Qwen3 is a cutting-edge large language model (LLM) series developed by the Qwen team at Alibaba Cloud. The latest updated version, Qwen3-235B-A22B-Instruct-2507, features significant improvements in instruction-following, reasoning, knowledge coverage, and long-context understanding up to 256K tokens. It delivers higher quality and more helpful text generation across multiple languages and domains, including mathematics, coding, science, and tool usage. Various quantized versions,...
    5.0 out of 5 stars 1 Review
    Downloads: 15 This Week
    Last Update:
    See Project
  • 13
    Text Generation Web UI

    Text Generation Web UI

    Oobabooga - The definitive Web UI for local AI, with powerful features

    A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA. Dropdown menu for switching between models. Notebook mode that resembles OpenAI's playground. Chat mode for conversation and role playing. Instruct mode compatible with Alpaca and Open Assistant formats. Nice HTML output for GPT-4chan. Markdown output for GALACTICA, including LaTeX rendering. Custom chat characters. Advanced chat features (send images, get audio responses with TTS)....
    Downloads: 13 This Week
    Last Update:
    See Project
  • 14
    SemIf

    SemIf

    Semantic ifs from open models, on a 3090 at home

    SemIf, formerly OpenJev, is an independent research project for making small semantic decisions with open language models. Instead of generating text and parsing it afterward, it reads probabilities directly from declared answer options. Each request supplies state, a runtime-defined question, and typed choices, allowing the model to behave like a semantic conditional. Shared-state execution can reuse a long context across many related decisions to reduce repeated computation. Backends...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 15
    MiniCPM5

    MiniCPM5

    MiniCPM5: SOTA on-device LLMs, small yet powerful

    MiniCPM is an open-source family of compact large language models developed by OpenBMB for efficient AI inference on personal devices and resource-constrained hardware. Its latest MiniCPM5 series includes 1B and 2B-class models optimized for local deployment. The models support text generation, coding, mathematical reasoning, and autonomous agent workflows. MiniCPM5-2B uses a Transformer architecture with grouped-query attention and a 131,072-token context window. Earlier releases explore...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16
    GPUStack

    GPUStack

    Performance-optimized AI inference on your GPUs

    GPUStack is an open-source GPU cluster management platform designed to simplify the deployment and operation of artificial intelligence models across heterogeneous hardware environments. The system aggregates GPU resources from multiple machines into a unified cluster so developers and administrators can run large language models and other AI workloads efficiently across distributed infrastructure. Instead of requiring complex orchestration systems such as Kubernetes, GPUStack provides a...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 17
    Orpheus TTS

    Orpheus TTS

    Towards Human-Sounding Speech

    Orpheus TTS is a state-of-the-art open-source text-to-speech system built on a Llama-3B backbone, treating speech synthesis as a large language model problem instead of a traditional TTS pipeline. It is designed to produce human-like speech with natural intonation, emotion, and rhythm, targeting quality comparable to or better than many closed-source systems. The project ships both pretrained and finetuned English models, as well as a family of multilingual models released as a research...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 18
    DocsGPT

    DocsGPT

    Private AI agents, assistants and search over your own documents

    DocsGPT is an open-source (MIT) platform for private AI agents, assistants and enterprise search over your own documents, built by Arc53. It ingests PDFs, Office files, scanned pages (OCR), web pages and repositories, syncs Google Drive, SharePoint, Confluence, GitHub and S3, and answers with citations to the source passage. Agents call tools, APIs and MCP servers with per-action approval; a visual builder chains AI steps, conditions and code into workflows; agents can produce DOCX, XLSX,...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 19
    Row-Bot

    Row-Bot

    A local-first AI assistant that gets real work done

    Row-Bot is a free, open-source, local-first AI assistant designed to help users automate tasks, manage information, and interact with applications while maintaining control over their personal data. The platform supports multiple AI models, including ChatGPT, Claude, Grok, and locally hosted models through Ollama, LM Studio, and other compatible services, allowing users to choose their preferred AI provider. Row-Bot features persistent memory powered by a personal knowledge graph, enabling...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 20
    Tiny LLM zh

    Tiny LLM zh

    Implement a small-parameter Chinese large language model from zero

    Tiny LLM zh is an open-source educational project that demonstrates how to build and train compact Chinese large language models from scratch. It covers the complete language model development process, including tokenization, pretraining, supervised fine-tuning, and human preference alignment. The architecture uses Transformer components such as RMSNorm, rotary positional embeddings, and multi-head attention. It supports several model sizes, ranging from approximately 16 million to 440...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    KoboldCpp

    KoboldCpp

    Run GGUF models easily with a UI or API. One File. Zero Install.

    KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models, inspired by the original KoboldAI. It's a single self-contained distributable that builds off llama.cpp and adds many additional powerful features.
    Leader badge
    Downloads: 384 This Week
    Last Update:
    See Project
  • 22
    llama2.c

    llama2.c

    Inference Llama 2 in one file of pure C

    llama2.c is a minimalist implementation of the Llama 2 language model architecture designed to run entirely in pure C. Created by Andrej Karpathy, this project offers an educational and lightweight framework for performing inference on small Llama 2 models without external dependencies. It provides a full training and inference pipeline: models can be trained in PyTorch and later executed using a concise 700-line C program (run.c). While it can technically load Meta’s official Llama 2...
    Downloads: 6 This Week
    Last Update:
    See Project
  • 23
    llama2-webui

    llama2-webui

    Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere

    Running Llama 2 with gradio web UI on GPU or CPU from anywhere (Linux/Windows/Mac).
    Downloads: 0 This Week
    Last Update:
    See Project
  • 24
    Chinese-LLaMA-Alpaca-2 v2.0

    Chinese-LLaMA-Alpaca-2 v2.0

    Chinese LLaMA & Alpaca large language model + local CPU/GPU training

    This project has open-sourced the Chinese LLaMA model and the Alpaca large model with instruction fine-tuning to further promote the open research of large models in the Chinese NLP community. Based on the original LLaMA , these models expand the Chinese vocabulary and use Chinese data for secondary pre-training, which further improves the basic semantic understanding of Chinese. At the same time, the Chinese Alpaca model further uses Chinese instruction data for fine-tuning, which...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 25
    Mellum-4b-base

    Mellum-4b-base

    JetBrains’ 4B parameter code model for completions

    ...With a context window of 8,192 tokens, it excels at code completion, fill-in-the-middle tasks, and intelligent code suggestions for professional developer tools and IDEs. The model is efficient for both cloud inference with vLLM and local deployment using llama.cpp or Ollama, thanks to its bf16 precision and AMP training. While the base model is not fine-tuned for downstream tasks, it is designed to be easily adapted through supervised fine-tuning (SFT) or reinforcement learning (RL). Benchmarks on RepoBench, SAFIM, and HumanEval demonstrate its competitive performance, with specialized fine-tuned versions for Python already showing strong improvements.
    Downloads: 0 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • Next