Showing 33 open source projects for "llama.cpp python"

View related business solutions
  • $300 Free Credits to Build on Google Cloud Icon
    $300 Free Credits to Build on Google Cloud

    New customers can spin up VMs, build with AI, and query data at no cost.

    Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
    Start Free
  • Patching and Endpoint Management for Under $1/Endpoint Icon
    Patching and Endpoint Management for Under $1/Endpoint

    OS and third-party patching, vulnerability insights, and remote control included

    Splashtop AEM gives IT teams automated patching, monitoring, and unattended remote access for Windows and macOS. No long setup, no paying extra for key features.
    Start Free Trial
  • 1
    llama.cpp Python Bindings

    llama.cpp Python Bindings

    Python bindings for llama.cpp

    llama-cpp-python provides Python bindings for llama.cpp, enabling the integration of LLaMA (Large Language Model Meta AI) language models into Python applications. This facilitates the use of LLaMA's capabilities in natural language processing tasks within Python environments.
    Downloads: 13 This Week
    Last Update:
    See Project
  • 2
    GPT4All

    GPT4All

    Run Local LLMs on Any Device. Open-source

    ...The software provides a simple, user-friendly application that can be downloaded and run on various platforms, including Windows, macOS, and Ubuntu, without requiring specialized hardware. It integrates with the llama.cpp implementation and supports multiple LLMs, allowing users to interact with AI models privately. This project also supports Python integrations for easy automation and customization. GPT4All is ideal for individuals and businesses seeking private, offline access to powerful LLMs.
    5.0 out of 5 stars 1 Review
    Downloads: 86 This Week
    Last Update:
    See Project
  • 3
    OpenJarvis

    OpenJarvis

    Personal AI, On Personal Devices

    OpenJarvis is an open-source framework designed to build personal AI agents that run primarily on local devices rather than relying on cloud infrastructure. Developed as part of the Intelligence Per Watt research initiative, it focuses on improving the efficiency and practicality of on-device AI systems. The framework provides shared primitives for building local-first agents, along with evaluation tools that measure performance using metrics such as energy consumption, latency, cost, and...
    Downloads: 290 This Week
    Last Update:
    See Project
  • 4
    OuteTTS

    OuteTTS

    Interface for OuteTTS models

    ...It provides a high-level Interface API that wraps model configuration, speaker handling, and audio generation so you can focus on integrating speech into your application rather than wiring up low-level engines. The project supports multiple backends including llama.cpp (Python bindings and server), Hugging Face Transformers, ExLlamaV2, VLLM and a JavaScript interface via Transformers.js, allowing it to run on CPUs, NVIDIA CUDA GPUs, AMD ROCm, Vulkan-capable GPUs, and Apple Metal. It also includes a notion of speaker profiles: you can create a speaker from a short audio sample, save it as JSON, and reuse it for consistent voice identity across generations and sessions. ...
    Downloads: 0 This Week
    Last Update:
    See Project
  • Build Securely on AWS with Proven Frameworks Icon
    Build Securely on AWS with Proven Frameworks

    Lay a foundation for success with Tested Reference Architectures developed by Fortinet’s experts. Learn more in this white paper.

    Moving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
    Download Now
  • 5
    mac code

    mac code

    Claude Code, but it runs on your Mac for free

    mac code is a local AI coding agent designed to run large language models directly on Apple Silicon machines without relying on cloud services, effectively transforming a Mac into a self-contained AI development environment. The project focuses on enabling models that traditionally exceed available RAM to run efficiently by streaming model weights from SSD storage, thereby overcoming hardware limitations through innovative memory management techniques. It operates as a CLI-based assistant...
    Downloads: 2 This Week
    Last Update:
    See Project
  • 6
    Operit AI

    Operit AI

    Powerful Android AI agent with tools, automation, and Linux shell

    Operit is a full-featured AI assistant and agent platform designed specifically for Android devices, aiming to go far beyond traditional chat-based interfaces. It integrates deep system-level capabilities with a wide range of tools, allowing the AI to perform real tasks such as file management, automation, and system control directly on the device. A standout aspect of the project is its built-in Ubuntu 24 environment, which enables users to run Linux commands, scripts, and development tools...
    Downloads: 1,353 This Week
    Last Update:
    See Project
  • 7
    Harbor LLM

    Harbor LLM

    Run a full local LLM stack with one command using Docker

    Harbor is an open source, containerized toolkit designed to simplify running local large language model (LLM) environments. It combines a CLI and companion app to launch backends, frontends, and supporting services with minimal setup. With a single command, users can start preconfigured tools like Ollama and Open WebUI, enabling chat, workflows, and integrations immediately. Harbor supports multiple inference engines, including llama.cpp and vLLM, and connects them seamlessly to user...
    Downloads: 2 This Week
    Last Update:
    See Project
  • 8
    CoPaw

    CoPaw

    Your Personal AI Assistant; easy to install, deploy on local or coud

    CoPaw is a personal AI assistant designed to run on your own machine or in the cloud, giving you full control over memory, models, and data. Built by the AgentScope team, it connects to multiple chat platforms—including DingTalk, Feishu, QQ, Discord, iMessage, and more—through a single unified assistant. CoPaw supports both cloud-based LLM providers and fully local models such as llama.cpp, MLX, and Ollama, allowing you to operate without API keys if preferred. It includes a browser-based...
    5.0 out of 5 stars 1 Review
    Downloads: 5 This Week
    Last Update:
    See Project
  • 9
    Bonsai 27B

    Bonsai 27B

    Run Bonsai (1-bit) and Ternary-Bonsai language models locally

    Bonsai 27B is a repository for downloading, configuring, and running PrismML’s highly compressed Bonsai language models on local hardware. It supports the 1-bit Bonsai and higher-quality Ternary-Bonsai families in 1.7B, 4B, 8B, and 27B sizes. The models can run on macOS, Linux, and Windows through CPU, Metal, CUDA, Vulkan, ROCm, llama.cpp, or MLX backends. Its 27B models process text, images, screenshots, and PDFs while supporting reasoning and long-context conversations. They also provide...
    Downloads: 29 This Week
    Last Update:
    See Project
  • Custom VMs From 1 to 96 vCPUs With 99.95% Uptime Icon
    Custom VMs From 1 to 96 vCPUs With 99.95% Uptime

    General-purpose, compute-optimized, or GPU/TPU-accelerated. Built to your exact specs.

    Live migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
    Start Free
  • 10
    Outlines

    Outlines

    Structured Outputs

    Outlines is a Python library for making language models generate predictable, structured outputs. Instead of repairing malformed responses afterward, it constrains generation to match a requested type or format. Developers can require predefined choices, primitive Python types, Pydantic models, JSON schemas, regular expressions, function signatures, or custom grammars.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 11
    Backtrack Sampler

    Backtrack Sampler

    An easy-to-understand framework for LLM samplers

    Backtrack Sampler is a framework designed for experimenting with custom sampling strategies for language models (LLMs), enabling the ability to rewind and revise generated tokens. It allows developers to create and test their own token generation strategies by providing a base structure for manipulating logits and probabilities, making it a flexible tool for those interested in fine-tuning the behavior of LLMs.
    Downloads: 0 This Week
    Last Update:
    See Project
  • 12
    Hy-MT2
    Hy-MT2 is a family of fast-thinking multilingual translation models built for complex real-world translation scenarios. It includes 1.8B, 7B, and 30B-A3B model sizes, giving users options for lightweight deployment, stronger general performance, or MoE-based capacity. The models support translation across 33 languages and are designed to follow detailed translation instructions in multiple languages. They can handle tasks involving terminology, style, personalization, delimiters, structured...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 13
    Qwen3

    Qwen3

    Qwen3 is the large language model series developed by Qwen team

    Qwen3 is a cutting-edge large language model (LLM) series developed by the Qwen team at Alibaba Cloud. The latest updated version, Qwen3-235B-A22B-Instruct-2507, features significant improvements in instruction-following, reasoning, knowledge coverage, and long-context understanding up to 256K tokens. It delivers higher quality and more helpful text generation across multiple languages and domains, including mathematics, coding, science, and tool usage. Various quantized versions,...
    5.0 out of 5 stars 1 Review
    Downloads: 15 This Week
    Last Update:
    See Project
  • 14
    Text Generation Web UI

    Text Generation Web UI

    Oobabooga - The definitive Web UI for local AI, with powerful features

    A gradio web UI for running Large Language Models like LLaMA, llama.cpp, GPT-J, Pythia, OPT, and GALACTICA. Dropdown menu for switching between models. Notebook mode that resembles OpenAI's playground. Chat mode for conversation and role playing. Instruct mode compatible with Alpaca and Open Assistant formats. Nice HTML output for GPT-4chan. Markdown output for GALACTICA, including LaTeX rendering. Custom chat characters. Advanced chat features (send images, get audio responses with TTS)....
    Downloads: 13 This Week
    Last Update:
    See Project
  • 15
    SemIf

    SemIf

    Semantic ifs from open models, on a 3090 at home

    SemIf, formerly OpenJev, is an independent research project for making small semantic decisions with open language models. Instead of generating text and parsing it afterward, it reads probabilities directly from declared answer options. Each request supplies state, a runtime-defined question, and typed choices, allowing the model to behave like a semantic conditional. Shared-state execution can reuse a long context across many related decisions to reduce repeated computation. Backends...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 16
    MiniCPM5

    MiniCPM5

    MiniCPM5: SOTA on-device LLMs, small yet powerful

    MiniCPM is an open-source family of compact large language models developed by OpenBMB for efficient AI inference on personal devices and resource-constrained hardware. Its latest MiniCPM5 series includes 1B and 2B-class models optimized for local deployment. The models support text generation, coding, mathematical reasoning, and autonomous agent workflows. MiniCPM5-2B uses a Transformer architecture with grouped-query attention and a 131,072-token context window. Earlier releases explore...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 17
    LoLLMs Hub Fortress

    LoLLMs Hub Fortress

    A proxy server for multiple ollama instances with Key security

    LoLLMs Hub Fortress is a high-performance AI orchestration platform designed to unify multiple large language model backends into a single, secure, and scalable API layer. It acts as a central gateway that connects different inference engines such as Ollama, llama.cpp, vLLM, and OpenAI-compatible services, allowing them to function as interchangeable compute nodes within one system. The architecture is built around a hierarchical “master and slave” hub model, enabling distributed deployments...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 18
    GPUStack

    GPUStack

    Performance-optimized AI inference on your GPUs

    GPUStack is an open-source GPU cluster management platform designed to simplify the deployment and operation of artificial intelligence models across heterogeneous hardware environments. The system aggregates GPU resources from multiple machines into a unified cluster so developers and administrators can run large language models and other AI workloads efficiently across distributed infrastructure. Instead of requiring complex orchestration systems such as Kubernetes, GPUStack provides a...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 19
    GLM-4

    GLM-4

    GLM-4 series: Open Multilingual Multimodal Chat LMs

    GLM-4 is a family of open models from ZhipuAI that spans base, chat, and reasoning variants at both 32B and 9B scales, with long-context support and practical local-deployment options. The GLM-4-32B-0414 models are trained on ~15T high-quality data (including substantial synthetic reasoning data), then post-trained with preference alignment, rejection sampling, and reinforcement learning to improve instruction following, coding, function calling, and agent-style behaviors. The...
    Downloads: 13 This Week
    Last Update:
    See Project
  • 20
    OpenAI Sublime Text Plugin

    OpenAI Sublime Text Plugin

    First class Sublime Text AI assistant with gpt-5, Opus 4.6, Gemini 3

    OpenAI Sublime Text is a full-featured AI assistant plugin designed to bring advanced language model capabilities into the Sublime Text editor with a deeply integrated user experience. It supports a wide range of providers, including OpenAI, Anthropic, Google Gemini, and local backends like Ollama or llama.cpp, making it highly flexible for different deployment scenarios. The plugin offers both chat-based interaction and inline assistance through “phantoms,” which display non-intrusive...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 21
    Orpheus TTS

    Orpheus TTS

    Towards Human-Sounding Speech

    Orpheus TTS is a state-of-the-art open-source text-to-speech system built on a Llama-3B backbone, treating speech synthesis as a large language model problem instead of a traditional TTS pipeline. It is designed to produce human-like speech with natural intonation, emotion, and rhythm, targeting quality comparable to or better than many closed-source systems. The project ships both pretrained and finetuned English models, as well as a family of multilingual models released as a research...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 22
    DocsGPT

    DocsGPT

    Private AI agents, assistants and search over your own documents

    DocsGPT is an open-source (MIT) platform for private AI agents, assistants and enterprise search over your own documents, built by Arc53. It ingests PDFs, Office files, scanned pages (OCR), web pages and repositories, syncs Google Drive, SharePoint, Confluence, GitHub and S3, and answers with citations to the source passage. Agents call tools, APIs and MCP servers with per-action approval; a visual builder chains AI steps, conditions and code into workflows; agents can produce DOCX, XLSX,...
    Downloads: 3 This Week
    Last Update:
    See Project
  • 23
    Row-Bot

    Row-Bot

    A local-first AI assistant that gets real work done

    Row-Bot is a free, open-source, local-first AI assistant designed to help users automate tasks, manage information, and interact with applications while maintaining control over their personal data. The platform supports multiple AI models, including ChatGPT, Claude, Grok, and locally hosted models through Ollama, LM Studio, and other compatible services, allowing users to choose their preferred AI provider. Row-Bot features persistent memory powered by a personal knowledge graph, enabling...
    Downloads: 1 This Week
    Last Update:
    See Project
  • 24
    Tiny LLM zh

    Tiny LLM zh

    Implement a small-parameter Chinese large language model from zero

    Tiny LLM zh is an open-source educational project that demonstrates how to build and train compact Chinese large language models from scratch. It covers the complete language model development process, including tokenization, pretraining, supervised fine-tuning, and human preference alignment. The architecture uses Transformer components such as RMSNorm, rotary positional embeddings, and multi-head attention. It supports several model sizes, ranging from approximately 16 million to 440...
    Downloads: 0 This Week
    Last Update:
    See Project
  • 25
    KoboldCpp

    KoboldCpp

    Run GGUF models easily with a UI or API. One File. Zero Install.

    KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF models, inspired by the original KoboldAI. It's a single self-contained distributable that builds off llama.cpp and adds many additional powerful features.
    Leader badge
    Downloads: 384 This Week
    Last Update:
    See Project
  • Previous
  • You're on page 1
  • 2
  • Next