Open Source Linux Large Language Models (LLM)
Sort By:
Large Language Models (LLM) for Linux
View 110 business solutionsBrowse free open source Large Language Models (LLM) and projects for Linux below. Use the toggles on the left to filter open source Large Language Models (LLM) by OS, license, language, programming language, and project status.
-
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
1
Ollama
Run models like Kimi-K2.5, GLM-5, DeepSeek, gpt-oss, Gemma, Qwen etc.
Ollama is an open-source platform that enables developers to run large language models locally on their own machines. It simplifies working with modern AI models by providing a unified interface to download, manage, and interact with them. Users can run models like Llama, Gemma, Qwen, and others directly from the command line or through APIs. Ollama also integrates with popular developer tools and AI agents, allowing seamless workflows across coding environments and applications. It supports REST APIs, Python, and JavaScript SDKs, making it easy to build AI-powered features into software projects. Overall, Ollama focuses on privacy, local-first AI execution, and developer-friendly tooling for building with open models. -
2
OmniRoute
OmniRoute is an AI gateway for multi-provider LLM
OmniRoute is a routing and orchestration framework designed to simplify the handling of requests, workflows, or data flows across multiple services or endpoints in a unified manner. It focuses on providing a flexible abstraction layer where developers can define routing logic that dynamically directs traffic based on conditions, context, or predefined rules. The project emphasizes modularity and extensibility, allowing users to plug in different services or handlers without tightly coupling components. It is particularly useful in distributed systems where requests need to be intelligently routed between APIs, microservices, or processing pipelines. OmniRoute aims to reduce boilerplate by centralizing routing logic and providing reusable patterns for managing complex flows. Its architecture supports scalability and maintainability, making it suitable for both small applications and larger systems with multiple integrations. -
3
SillyTavern
LLM Frontend for Power Users
Mobile-friendly, Multi-API (KoboldAI/CPP, Horde, NovelAI, Ooba, OpenAI, OpenRouter, Claude, Scale), VN-like Waifu Mode, Horde SD, System TTS, WorldInfo (lorebooks), customizable UI, auto-translate, and more prompt options than you'd ever want or need. Optional Extras server for more SD/TTS options + ChromaDB/Summarize. SillyTavern is a user interface you can install on your computer (and Android phones) that allows you to interact with text generation AIs and chat/roleplay with characters you or the community create. SillyTavern is a fork of TavernAI 1.2.8 which is under more active development and has added many major features. At this point, they can be thought of as completely independent programs. -
4
llama.cpp
Port of Facebook's LLaMA model in C/C++
The llama.cpp project enables the inference of Meta's LLaMA model (and other models) in pure C/C++ without requiring a Python runtime. It is designed for efficient and fast model execution, offering easy integration for applications needing LLM-based capabilities. The repository focuses on providing a highly optimized and portable implementation for running large language models directly within C/C++ environments. -
Veeam Data Platform v13.1Try Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
5
BruteForceAI
Advanced LLM-powered brute-force tool combining AI intelligence
BruteForceAI is an open-source security testing tool that applies large language models to the analysis of login forms and authentication flows in web applications. At a high level, the project uses AI to inspect HTML content, identify the relevant form elements, and automate selector discovery so that a tester does not need to hand-map every field before evaluation. It combines that analysis layer with automated credential testing workflows, framing itself as a more adaptive alternative to older brute-force tooling that depends heavily on manual configuration. The repository emphasizes features such as threaded execution, logging, and notification integrations, which position it as an automation-oriented project for controlled security assessment environments. From a software design perspective, its distinguishing idea is the use of language models as a front-end analysis layer that interprets a target page before the rest of the workflow proceeds. -
6
Hands-On Large Language Models
Official code repo for the O'Reilly Book
Hands-On-Large-Language-Models is the official GitHub code repository accompanying the practical technical book Hands-On Large Language Models authored by Jay Alammar and Maarten Grootendorst, providing a comprehensive collection of example notebooks, code labs, and supporting materials that illustrate the core concepts and real-world applications of large language models. The repository is structured into chapters that align with the educational progression of the book — covering everything from foundational topics like tokens, embeddings, and transformer architecture to advanced techniques such as prompt engineering, semantic search, retrieval-augmented generation (RAG), multimodal LLMs, and fine-tuning. Each chapter contains executable Jupyter notebooks that are designed to be run in environments like Google Colab, making it easy for learners to experiment interactively with models, visualize attention patterns, implement classification and generation tasks. -
7
llama.cpp
LLM inference in C/C++
llama.cpp is a high-performance C and C++ project for running large language models locally and in the cloud with minimal setup. It is built around efficient inference, broad hardware support, and the GGUF model format. The project supports many model families and has become a major foundation for local AI tools, model serving, and embedded inference workflows. It provides command-line tools, a server mode with an OpenAI-compatible API style, model conversion utilities, and extensive backend acceleration options. llama.cpp runs on CPUs and GPUs, with support for Apple silicon, x86, RISC-V, CUDA, HIP, Vulkan, SYCL, Metal, and hybrid CPU-GPU execution. Its main value is making practical LLM inference accessible across consumer machines, servers, and specialized deployment environments. -
8
AnythingLLM
The all-in-one Desktop & Docker AI application with full RAG and AI
A full-stack application that enables you to turn any document, resource, or piece of content into a context that any LLM can use as references during chatting. This application allows you to pick and choose which LLM or Vector Database you want to use as well as supporting multi-user management and permissions. AnythingLLM is a full-stack application where you can use commercial off-the-shelf LLMs or popular open-source LLMs and vectorDB solutions to build a private ChatGPT with no compromises that you can run locally as well as host remotely and be able to chat intelligently with any documents you provide it. AnythingLLM divides your documents into objects called workspaces. A Workspace functions a lot like a thread, but with the addition of containerization of your documents. Workspaces can share documents, but they do not talk to each other so you can keep your context for each workspace clean. -
9
omp (Oh My Pi)
AI Coding agent for the terminal
omp (Oh-My-Pi) is an open-source AI agent toolkit focused on creating intelligent coding assistants that operate directly from the terminal environment. The project provides a command-line coding agent capable of analyzing repositories, generating commits, editing code, and interacting with development tools through an integrated tool system. Instead of functioning as a simple prompt-based assistant, the system includes an agent architecture that can inspect Git repositories, analyze changes, and perform development actions with fine-grained control. The platform also supports tool-based workflows where the agent can run shell commands, read files, modify code, and stage changes during development tasks. It includes infrastructure for integrating different AI providers and models through a unified API layer, allowing developers to switch between models while keeping the same agent interface. -
Go from Code to Production URL in SecondsSkip the Kubernetes configs. Cloud Run handles HTTPS, scaling, and infrastructure automatically. Two million requests free per month.
-
10
rtk
CLI proxy that reduces LLM token consumption
rtk is an open-source command-line proxy designed to optimize interactions between AI coding agents and the terminal by reducing unnecessary token consumption. When AI assistants execute shell commands during software development tasks, the resulting terminal output often contains large amounts of repetitive or irrelevant information that can overwhelm the model’s context window. RTK intercepts these command outputs and compresses them into concise summaries before sending them to the language model. This process helps maintain important information while removing redundant data such as boilerplate logs, long directory listings, or repetitive test outputs. By minimizing the amount of noise sent to the AI model, the tool improves reasoning quality and allows longer development sessions within the same context window. The system is implemented as a lightweight Rust binary that runs locally and integrates easily with common AI coding environments. -
11
WeChatMsg
Project aimed at extracting, exporting, and analyzing chat records
WeChatMsg repository hosts an open-source project aimed at extracting, exporting, and analyzing chat records from the WeChat messaging platform. It provides tools that read local WeChat database files and allow users to convert chat data into readable formats such as HTML, Word, and CSV, making it possible to inspect conversations outside the mobile app environment. Beyond simple export, the project includes mechanisms for analyzing chat histories and generating annual reports or visual summaries about messaging trends, interaction patterns, and more. The original README communicates a guiding philosophy about owning personal data and using it responsibly to train personalized AI agents or preserve memories. Although the repository has seen periods of inactivity and may not receive frequent updates, its widespread use indicates community interest in preserving chat logs and understanding conversation data outside of the WeChat interface. -
12
DeepSeek R1
Open-source, high-performance AI model with advanced reasoning
DeepSeek-R1 is an open-source large language model developed by DeepSeek, designed to excel in complex reasoning tasks across domains such as mathematics, coding, and language. DeepSeek R1 offers unrestricted access for both commercial and academic use. The model employs a Mixture of Experts (MoE) architecture, comprising 671 billion total parameters with 37 billion active parameters per token, and supports a context length of up to 128,000 tokens. DeepSeek-R1's training regimen uniquely integrates large-scale reinforcement learning (RL) without relying on supervised fine-tuning, enabling the model to develop advanced reasoning capabilities. This approach has resulted in performance comparable to leading models like OpenAI's o1, while maintaining cost-efficiency. To further support the research community, DeepSeek has released distilled versions of the model based on architectures such as LLaMA and Qwen. -
13
vLLM
A high-throughput and memory-efficient inference and serving engine
vLLM is a fast and easy-to-use library for LLM inference and serving. High-throughput serving with various decoding algorithms, including parallel sampling, beam search, and more. -
14
llamafile
Distribute and run LLMs with a single file
llamafile lets you distribute and run LLMs with a single file. (announcement blog post). Our goal is to make open LLMs much more accessible to both developers and end users. We're doing that by combining llama.cpp with Cosmopolitan Libc into one framework that collapses all the complexity of LLMs down to a single-file executable (called a "llamafile") that runs locally on most computers, with no installation. The easiest way to try it for yourself is to download our example llamafile for the LLaVA model (license: LLaMA 2, OpenAI). LLaVA is a new LLM that can do more than just chat; you can also upload images and ask it questions about them. With llamafile, this all happens locally; no data ever leaves your computer. -
15
GPT4All
Run Local LLMs on Any Device. Open-source
GPT4All is an open-source project that allows users to run large language models (LLMs) locally on their desktops or laptops, eliminating the need for API calls or GPUs. The software provides a simple, user-friendly application that can be downloaded and run on various platforms, including Windows, macOS, and Ubuntu, without requiring specialized hardware. It integrates with the llama.cpp implementation and supports multiple LLMs, allowing users to interact with AI models privately. This project also supports Python integrations for easy automation and customization. GPT4All is ideal for individuals and businesses seeking private, offline access to powerful LLMs. -
16
DeepSeek-V3
Powerful AI language model (MoE) optimized for efficiency/performance
DeepSeek-V3 is a robust Mixture-of-Experts (MoE) language model developed by DeepSeek, featuring a total of 671 billion parameters, with 37 billion activated per token. It employs Multi-head Latent Attention (MLA) and the DeepSeekMoE architecture to enhance computational efficiency. The model introduces an auxiliary-loss-free load balancing strategy and a multi-token prediction training objective to boost performance. Trained on 14.8 trillion diverse, high-quality tokens, DeepSeek-V3 underwent supervised fine-tuning and reinforcement learning to fully realize its capabilities. Evaluations indicate that it outperforms other open-source models and rivals leading closed-source models, achieving this with a training duration of 55 days on 2,048 Nvidia H800 GPUs, costing approximately $5.58 million. -
17
FreeLLMAPI
OpenAI-compatible proxy that aggregates free-tier keys from ~14 AI
FreeLLMAPI is an OpenAI-compatible proxy that aggregates free-tier API keys from multiple AI providers into one unified endpoint. It is designed for personal experimentation, testing, and lightweight development workflows where users want to route requests through several providers without rewriting client code for each one. The project can automatically fail over between configured providers when one is unavailable or exhausted. Its OpenAI-compatible design makes it easier to use with existing tools, SDKs, and applications that already expect that API shape. It is not positioned as an enterprise-grade service or a way to bypass provider terms, but as a local coordination layer for personally owned free-tier credentials. freellmapi is useful for developers who want a practical testing proxy for comparing models, managing limits, and improving request continuity. -
18
LLPlayer
The media player for language learning, with dual subtitles
LLPlayer is an open-source media player designed specifically for language learning through video content. Unlike traditional media players, the application focuses on advanced subtitle-related features that help learners understand and interact with foreign language media more effectively. The player supports dual subtitles so users can simultaneously view text in both the original language and their native language while watching videos. It can also automatically generate subtitles in real time using speech-to-text systems such as Whisper, allowing subtitles to be created even when none are available. Real-time translation capabilities enable subtitles to be translated using multiple translation engines and language models. Additional tools such as instant word lookup, contextual translation, and subtitle search allow learners to interact with the text while watching videos. -
19
GLM-5.1
GLM-5: From Vibe Coding to Agentic Engineering
GLM-5.1 is a next-generation large language model developed by Z.ai for advanced coding, reasoning, and long-horizon agentic engineering tasks. Built as the successor to GLM-5, the model significantly improves performance in software engineering benchmarks, repository generation, and real-world terminal-based workflows. GLM-5.1 is designed to remain effective over extended problem-solving sessions, allowing it to iteratively refine strategies, analyze failures, and sustain productivity across hundreds of reasoning cycles and tool calls. The model leverages large-scale pretraining, reinforcement learning infrastructure, and sparse attention mechanisms to improve efficiency while maintaining strong long-context understanding. It supports deployment through frameworks such as vLLM, SGLang, xLLM, and KTransformers, enabling scalable local inference for enterprise and research use cases. -
20
GLM-5
From Vibe Coding to Agentic Engineering
GLM-5 is a next-generation open-source large language model (LLM) developed by the Z .ai team under the zai-org organization that pushes the boundaries of reasoning, coding, and long-horizon agentic intelligence. Building on earlier GLM series models, GLM-5 dramatically scales the parameter count (to roughly 744 billion) and expands pre-training data to significantly improve performance on complex tasks such as multi-step reasoning, software engineering workflows, and agent orchestration compared to its predecessors like GLM-4.5. It incorporates innovations like DeepSeek Sparse Attention (DSA) to preserve massive context windows while reducing deployment costs and supporting long context processing, which is crucial for detailed plans and agent tasks. -
21
llmfit
157 models, 30 providers, one command to find what runs on hardware
llmfit is a terminal-based utility that helps developers determine which large language models can realistically run on their local hardware by analyzing system resources and model requirements. The tool automatically detects CPU, RAM, GPU, and VRAM specifications, then ranks available models based on performance factors such as speed, quality, and memory fit. It provides both an interactive terminal user interface and a traditional CLI mode, enabling flexible workflows for different user preferences. llmfit also supports advanced configurations including multi-GPU setups, mixture-of-experts architectures, and dynamic quantization recommendations. By presenting clear performance estimates and compatibility guidance, the project reduces the trial-and-error typically involved in local LLM experimentation. Overall, llmfit serves as a practical decision assistant for developers who want to run language models efficiently on their own machines. -
22
Heretic
Fully automatic censorship removal for language models
Heretic is an open-source Python tool that automatically removes the built-in censorship or “safety alignment” from transformer-based language models so they respond to a broader range of prompts with fewer refusals. It works by applying directional ablation techniques and a parameter optimization strategy to adjust internal model behaviors without expensive post-training or altering the core capabilities. Designed for researchers and advanced users, Heretic makes it possible to study and experiment with uncensored model responses in a reproducible, automated way. The project can decensor many popular dense and some mixture-of-experts (MoE) models, supporting workflows that would otherwise require manual tuning. Beyond simple decensoring, Heretic includes research-oriented options for analyzing model internals and interpretability data. -
23
Flowise
Drag & drop UI to build your customized LLM flow
Open source UI visual tool to build your customized LLM flow using LangchainJS, written in Node Typescript/Javascript. Conversational agent for a chat model which utilizes chat-specific prompts and buffer memory. Open source is the core of Flowise, and it will always be free for commercial and personal usage. Flowise support different environment variables to configure your instance. You can specify the following variables in the .env file inside the packages/server folder. -
24
LiteLLM
lightweight package to simplify LLM API calls
Call all LLM APIs using the OpenAI format [Anthropic, Huggingface, Cohere, Azure OpenAI etc.] liteLLM supports streaming the model response back, pass stream=True to get a streaming iterator in response. Streaming is supported for OpenAI, Azure, Anthropic, and Huggingface models. -
25
MiroFish
A Simple and Universal Swarm Intelligence Engine
MiroFish is a next-generation artificial intelligence prediction engine that leverages multi-agent technology and swarm-intelligence simulation to model, simulate, and forecast complex real-world scenarios. The system extracts “seed” information from sources such as breaking news, policy documents, and market signals to construct a high-fidelity digital parallel world populated by thousands of virtual agents with independent memory and behavior rules. Users can inject variables or conditions into this simulated environment from a “god’s eye view,” enabling iterative prediction of future trends under different assumptions, which can be useful for decision support, scenario planning, or creative exploration. The engine includes both backend and frontend components, with configuration and deployment instructions for local and containerized setups, and is designed to produce detailed predictive reports based on interactions and emergent patterns within the simulated world.
