Open Source Linux LLM Gateways
Sort By:
LLM Gateways for Linux
View 11 business solutionsBrowse free open source LLM Gateways and projects for Linux below. Use the toggles on the left to filter open source LLM Gateways by OS, license, language, programming language, and project status.
-
99.99% Uptime for MySQL and PostgreSQL DatabasesCloud SQL Enterprise Plus delivers near-zero downtime with 35 days of point-in-time recovery. Supports MySQL, PostgreSQL, and SQL Server.
-
Paessler: Easy to Use With Enterprise Power. Free TrialYou shouldn't have to choose between a monitoring tool that's easy to use and one that's powerful enough for a complex environment. PRTG's low-code interface lets any admin build dashboards, set alerts and monitor devices without scripting, while custom sensors and full API access are there when your team needs deeper control. One platform, no compromise. Download a free 30-day trial now.
-
1
Kong
The Cloud-Native API Gateway
Kong is a next generation cloud-native API platform for multi-cloud and hybrid organizations. When building for the web, mobile, or Internet of Things, you’ll need a common functionality to run your software, and Kong is that solution. Kong acts as a gateway, connecting microservices requests and APIs natively while also providing load balancing, logging, monitoring, authentication, rate-limiting, and so much more through plugins. Kong is highly extensible as well as platform agnostic, connecting APIs across different environments, platforms and patterns. Achieve architectural freedom with Kong today. -
2
AQBot
Lightweight, high-performance cross-platform AI dialogue + AI Agent
AQBot is a free, open-source AI desktop workspace that combines multi-provider conversations, coding agents, knowledge management, and API gateway functionality. It supports OpenAI, Claude, Gemini, DeepSeek, Qwen, and custom compatible endpoints. Users can compare model responses, branch conversations, and manage reusable AI personas. Its Agent Client Protocol workspace connects external coding agents such as Codex, Claude Agent, and OpenCode. Integrated tools allow agents to edit files, execute commands, and interact with MCP servers under configurable permissions. A local knowledge base supports document retrieval and contextual conversations. The built-in API gateway exposes multiple provider-compatible endpoints for external applications. Available for Windows, macOS, and Linux, AQBot stores application data locally and supports encrypted backups. -
3
Bifrost
The Fastest LLM Gateway with built in OTel observability
Bifrost is an LLM gateway designed to provide a unified OpenAI-compatible API front for many different model providers. It abstracts away the complexity of working directly with multiple backend providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, etc.), enabling you to plug in providers and switch between them without touching your client code. It is built to be high performance: in benchmark tests at 5,000 requests per second, it reportedly adds only microseconds of overhead and achieves perfect success rates with no failed requests. Bifrost supports features such as automatic fallback (failover between providers), load balancing across API keys/providers, and semantic caching to reduce latency and cost. It also includes observability with built-in metrics, tracing, logging, and supports governance features like rate limiting, access control, and cost budgeting. The architecture is modular: there is a core engine, plugin layers, and transport layers (HTTP APIs). -
4
LLM Gateway
Route, manage, and analyze your LLM requests across multiple providers
LLM Gateway is an open-source middleware that consolidates interactions with multiple LLM providers—such as OpenAI, Anthropic, Google Vertex AI—behind a single, unified API compatible with OpenAI's spec. Designed for both self-hosted and cloud use, it enables developers to route requests dynamically, secure and manage API keys, monitor token usage and costs, and analyze performance metrics. With optional UI, telemetry, and Docker deployment, it's ideal for teams aiming to centralize LLM orchestration and gain visibility into AI usage. -
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
5
Headroom Desktop
Menu bar app for macOS, Windows and Linux that cuts Claude Code
Headroom Desktop is a cross-platform utility that reduces token consumption in AI coding assistants through local prompt and output optimization. It works with Claude Code, ChatGPT Codex, OpenCode, and Grok Build without requiring changes to established coding workflows. A local proxy compresses repetitive logs, structured data, and unnecessary tool output before forwarding requests to AI providers. Original content can be retrieved when needed to preserve context. The application manages an isolated Python runtime without modifying the system installation. Optional tools provide command output compression, document conversion, and codebase analysis. A dashboard displays token savings, usage statistics, and pipeline health. Headroom runs in the system tray on Windows, macOS, and Linux, with automatic recovery and configuration cleanup. -
6
LangDB AI Gateway
Govern, secure, and optimize your AI traffic
AI Gateway is a high-performance, open-source API gateway optimized for managing and monitoring LLM traffic at scale. Developed by the LangDB team, AI Gateway acts as an intermediary between clients and backend LLMs, providing advanced features like caching, rate limiting, prompt management, and observability. It helps teams secure and optimize their LLM deployments, whether using local models or external APIs like OpenAI or Anthropic. With native support for multi-tenant environments and low-latency inference routing, AI Gateway is an essential tool for companies building production-grade generative AI services. -
7
Portkey AI Gateway
A blazing fast AI Gateway with integrated guardrails
Portkey AI Gateway aims to offer a blazing fast, secure, and flexible gateway for interacting with a wide variety of models and enforcing guardrails. It presents a single, friendly API through which you can route to 200+ LLMs, while applying configurable input/output guardrails to enforce policies or restrict certain content. It supports automatic retries, fallbacks, load balancing across providers or keys, and request timeouts to avoid latency spikes. The gateway is multimodal: it can handle text, vision, audio, and image models under a common interface. It also offers features for governance: role-based access, compliance with standards (SOC2, HIPAA, GDPR), secure key management, and logging/analytics of usage, latency, errors, and cost. The system integrates with agent frameworks like LangChain, Autogen, and others, enabling the building of more complex AI applications. It’s lightweight and optimized for low latency with a small footprint. -
8
GoModel
The best AI Gateway 2026 - GoModel
GoModel is the last AI Gateway you'll ever try. It's the most secure, the fastest, and the most reliable AI Gateway according to self-verifiable benchmarks. The best open-source alternative to LiteLLM, GoModel provides a unified OpenAI-compatible API for OpenAI, Anthropic, Gemini, Groq, xAI, Ollama, vLLM, Amazon Bedrock, OpenRouter, and many more providers. Built in Go, it's lightweight, resource-efficient, and designed for production workloads with extremely low latency. GoModel includes intelligent request routing, observability, guardrails, live logs, usage and cost tracking, rate limiting, virtual models, prompt rewriting, streaming support, and enterprise-ready authentication. Whether you're building AI applications, running coding agents, or deploying LLM infrastructure at scale, GoModel gives you a fast, reliable, and self-hostable foundation. If you're looking for a modern AI Gateway that is faster, simpler, and more efficient than LiteLLM, GoModel is built for you. -
9
Kaeso
Open-source self-hosted AI gateway
Kaeso is an open-source AI gateway you run yourself. Applications talk to a single OpenAI- and Anthropic-compatible API while you keep full control over provider keys, routing, usage limits, and logging. Self-host with Docker and manage everything through a simple web console. -
Host LLMs in Production With On-Demand GPUsDeploy your model, get an endpoint, pay only for compute time. No GPU provisioning or infrastructure management required.
-
10
APIPark
APIPark is the #1 open-source AI Gateway and Developer Portal
APIPark is an open-source, all-in-one AI gateway and API developer portal, that helps developers and enterprises easily manage, integrate, and deploy AI services. No matter which AI model you use, APIPark provides a one-stop integration solution. It unifies the management of all authentication information and tracks the costs of API calls. Standardize the request data format for all AI models. When switching AI models or modifying prompts, it won’t affect your app or microservices, simplifying your AI usage and reducing maintenance costs. You can quickly combine AI models and prompts into new APIs. For example, using OpenAI GPT-4 and custom prompts, you can create sentiment analysis APIs, translation APIs, or data analysis APIs. API lifecycle management helps standardize the process of managing APIs, including traffic forwarding, load balancing, and managing different versions of publicly accessible APIs. This improves API quality and maintainability. -
11
MagicAPI AI Gateway
Built for demanding AI workflows
The world's fastest AI Gateway proxy, written in Rust and optimized for maximum performance. This high-performance API gateway routes requests to various AI providers (OpenAI, GROQ) with streaming support, making it perfect for developers who need reliable and blazing-fast AI API access. -
12
TagForge
Cross-platform AI tagging and prompt engineering studio
TagForge is a professional, cross-platform desktop application for AI prompt engineering and image tagging. Built on .NET 9 and Avalonia UI, it provides a sleek, high-performance workspace for crafting perfect prompts for Stable Diffusion, FLUX, Midjourney, and other generative AI models. ## Key Capabilities - Multi-model tagging: Specialized generators for Stable Diffusion (tags), FLUX/Midjourney (prose), and custom LLM workflows - Multimodal vision: Extract metadata or generate descriptive captions directly from images - Contextual chat assistant: Persistent, coding-capable AI helper with real-time provider features - Persona system: Create custom identities with dynamic input injection and role-based templates - Chat rules: Modular behavioral controls (Concise, Detailed, etc.) with CRUD interface - Agent orchestrator: Centralized management of API keys, endpoints, and model parameters -
13
free llm plateform on-premise
- Previous
- You're on page 1
- Next
