Search Results for "routing"
Sort By:
Showing 126 open source projects for "routing"
View related business solutions-
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
OmniRoute
OmniRoute is an AI gateway for multi-provider LLM
OmniRoute is a routing and orchestration framework designed to simplify the handling of requests, workflows, or data flows across multiple services or endpoints in a unified manner. It focuses on providing a flexible abstraction layer where developers can define routing logic that dynamically directs traffic based on conditions, context, or predefined rules. -
2
Switchyard
Switchyard lets LLM applications route traffic across models
Switchyard is a Rust proxy and library for routing traffic between LLM clients and model backends. It translates among OpenAI Chat, OpenAI Responses, and Anthropic Messages formats so agents can keep using their native APIs. Requests can be distributed across vLLM, NVIDIA NIM, Ollama, OpenRouter, and other compatible endpoints. Routing strategies include random splits, LLM classification, signal-driven stage routing, escalation, and custom algorithms. -
3
Claude Code Router
Use Claude Code as the foundation for coding infrastructure
Claude Code Router is a tool for routing Claude Code requests to different model providers and customizing how those requests are handled. It acts as a configurable layer between Claude Code and multiple LLM backends, allowing users to control routing, provider selection, transformations, and workflow behavior. The project is useful for developers who want to keep the Claude Code interface while experimenting with different models, cost profiles, and specialized request handling. ... -
4
Envoy AI Gateway
Manages Unified Access to Generative AI Services
...The project extends Envoy Gateway to support AI-specific workloads, enabling organizations to route, secure, and scale requests to large language models and other generative AI services. In a typical deployment, the architecture uses a two-tier gateway model where an outer gateway handles authentication, routing, and global rate limiting while a second gateway manages traffic to self-hosted model serving clusters. This design allows organizations to centralize control over AI service access while maintaining flexibility in how backend model infrastructure is deployed. The gateway provides policy enforcement, observability, and routing capabilities that are specifically designed for AI inference workloads, including intelligent endpoint selection and request optimization. -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
5
9Router
Unlimited FREE AI coding. Connect Claude Code, Codex, Cursor, Cline
...It includes smart routing logic that can automatically redirect requests between premium, low-cost, or free providers based on quotas, availability, or configuration rules. The system also offers a local dashboard for monitoring quotas, providers, models, and routing behavior in real time. 9router emphasizes flexibility and uninterrupted workflows by supporting failover mechanisms, provider pooling, and configuration-driven request management. -
6
ClawRouter
Smart LLM router
ClawRouter is a flexible networking and routing framework designed to support AI-oriented distributed systems and agent ecosystems by managing how messages, requests, and responses are routed between components. It provides a programmable router abstraction that can handle complex traffic patterns, enabling dynamic message forwarding, load balancing, and custom routing logic based on content, context, or policy rules. -
7
OpenSquilla
Token-Efficient AI Agent with same budget, higher intelligence density
...It includes persistent memory, built-in web search, on-device embeddings, and sandboxing for safer execution. OpenSquilla is designed for users who want stronger agent capabilities without wasting tokens on every interaction. Its main value is combining cost-aware routing, durable context, and multi-channel agent execution in one local runtime. -
8
Sprix SAGE Router
State-aware SELF/COLLABORATE/HANDOFF routing for A2A agent networks
Sprix SAGE Router is an open-source routing layer for coordinating agents in Agent2Agent networks. It decides whether an active agent should continue alone, recruit collaborators, or hand a task to another specialist. Routing decisions account for current progress, capability coverage, context-transfer costs, permissions, budgets, and deadlines. The system assigns remaining requirements to agents through task dependency graphs and estimates execution schedules. -
9
caveman
Why use many token when few token do trick
...The project often serves as a foundation for developers who want to build applications quickly without being constrained by heavy conventions or extensive configuration. It may include utilities for routing, state handling, or simple server logic, depending on its implementation scope. Caveman embraces a philosophy of “less is more,” encouraging developers to focus on core functionality rather than framework overhead. Its design makes it particularly useful for experimentation, small tools, or proof-of-concept applications. -
Save Up to 91% on Cloud Compute With Spot VMsRun batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
-
10
litelm
litellm without the bloat
litelm is a lightweight Python library that preserves the core routing and message-translation behavior of LiteLLM without its larger proxy and infrastructure layers. It supports provider/model routing, streaming, embeddings, text completions, tool calling, and the OpenAI Responses API. The package mirrors LiteLLM function names and response shapes so existing code can often switch with only an import change. -
11
MSA: Memory Sparse Attention
Trainable latent-memory framework for 100M-token contexts
MSA, or Memory Sparse Attention, is a research framework for scaling language-model memory to extremely long contexts. It replaces full attention over all tokens with sparse selection of compressed latent memory states. Document-wise rotary position encoding and top-k routing keep training and inference close to linear complexity. A tiered KV-cache design stores routing keys on GPU while larger content states can remain on CPU. Its Memory Parallel engine distributes scoring and transfers only selected memory back to the accelerator. Memory Interleave alternates retrieval, context expansion, and generation to improve multi-hop reasoning across distant segments. ... -
12
Weave Router
Model router for agentic systems
...It supports streaming, tool use, vision, and multiple API formats. Provider credentials remain local in self-hosted deployments and can be encrypted at rest. Built-in observability exposes routing decisions, costs, and OTLP traces through a dashboard or external monitoring tools. It integrates with Claude Code, Codex, OpenCode, Pi, Cursor, and custom applications. -
13
Plano
Delivery infrastructure for agentic apps
Plano is an AI-native proxy and data plane designed to simplify the infrastructure required to deploy and operate agentic applications in production environments. It removes repetitive plumbing work from application code by centralizing capabilities such as agent routing, orchestration, guardrails, observability, and model selection. Built on modern proxy technology and compatible with any language or AI framework, Plano enables developers to focus on core agent logic instead of infrastructure complexity. The system provides intelligent LLM routing APIs that support model agility, along with filter chains for safety, moderation, and memory hooks. ... -
14
Microsoft MCP Gateway
MCP Gateway is a reverse proxy and management layer for MCP servers
...It also includes local and Azure deployment recipes, making it straightforward to try on a laptop and then promote to cloud infrastructure. Core concepts—like session routing, server pools, and pluggable policies—are documented to help teams reason about stateful agent sessions at scale. By standardizing discovery and control, it reduces operational toil and improves reliability for agent platforms built on MCP. -
15
Colibrì
Run frontier MoE models on hardware you already own
Colibrì is an open-source inference engine designed to run frontier-scale mixture-of-experts (MoE) models on consumer and heterogeneous hardware. It treats VRAM, system RAM, and NVMe storage as a unified memory hierarchy, dynamically placing and streaming model weights where they can be accessed most efficiently. This architecture enables users to run models ranging from 7 billion to 2.8 trillion parameters without requiring the entire model to fit in expensive GPU memory. Written primarily... -
16
OpenMythos
A theoretical reconstruction of the Claude Mythos architecture
...It divides computation into three main stages, including a pre-processing phase, a looped recurrent reasoning block, and a final output refinement stage, creating a structured pipeline for inference. The architecture incorporates advanced techniques such as mixture-of-experts routing, adaptive computation time, and multiple attention mechanisms to dynamically allocate compute where needed. It is highly configurable through a centralized configuration system, allowing experimentation with different architectural parameters such as loop depth, attention type. -
17
gemini-web2api
Convert Google Gemini web into OpenAI-compatible API
...The project can run as a simple local server and uses a mostly single-file design with an optional dependency for streaming. It supports model aliases for Flash, Thinking, Pro-style routing, Auto, and Lite variants. The tool also includes optional API keys, function calling, SSE streaming, web search access, Docker deployment, and client examples for OpenAI SDK-style usage. It is useful for developers who want local experimentation with OpenAI-compatible tooling while routing requests through Gemini web behavior. -
18
Experiential
Experiential is the open source, zero markup gateway for BYOK
...The platform controls which users and agents can access specific models and how much they can spend. It collects execution traces to evaluate model performance, costs, and response speed. Developers can optimize custom routing strategies using real agent traffic and simulation workflows. It also supports fine-tuning specialized open-source models based on collected interactions. Experiential integrates with coding agents such as Claude Code, Codex, and Cursor and provides configurable telemetry, access controls, and local deployment options. -
19
Laya
Non-autoregressive System 1 decision engine
...It can run locally on CPU or GPU hardware and includes Python and TypeScript tooling. Integrations are available for HTTP, MCP, LangChain, LangGraph, LlamaIndex, CrewAI, and ONNX workflows. Laya is intended for fast classification, routing, triage, scoring, and other structured decisions where predictable outputs are more useful than generated prose. -
20
clawhip
claw + whip: Event-to-channel notification router
...It acts as a central event-processing system that listens to sources such as Git, GitHub, tmux sessions, and custom CLI events, then routes them through a typed pipeline. Built with a clean separation between routing, rendering, and delivery, Clawhip ensures reliable and organized notifications without polluting AI agent contexts. It integrates seamlessly with tools like OpenClaw, OMX (oh-my-codex), and OMC (oh-my-claudecode) to monitor coding sessions and automate updates. The system emphasizes normalized event contracts (like session.* and agent.*) for consistent automation and routing logic. ... -
21
kgateway
The Cloud-Native API Gateway and AI Gateway
...It implements the Kubernetes Gateway API and can operate as both a lightweight in-cluster microgateway and a centralized gateway capable of handling billions of API calls with high performance and low latency. By integrating with Envoy and advanced data planes, it handles modern ingress concerns such as traffic routing, authentication, authorization, rate limiting, and observability for traditional HTTP/gRPC services and AI workloads alike. Beyond standard API traffic, kgateway also supports gateway patterns tailored for large language model (LLM) consumption, inference routing, and Model Context Protocol (MCP) orchestration, enabling secure access to models, tools, and agent interactions. -
22
Mastra
The TypeScript AI agent framework
...It integrates cleanly with React, Next.js, and Node-based backends, but can also run as a standalone server, giving teams flexibility in how they deploy their AI logic. At its core, Mastra provides abstractions for agents, workflows, tools, memory, retrieval, and model routing, so developers can focus on specifying behavior rather than wiring infrastructure from scratch. Model routing lets you connect to dozens of providers (OpenAI, Anthropic, Gemini, and others) through a single standardized interface, while agents orchestrate LLM calls and tools to solve open-ended tasks with internal reasoning loops. ... -
23
Flash-MoE
Running a big model on a small laptop
Flash-MoE is a high-performance implementation of mixture-of-experts (MoE) architectures designed to optimize the efficiency and scalability of large AI models. It focuses on accelerating routing and computation by leveraging optimized kernels and memory management techniques, allowing models to dynamically select specialized sub-networks during inference. The project aims to reduce the computational cost typically associated with MoE systems while maintaining or improving performance. It likely includes support for GPU acceleration and parallel processing, enabling it to handle large-scale workloads effectively. ... -
24
Arch Gateway
The AI-native (edge and LLM) proxy for agents
Arch is an AI-native proxy designed to facilitate the development of agentic applications by handling complex tasks such as input clarification, agent routing, and seamless integration of prompts with tools for common tasks. It provides unified access and observability of Large Language Models (LLMs), enabling developers to build applications more efficiently. Arch supports both edge and LLM deployments, offering flexibility in various environments. -
25
VoiceStudio
VoiceStudio is the open-source, fully-local ElevenLabs alternative
VoiceStudio is an open-source, local-first platform for speech generation, voice cloning, transcription, dubbing, and audio production. It can clone voices from short reference recordings or design new voices from characteristics such as accent, pitch, age, and delivery style. Video dubbing combines transcription, translation, speaker preservation, speech synthesis, and final video export. It also supports multi-voice stories, audiobooks, dictation, vocal isolation, and speaker diarization....
