Search Results for "routing"
Sort By:
Showing 20 open source projects for "routing"
View related business solutions-
Paessler - Monitor Your Whole Network in MinutesWaiting weeks for a monitoring rollout isn't an option when infrastructure doesn't stop running. PRTG's auto-discovery scans your network and suggests from over 200 pre-configured sensor types, so you're watching servers, applications and devices within minutes, not after a multi-week deployment. Enterprise-strength monitoring, without the enterprise complexity. Start your free trial today.
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
1
OmniRoute
OmniRoute is an AI gateway for multi-provider LLM
OmniRoute is a routing and orchestration framework designed to simplify the handling of requests, workflows, or data flows across multiple services or endpoints in a unified manner. It focuses on providing a flexible abstraction layer where developers can define routing logic that dynamically directs traffic based on conditions, context, or predefined rules. -
2
Envoy AI Gateway
Manages Unified Access to Generative AI Services
...The project extends Envoy Gateway to support AI-specific workloads, enabling organizations to route, secure, and scale requests to large language models and other generative AI services. In a typical deployment, the architecture uses a two-tier gateway model where an outer gateway handles authentication, routing, and global rate limiting while a second gateway manages traffic to self-hosted model serving clusters. This design allows organizations to centralize control over AI service access while maintaining flexibility in how backend model infrastructure is deployed. The gateway provides policy enforcement, observability, and routing capabilities that are specifically designed for AI inference workloads, including intelligent endpoint selection and request optimization. -
3
AxonHub
Use any SDK to call 100+ LLMs
...The system also provides infrastructure features such as request routing, failover mechanisms, load balancing, and cost management for AI applications. This architecture makes it easier to experiment with multiple models and manage production deployments that rely on several providers simultaneously. -
4
MCP Router
A Unified MCP Server Management App (MCP Manager)
...This enables AI applications to access multiple tools and knowledge sources through a unified interface rather than connecting to each service individually. The project provides infrastructure for routing requests between clients and MCP servers, enabling scalable multi-tool agent systems. Developers building AI agents can use the platform to manage tool endpoints, control service availability, and simplify agent integration workflows. -
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
5
MoBA
MoBA: Mixture of Block Attention for Long-Context LLMs
...Instead of forcing each token to attend to every other token in the sequence, MoBA divides the context into blocks and dynamically routes queries to only the most relevant segments of information. This routing strategy reduces the computational cost associated with traditional attention while preserving performance on reasoning and long-context tasks. The approach allows language models to scale to significantly longer input contexts without the quadratic computational cost normally associated with transformer attention mechanisms. -
6
Chat UI
The open source codebase powering HuggingChat
...Chat UI connects to any service that exposes an OpenAI-compatible API endpoint, allowing it to work with a wide range of models and inference providers. The platform supports advanced capabilities such as multimodal input, tool integration through Model Context Protocol servers, and intelligent routing that selects the most appropriate model for each request. -
7
Kimi K2
Kimi K2 is the large language model series developed by Moonshot AI
...The model family includes variants like a foundational base model that researchers can fine-tune for specific use cases and an instruct-optimized variant primed for general-purpose chat and agent-style interactions, offering flexibility for both experimentation and deployment. With its high-dimensional attention mechanisms and expert routing, Kimi-K2 excels across benchmarks in live coding, math reasoning, and problem solving. -
8
mistral.rs
Fast, flexible LLM inference
...The project includes hardware-aware tooling that can benchmark a system and choose sensible quantization and device-mapping strategies, helping users get strong performance without manual tuning. It also supports serving multiple models from the same server process, enabling routing or quick switching between models depending on workload needs. For user-facing testing, mistral.rs can provide a built-in web UI, and it also offers a dedicated lightweight web chat interface that supports richer interaction patterns. -
9
Contrastive Language Models
Contrastive Language Models (CLMs) are a new class of System One model
...It includes inference servers, a browser playground, evaluation tools, preprocessing utilities, and fine-tuning scripts. The reference model uses a Qwen3-8B backbone with lightweight trainable projection heads. It targets routing, verification, tool selection, gaming, and other fast decision workloads. -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
10
Council of High Intelligence
18 AI personas deliberate your hardest decisions
Council of High Intelligence is an AI deliberation framework that lets multiple expert-style personas analyze hard decisions together. Instead of relying on one model response, it creates structured disagreement between different intellectual roles such as Socrates, Feynman, Kahneman, Torvalds, Aristotle, and others. The project supports multi-round reasoning, cross-examination, dissent quotas, novelty checks, and model diversity across different providers. It is designed for decisions where... -
11
Parallax
Parallax is a distributed model serving framework
Parallax is a decentralized inference framework designed to run large language models across distributed computing resources. Instead of relying on centralized GPU clusters in data centers, the system allows multiple heterogeneous machines to collaborate in serving AI inference workloads. Parallax divides model layers across different nodes and dynamically coordinates them to form a complete inference pipeline. A two-stage scheduling architecture determines how model layers are allocated to... -
12
Bedrock Chat
AWS-native chatbot using Bedrock
Bedrock Chat is a mirrored version of an open-source project that provides a conversational interface for interacting with large language models and AI services through a chat-style application. The project typically focuses on delivering a user interface that allows individuals or teams to communicate with AI models, manage conversations, and experiment with prompts and responses. Implementations like Bedrock Chat often integrate with model hosting platforms or APIs that provide access to... -
13
Paddler
Open-source LLM load balancer and serving platform for hosting LLMs
Paddler is an open-source LLM infrastructure platform designed to deploy, manage, and scale large language models on private infrastructure. The system acts as a specialized load balancer and serving layer for language models, enabling organizations to run inference workloads without relying on external API providers. It supports running models locally through engines such as llama.cpp while distributing requests across multiple compute nodes to improve performance and reliability. The... -
14
vLLM Semantic Router
System Level Intelligent Router for Mixture-of-Models at Cloud
Semantic Router is an open-source system designed to intelligently route requests across multiple large language models based on the semantic meaning and complexity of user queries. Instead of sending every prompt to the same model, the system analyzes the intent and reasoning requirements of the request and dynamically selects the most appropriate model to process it. This approach allows developers to combine multiple models with different strengths, such as lightweight models for simple... -
15
Flock
Flock is a workflow-based low-code platform for building chatbots
Flock is a workflow-based low-code platform designed for building AI applications such as chatbots, retrieval-augmented generation systems, and multi-agent workflows. The platform uses a visual workflow architecture where different nodes represent processing steps such as input processing, model inference, retrieval operations, and tool execution. Developers can connect these nodes to create complex pipelines that orchestrate multiple language models and external services. Built on... -
16
Cake
Distributed LLM and StableDiffusion inference
Cake is a compact, powerful toolkit that combines a flexible TCP/UDP proxy, port forwarding system, and connection manager designed for both development and penetration testing scenarios. It enables users to create complex networking flows where traffic can be proxied, relayed, and manipulated between endpoints — useful for debugging networked applications, inspecting protocols, or tunneling traffic through different hops. The tool is designed to work with multiple protocols and supports... -
17
MiniMax-01
Large-language-model & vision-language-model based on Linear Attention
MiniMax-01 is the official repository for two flagship models: MiniMax-Text-01, a long-context language model, and MiniMax-VL-01, a vision-language model built on top of it. MiniMax-Text-01 uses a hybrid attention architecture that blends Lightning Attention, standard softmax attention, and Mixture-of-Experts (MoE) routing to achieve both high throughput and long-context reasoning. It has 456 billion total parameters with 45.9 billion activated per token and is trained with advanced parallel strategies such as LASP+, varlen ring attention, and Expert Tensor Parallelism, enabling a training context of 1 million tokens and up to 4 million tokens at inference. ... -
18
OpenAI Forward
An efficient forwarding service designed for LLMs
...Its main purpose is to make model access more manageable and efficient by adding operational controls such as request rate limiting, token rate limiting, caching, logging, routing, and key management around existing LLM endpoints. The project can proxy both local and cloud-hosted language model services, which makes it useful for teams that want a single control layer regardless of whether they are using something like LocalAI or a hosted provider compatible with OpenAI-style APIs. A major emphasis of the repository is asynchronous performance, using tools such as uvicorn, aiohttp, and asyncio to support high-throughput forwarding workloads. -
19
LLaMA-MoE
Building Mixture-of-Experts from LLaMA with Continual Pre-training
LLaMA-MoE is an open-source project that builds mixture-of-experts language models from LLaMA through expert partitioning and continual pre-training. The repository is centered on making MoE research more accessible by offering smaller and more affordable models with only about 3.0 to 3.5 billion activated parameters, which helps reduce deployment and experimentation costs. Its architecture works by splitting LLaMA feed-forward networks into sparse experts and adding gating mechanisms so... -
20
ZAYA1-8B
Efficient MoE reasoning model for coding and math workloads
...ZAYA1-8B is optimized for long-form reasoning and test-time compute workflows, making it particularly effective for mathematical problem solving, coding tasks, and advanced reasoning chains. It introduces architectural innovations such as Compressed Convolutional Attention, a novel MLP-based expert router, and learned residual scaling to improve routing stability and inference efficiency. The model was trained entirely on AMD infrastructure and refined through supervised fine-tuning and multi-stage reinforcement learning focused on reasoning and coding.
- Previous
- You're on page 1
- Next
