Search Results for "load balancing"
Sort By:
13 projects for "load balancing" with 2 filters applied:
-
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
Veeam Data Platform v13.1 - Get Your Free TrialTry Veeam Data Platform today. Experience the unified platform that's secure by design, portable by default, and proven to recover clean, fast, and anywhere.
-
1
AxonHub
Use any SDK to call 100+ LLMs
...AxonHub translates requests from one provider’s API format into another, enabling seamless interoperability across different AI platforms. The system also provides infrastructure features such as request routing, failover mechanisms, load balancing, and cost management for AI applications. This architecture makes it easier to experiment with multiple models and manage production deployments that rely on several providers simultaneously. -
2
ClawRouter
Smart LLM router
ClawRouter is a flexible networking and routing framework designed to support AI-oriented distributed systems and agent ecosystems by managing how messages, requests, and responses are routed between components. It provides a programmable router abstraction that can handle complex traffic patterns, enabling dynamic message forwarding, load balancing, and custom routing logic based on content, context, or policy rules. Because distributed AI systems often involve many services, agents, and runtime components interacting with each other and with external APIs, ClawRouter helps ensure that communication paths remain clear, efficient, and adaptable as systems scale. The framework supports plugin-based extensions so developers can define custom protocols, transformation hooks, and monitoring handlers without modifying core routing logic. ... -
3
DeepSeek-V3
Powerful AI language model (MoE) optimized for efficiency/performance
DeepSeek-V3 is a robust Mixture-of-Experts (MoE) language model developed by DeepSeek, featuring a total of 671 billion parameters, with 37 billion activated per token. It employs Multi-head Latent Attention (MLA) and the DeepSeekMoE architecture to enhance computational efficiency. The model introduces an auxiliary-loss-free load balancing strategy and a multi-token prediction training objective to boost performance. Trained on 14.8 trillion diverse, high-quality tokens, DeepSeek-V3 underwent supervised fine-tuning and reinforcement learning to fully realize its capabilities. Evaluations indicate that it outperforms other open-source models and rivals leading closed-source models, achieving this with a training duration of 55 days on 2,048 Nvidia H800 GPUs, costing approximately $5.58 million. -
4
Paddler
Open-source LLM load balancer and serving platform for hosting LLMs
Paddler is an open-source LLM infrastructure platform designed to deploy, manage, and scale large language models on private infrastructure. The system acts as a specialized load balancer and serving layer for language models, enabling organizations to run inference workloads without relying on external API providers. It supports running models locally through engines such as llama.cpp while distributing requests across multiple compute nodes to improve performance and reliability. The... -
$300 Free Credits to Build on Google CloudPut your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
-
5
System Design Academy
If you want to become good at AI & system design, join this newsletter
...Real-world examples examine how major services scale storage, messaging, streaming, payments, collaboration, and other workloads. Fundamental topics include APIs, caching, consistency, databases, DNS, Docker, Kafka, Kubernetes, load balancing, microservices, and service discovery. Separate guides cover interview frameworks and design exercises for products such as ChatGPT, Spotify, WhatsApp, and YouTube. The repository also collects AI agent concepts and selected software architecture white papers. -
6
One API
The LLM API management & distribution system
One API is an open-source platform designed to unify access to multiple AI model providers through a single, consistent API interface. It acts as a gateway that allows developers to manage and route requests to different large language models and AI services without needing to integrate each provider separately. The system supports multiple backends, enabling users to switch between providers or balance usage based on cost, performance, or availability. It includes features such as... -
7
Bifrost
The Fastest LLM Gateway with built in OTel observability
...It is built to be high performance: in benchmark tests at 5,000 requests per second, it reportedly adds only microseconds of overhead and achieves perfect success rates with no failed requests. Bifrost supports features such as automatic fallback (failover between providers), load balancing across API keys/providers, and semantic caching to reduce latency and cost. It also includes observability with built-in metrics, tracing, logging, and supports governance features like rate limiting, access control, and cost budgeting. The architecture is modular: there is a core engine, plugin layers, and transport layers (HTTP APIs). -
8
Portkey AI Gateway
A blazing fast AI Gateway with integrated guardrails
...It presents a single, friendly API through which you can route to 200+ LLMs, while applying configurable input/output guardrails to enforce policies or restrict certain content. It supports automatic retries, fallbacks, load balancing across providers or keys, and request timeouts to avoid latency spikes. The gateway is multimodal: it can handle text, vision, audio, and image models under a common interface. It also offers features for governance: role-based access, compliance with standards (SOC2, HIPAA, GDPR), secure key management, and logging/analytics of usage, latency, errors, and cost. ... -
9
Swarms
Enterprise multi-agent orchestration framework for scalable AI apps
Swarms is an enterprise-grade multi-agent orchestration framework designed to help developers build, manage, and scale collaborative AI systems composed of multiple agents. It provides a structured infrastructure for coordinating agents in hierarchical, parallel, or sequential workflows, enabling complex task execution across distributed components. It emphasizes production readiness, offering modular architecture, high availability, and observability features suitable for large-scale... -
Custom VMs From 1 to 96 vCPUs With 99.95% UptimeLive migration and automatic failover keep workloads online through maintenance. One free e2-micro VM every month.
-
10
Parallax
Parallax is a distributed model serving framework
Parallax is a decentralized inference framework designed to run large language models across distributed computing resources. Instead of relying on centralized GPU clusters in data centers, the system allows multiple heterogeneous machines to collaborate in serving AI inference workloads. Parallax divides model layers across different nodes and dynamically coordinates them to form a complete inference pipeline. A two-stage scheduling architecture determines how model layers are allocated to... -
11
Envoy AI Gateway
Manages Unified Access to Generative AI Services
Envoy AI Gateway is an open-source gateway system designed to manage network traffic between applications and generative AI services using the Envoy proxy ecosystem. The project extends Envoy Gateway to support AI-specific workloads, enabling organizations to route, secure, and scale requests to large language models and other generative AI services. In a typical deployment, the architecture uses a two-tier gateway model where an outer gateway handles authentication, routing, and global rate... -
12
GPTLink
Build a free commercial AI dialogue environment in 10 minutes
GPTLink is an open-source platform designed to manage, proxy, and distribute access to large language model APIs, enabling organizations and developers to centralize AI usage and optimize resource allocation. It acts as an intermediary layer between users and multiple AI providers, allowing requests to be routed, balanced, and monitored through a unified interface. The system supports API key management, usage tracking, and rate limiting, making it particularly useful for teams that need to... -
13
Agent.GUI
The Project moved to github https://github.com/EnFlexIT/AgentWorkbench
The project has moved to github https://github.com/EnFlexIT/AgentWorkbench Agent.GUI is a simulation framework and toolkit based on the JADE framework. It provides functionalities for time aspects, agent-environment interaction, visualization and load balancing, Furthermore, the included application focuses the usability for end users.
- Previous
- You're on page 1
- Next