Search Results for "routing"

...It helps teams secure and optimize their LLM deployments, whether using local models or external APIs like OpenAI or Anthropic. With native support for multi-tenant environments and low-latency inference routing, AI Gateway is an essential tool for companies building production-grade generative AI services.

The world's fastest AI Gateway proxy, written in Rust and optimized for maximum performance. This high-performance API gateway routes requests to various AI providers (OpenAI, GROQ) with streaming support, making it perfect for developers who need reliable and blazing-fast AI API access.

Bifrost is an LLM gateway designed to provide a unified OpenAI-compatible API front for many different model providers. It abstracts away the complexity of working directly with multiple backend providers (OpenAI, Anthropic, AWS Bedrock, Google Vertex, etc.), enabling you to plug in providers and switch between them without touching your client code. It is built to be high performance: in benchmark tests at 5,000 requests per second, it reportedly adds only microseconds of overhead and...

Kaeso is an open-source AI gateway you run yourself. Applications talk to a single OpenAI- and Anthropic-compatible API while you keep full control over provider keys, routing, usage limits, and logging. Self-host with Docker and manage everything through a simple web console.

...The best open-source alternative to LiteLLM, GoModel provides a unified OpenAI-compatible API for OpenAI, Anthropic, Gemini, Groq, xAI, Ollama, vLLM, Amazon Bedrock, OpenRouter, and many more providers. Built in Go, it's lightweight, resource-efficient, and designed for production workloads with extremely low latency. GoModel includes intelligent request routing, observability, guardrails, live logs, usage and cost tracking, rate limiting, virtual models, prompt rewriting, streaming support, and enterprise-ready authentication. Whether you're building AI applications, running coding agents, or deploying LLM infrastructure at scale, GoModel gives you a fast, reliable, and self-hostable foundation. ...