RouteLLM

RouteLLM

LMSYS
+
+

Related Products

  • Google AI Studio
    4 Ratings
    Visit Website
  • Amazon API Gateway
    714 Ratings
    Visit Website
  • Site24x7
    750 Ratings
    Visit Website
  • Google Kubernetes Engine (GKE)
    424 Ratings
    Visit Website
  • Pepperi
    141 Ratings
    Visit Website
  • Boomi
    839 Ratings
    Visit Website
  • Control D
    180 Ratings
    Visit Website
  • AI Video Cut
    1 Rating
    Visit Website
  • groundcover
    32 Ratings
    Visit Website
  • Windsurf Editor
    137 Ratings
    Visit Website

About

​Arch is an intelligent gateway designed to protect, observe, and personalize AI agents through seamless integration with your APIs. Built on Envoy Proxy, Arch offers secure handling, intelligent routing, robust observability, and integration with backend systems, all external to business logic. It features an out-of-process architecture compatible with various application languages, enabling quick deployment and transparent upgrades. Engineered with specialized sub-billion parameter Large Language Models (LLMs), Arch excels in critical prompt-related tasks such as function calling for API personalization, prompt guards to prevent toxic or jailbreak prompts, and intent-drift detection to enhance retrieval accuracy and response efficiency. Arch extends Envoy's cluster subsystem to manage upstream connections to LLMs, providing resilient AI application development. It also serves as an edge gateway for AI applications, offering TLS termination, rate limiting, and prompt-based routing.

About

Developed by LM-SYS, RouteLLM is an open-source toolkit that allows users to route tasks between different large language models to improve efficiency and manage resources. It supports strategy-based routing, helping developers balance speed, accuracy, and cost by selecting the best model for each input dynamically.

Platforms Supported

Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook

Platforms Supported

Windows
Mac
Linux
Cloud
On-Premises
iPhone
iPad
Android
Chromebook

Audience

Enterprises interested in a solution to build AI apps and integrate AI agents securely into their systems

Audience

AI developers and researchers looking to optimize workflows using multiple LLMs through intelligent routing and performance-aware model selection

Support

Phone Support
24/7 Live Support
Online

Support

Phone Support
24/7 Live Support
Online

API

Offers API

API

Offers API

Screenshots and Videos

Screenshots and Videos

Pricing

Free
Free Version
Free Trial

Pricing

No information available.
Free Version
Free Trial

Reviews/Ratings

Overall 0.0 / 5
ease 0.0 / 5
features 0.0 / 5
design 0.0 / 5
support 0.0 / 5

This software hasn't been reviewed yet. Be the first to provide a review:

Review this Software

Reviews/Ratings

Overall 0.0 / 5
ease 0.0 / 5
features 0.0 / 5
design 0.0 / 5
support 0.0 / 5

This software hasn't been reviewed yet. Be the first to provide a review:

Review this Software

Training

Documentation
Webinars
Live Online
In Person

Training

Documentation
Webinars
Live Online
In Person

Company Information

Arch
United States
www.archgw.com

Company Information

LMSYS
github.com/lm-sys/RouteLLM

Alternatives

Alternatives

Kong AI Gateway

Kong AI Gateway

Kong Inc.

Categories

Categories

Integrations

OpenAI
Amazon Bedrock
Amazon Web Services (AWS)
Anyscale
Chatbot Arena
Envoy
GPT-4
Gemini
GitHub
Honeycomb
Jaeger
Llama
Mistral AI
Mixtral 8x7B
Ollama
OpenTelemetry
Python
SigNoz
Taam Cloud
Together AI

Integrations

OpenAI
Amazon Bedrock
Amazon Web Services (AWS)
Anyscale
Chatbot Arena
Envoy
GPT-4
Gemini
GitHub
Honeycomb
Jaeger
Llama
Mistral AI
Mixtral 8x7B
Ollama
OpenTelemetry
Python
SigNoz
Taam Cloud
Together AI
Claim Arch and update features and information
Claim Arch and update features and information
Claim RouteLLM and update features and information
Claim RouteLLM and update features and information