Call all LLM APIs using the OpenAI format [Anthropic, Huggingface, Cohere, Azure OpenAI etc.] liteLLM supports streaming the model response back, pass stream=True to get a streaming iterator in response. Streaming is supported for OpenAI, Azure, Anthropic, and Huggingface models.

Features

  • Translating inputs to the provider's completion and embedding endpoints
  • Guarantees consistent output, text responses will always be available
  • Exception mapping
  • Common exceptions across providers are mapped to the OpenAI exception types
  • LiteLLM Client - debugging & 1-click add new LLMs
  • liteLLM supports streaming the model response back

Project Samples

LiteLLM Screenshot 1

Project Activity

See All Activity >

Categories

Large Language Models (LLM), MCP Gateways

License

MIT License

Follow LiteLLM

LiteLLM Web Site

Other Useful Business Software
$300 Free Credits to Build on Google Cloud Icon
$300 Free Credits to Build on Google Cloud

New customers can spin up VMs, build with AI, and query data at no cost.

Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
Start Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of LiteLLM!

Additional Project Details

Programming Language

Python

Related Categories

Python Large Language Models (LLM), Python MCP Gateways

Registered

2023-08-25
Report inappropriate content