Download
API-for-Open-LLM is a lightweight API server designed for deploying and serving open large language models (LLMs), offering a simple way to integrate LLMs into applications.
Features
- Provides a REST API for serving open LLMs
- Supports multiple backends, including Hugging Face models
- Enables GPU and CPU-based inference
- Offers token streaming for real-time responses
- Supports user authentication and request management
- Open-source and customizable for different use cases
Project Samples
License
Apache License V2.0Follow API-for-Open-LLM
Other Useful Business Software
Go From AI Idea to AI App Fast
Access Gemini 3 and 200+ models. Build chatbots, agents, or custom models with built-in monitoring and scaling.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of API-for-Open-LLM!
