Download
API-for-Open-LLM is a lightweight API server designed for deploying and serving open large language models (LLMs), offering a simple way to integrate LLMs into applications.
Features
- Provides a REST API for serving open LLMs
- Supports multiple backends, including Hugging Face models
- Enables GPU and CPU-based inference
- Offers token streaming for real-time responses
- Supports user authentication and request management
- Open-source and customizable for different use cases
Project Samples
License
Apache License V2.0Follow API-for-Open-LLM
Other Useful Business Software
Earn up to 16% annual interest with Nexo.
Generate interest, borrow against your crypto, and trade a range of cryptocurrencies — all in one platform.
Geographic restrictions, eligibility, and terms apply.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of API-for-Open-LLM!
