Download
API-for-Open-LLM is a lightweight API server designed for deploying and serving open large language models (LLMs), offering a simple way to integrate LLMs into applications.
Features
- Provides a REST API for serving open LLMs
- Supports multiple backends, including Hugging Face models
- Enables GPU and CPU-based inference
- Offers token streaming for real-time responses
- Supports user authentication and request management
- Open-source and customizable for different use cases
Project Samples
License
Apache License V2.0Follow API-for-Open-LLM
Other Useful Business Software
Ship Agents Faster
Gemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
Rate This Project
Login To Rate This Project
User Reviews
Be the first to post a review of API-for-Open-LLM!
