Run Local LLMs on Any Device. Open-source
Openai style api for open large language models
Library for OCR-related tasks powered by Deep Learning
A high-performance ML model serving framework, offers dynamic batching
Tensor search for humans
An easy-to-use LLMs quantization package with user-friendly apis
Database system for building simpler and faster AI-powered application
A computer vision framework to create and deploy apps in minutes
LLMFlows - Simple, Explicit and Transparent LLM Apps
CPU/GPU inference server for Hugging Face transformer models