Openai style api for open large language models
Run Local LLMs on Any Device. Open-source
FlashInfer: Kernel Library for LLM Serving
Ready-to-use OCR with 80+ supported languages
A high-throughput and memory-efficient inference and serving engine
Multilingual Automatic Speech Recognition with word-level timestamps
The official Python client for the Huggingface Hub
Everything you need to build state-of-the-art foundation models
GPU environment management and cluster orchestration
Operating LLMs in production
Uncover insights, surface problems, monitor, and fine tune your LLM
A library for accelerating Transformer models on NVIDIA GPUs
LLM training code for MosaicML foundation models
Single-cell analysis in Python
Trainable, memory-efficient, and GPU-friendly PyTorch reproduction
Replace OpenAI GPT with another LLM in your app
The Triton Inference Server provides an optimized cloud
Training and deploying machine learning models on Amazon SageMaker
Optimizing inference proxy for LLMs
Library for OCR-related tasks powered by Deep Learning
Create HTML profiling reports from pandas DataFrame objects
Superduper: Integrate AI models and machine learning workflows
LMDeploy is a toolkit for compressing, deploying, and serving LLMs
A set of Docker images for training and serving models in TensorFlow
Phi-3.5 for Mac: Locally-run Vision and Language Models