Port of OpenAI's Whisper model in C/C++
Training and deploying machine learning models on Amazon SageMaker
Run Local LLMs on Any Device. Open-source
Port of Facebook's LLaMA model in C/C++
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
Ready-to-use OCR with 80+ supported languages
C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)
AIMET is a library that provides advanced quantization and compression
Phi-3.5 for Mac: Locally-run Vision and Language Models
LMDeploy is a toolkit for compressing, deploying, and serving LLMs
Easy-to-use deep learning framework with 3 key features
An MLOps framework to package, deploy, monitor and manage models
A high-throughput and memory-efficient inference and serving engine
Sparsity-aware deep learning inference runtime for CPUs
OpenMLDB is an open-source machine learning database
Optimizing inference proxy for LLMs
Single-cell analysis in Python
A library for accelerating Transformer models on NVIDIA GPUs
PyTorch library of curated Transformer models and their components
Uncover insights, surface problems, monitor, and fine tune your LLM
FlashInfer: Kernel Library for LLM Serving
Database system for building simpler and faster AI-powered application
Large Language Model Text Generation Inference
Superduper: Integrate AI models and machine learning workflows
OpenMMLab Model Deployment Framework