Openai style api for open large language models
Run Local LLMs on Any Device. Open-source
Ready-to-use OCR with 80+ supported languages
Everything you need to build state-of-the-art foundation models
A high-throughput and memory-efficient inference and serving engine
Uncover insights, surface problems, monitor, and fine tune your LLM
GPU environment management and cluster orchestration
Library for OCR-related tasks powered by Deep Learning
The Triton Inference Server provides an optimized cloud
State-of-the-art diffusion models for image and audio generation
Data manipulation and transformation for audio signal processing
Official inference library for Mistral models
The official Python client for the Huggingface Hub
Single-cell analysis in Python
Pytorch domain library for recommendation systems
Training and deploying machine learning models on Amazon SageMaker
Bring the notion of Model-as-a-Service to life
A toolkit to optimize ML models for deployment for Keras & TensorFlow
Unified Model Serving Framework
Trainable models and NN optimization tools
A set of Docker images for training and serving models in TensorFlow
Optimizing inference proxy for LLMs
LMDeploy is a toolkit for compressing, deploying, and serving LLMs
Neural Network Compression Framework for enhanced OpenVINO
Operating LLMs in production