Run Local LLMs on Any Device. Open-source
Unified Model Serving Framework
AIMET is a library that provides advanced quantization and compression
Library for serving Transformers models on Amazon SageMaker
Official inference library for Mistral models
Library for OCR-related tasks powered by Deep Learning
Deep learning optimization library: makes distributed training easy
A unified framework for scalable computing
Toolkit for allowing inference and serving with MXNet in SageMaker