MNN is a blazing fast, lightweight deep learning framework
High-performance neural network inference framework for mobile
Port of OpenAI's Whisper model in C/C++
Run Local LLMs on Any Device. Open-source
Set of comprehensive computer vision & machine intelligence libraries
C++ library for high performance inference on NVIDIA GPUs
ONNX Runtime: cross-platform, high performance ML inferencing
Fast inference engine for Transformer models
A GPU-accelerated library containing highly optimized building blocks
Serving system for machine learning models
Deep Learning API and Server in C++14 support for Caffe, PyTorch
Easy-to-use deep learning framework with 3 key features
Guide to deploying deep-learning inference networks
Deep learning inference framework optimized for mobile platforms
Uniform deep learning inference framework for mobile