Port of Facebook's LLaMA model in C/C++
Run Local LLMs on Any Device. Open-source
C++ library for high performance inference on NVIDIA GPUs
C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)
Lightweight inference library for ONNX files, written in C++
OpenMLDB is an open-source machine learning database
Guide to deploying deep-learning inference networks
Deep learning inference framework optimized for mobile platforms