User-friendly AI Interface
Port of OpenAI's Whisper model in C/C++
Port of Facebook's LLaMA model in C/C++
Run Local LLMs on Any Device. Open-source
FlashInfer: Kernel Library for LLM Serving
A high-throughput and memory-efficient inference and serving engine
ONNX Runtime: cross-platform, high performance ML inferencing
OpenVINO™ Toolkit repository
The free, Open Source alternative to OpenAI, Claude and others
High-performance neural network inference framework for mobile
Open-Source AI Camera. Empower any camera/CCTV
Protect and discover secrets using Gitleaks
Pure C++ implementation of several models for real-time chatting
Open standard for machine learning interoperability
Operating LLMs in production
Neural Network Compression Framework for enhanced OpenVINO
Official inference library for Mistral models
Efficient few-shot learning with Sentence Transformers
LLMs as Copilots for Theorem Proving in Lean
An MLOps framework to package, deploy, monitor and manage models
On-device AI across mobile, embedded and edge for PyTorch
LLM training code for MosaicML foundation models
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Everything you need to build state-of-the-art foundation models
LLM.swift is a simple and readable library