Run Local LLMs on Any Device. Open-source
Port of Facebook's LLaMA model in C/C++
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
C++ implementation of ChatGLM-6B & ChatGLM2-6B & ChatGLM3 & GLM4(V)
Phi-3.5 for Mac: Locally-run Vision and Language Models
LMDeploy is a toolkit for compressing, deploying, and serving LLMs
Sparsity-aware deep learning inference runtime for CPUs
A high-throughput and memory-efficient inference and serving engine
Database system for building simpler and faster AI-powered application
Optimizing inference proxy for LLMs
Large Language Model Text Generation Inference
PyTorch library of curated Transformer models and their components
Operating LLMs in production
Run 100B+ language models at home, BitTorrent-style
State-of-the-art Parameter-Efficient Fine-Tuning
DoWhy is a Python library for causal inference
Replace OpenAI GPT with another LLM in your app
Visual Instruction Tuning: Large Language-and-Vision Assistant
LLM training code for MosaicML foundation models
A high-performance ML model serving framework, offers dynamic batching
Bring the notion of Model-as-a-Service to life
Run any Llama 2 locally with gradio UI on GPU or CPU from anywhere
Open platform for training, serving, and evaluating language models
Easiest and laziest way for building multi-agent LLMs applications
Tensor search for humans