Redundancy-aware KV Cache Compression for Reasoning Models
Unified KV Cache Compression Methods for Auto-Regressive Models
Cache-Augmented Generation: A Simple, Efficient Alternative to RAG
LLM inference server with continuous batching & SSD caching
Supercharge Your LLM with the Fastest KV Cache Layer
Trainable latent-memory framework for 100M-token contexts
Claude + Obsidian knowledge companion
High-performance Inference and Deployment Toolkit for LLMs and VLMs
Claude Code, but it runs on your Mac for free
TensorRT LLM provides users with an easy-to-use Python API
Build a modern LLM from scratch. Every line commented
Bring the notion of Model-as-a-Service to life
RGBD video generation model conditioned on camera input
A Model Context Protocol (MCP) Gateway & Registry
A course of learning LLM inference serving on Apple Silicon
AI Desktop Translator & Universal Voice Typing for Windows (5 Engines)
Semantic cache for LLMs. Fully integrated with LangChain
VITS2 backbone with multilingual-bert
RNN with great LLM performance
A webui for different audio related Neural Networks
A modern, web-based photo management server
fastNLP: A Modularized and Extensible NLP Framework