BaseRT
BaseRT is a high-performance LLM inference runtime for Apple Silicon that lets developers pull models from Hugging Face, chat with them locally, or serve an OpenAI-compatible API from one CLI. Accelerated by hand-written Metal kernels, it is designed to deliver fast prefill and decode performance on M-series Macs, with published benchmarks showing up to 6.4× faster prefill than llama.cpp, 3.9× faster than MLX, and up to 1.33× faster decode. The basert CLI handles model downloading, conversion, interactive chat, serving, completion, benchmarking, inspection, and bundle signing. Its server supports chat, completions, embeddings, transcription, tool calls, continuous batching, paged KV caching, and prefix caching, while supported models can process text, vision, and audio. BaseRT uses its own .base model format with Q2–Q8 affine quantization, optional AWQ calibration, and signed bundles, and can convert GGUF, Hugging Face, and MLX checkpoints.
Learn more
Private LLM
Private LLM is a local AI chatbot for iOS and macOS that works offline, keeping your information completely on-device, safe, and private. It doesn't need the internet to work, so your data never leaves your device. It stays just with you. With no subscription fees, you pay once and use it on all your Apple devices. It's designed for everyone, with easy-to-use features for generating text, helping with language, and a whole lot more. Private LLM uses the latest AI models quantized with state-of-the-art quantization techniques to provide a high-quality on-device AI experience without compromising your privacy. It's a smart, secure way to get creative and productive, anytime and anywhere. Private LLM opens the door to the vast possibilities of AI with support for an extensive selection of open-source LLM models, including the Llama 3, Google Gemma, Microsoft Phi-2, Mixtral 8x7B family and many more on both your iPhones, iPads and Macs.
Learn more
Google AI Edge Gallery
Google AI Edge Gallery is an experimental, open source Android app that demonstrates on-device machine learning and generative AI use cases, letting users download and run models locally (so they work offline once installed). It offers several features including AI Chat (multi-turn conversation), Ask Image (upload or use images to ask questions, identify objects, get descriptions), Audio Scribe (transcribe or translate recorded/uploaded audio), Prompt Lab (for single-turn tasks such as summarization, rewriting, code generation), and performance insights (metrics like latency, decode speed, etc.). Users can switch between different compatible models (including Gemma 3n and models from Hugging Face), bring their own LiteRT models, and explore model cards and source code for transparency. The app aims to protect privacy by doing all processing on the device, no internet connection needed for core operations after models are loaded, reducing latency, and enhancing data security.
Learn more
Private Mind
Private Mind is an on-device AI assistant that works entirely offline, giving users local AI with total privacy. It is built around the belief that AI should live on the user’s device, with conversations, files, prompts, and data staying local instead of being sent to the cloud. Users can chat with the assistant without Wi-Fi, sign-ups, tracking, or cloud dependency, making it useful for planning trips, translating text, brainstorming ideas, analyzing data, learning new things, or getting help when internet access is unavailable. Private Mind supports chat with files, allowing users to interact with their own documents through on-device AI and intelligent retrieval without sending private material outside the device. It also includes speech-to-text, so users can speak naturally and get instant local transcriptions using Whisper. It supports multiple open-source AI models.
Learn more
