BaseRT
BaseRT is a high-performance LLM inference runtime for Apple Silicon that lets developers pull models from Hugging Face, chat with them locally, or serve an OpenAI-compatible API from one CLI. Accelerated by hand-written Metal kernels, it is designed to deliver fast prefill and decode performance on M-series Macs, with published benchmarks showing up to 6.4× faster prefill than llama.cpp, 3.9× faster than MLX, and up to 1.33× faster decode. The basert CLI handles model downloading, conversion, interactive chat, serving, completion, benchmarking, inspection, and bundle signing. Its server supports chat, completions, embeddings, transcription, tool calls, continuous batching, paged KV caching, and prefix caching, while supported models can process text, vision, and audio. BaseRT uses its own .base model format with Q2–Q8 affine quantization, optional AWQ calibration, and signed bundles, and can convert GGUF, Hugging Face, and MLX checkpoints.
Learn more
Liquid Apollo
Liquid Apollo is a lightweight mobile application designed for fully on-device, cloud-free AI interactions, enabling users to engage with advanced language and vision models securely, privately, and with low latency. It supports a library of small foundation models from the company’s LEAP platform, allowing users to draft messages, emails, chat with a private AI assistant, craft digital characters, or use image-to-text capabilities, all without an internet connection and with no data leaving the device. Apollo is optimized for real-time responsiveness and offline operation, ensuring that inference happens entirely locally, with no API calls, servers, or user-data logging involved. It serves as both a personal AI playground and a testing bed for developers using LEAP models, letting one “vibe-check” how a model performs on their own mobile hardware before broader deployment.
Learn more
Private Mind
Private Mind is an on-device AI assistant that works entirely offline, giving users local AI with total privacy. It is built around the belief that AI should live on the user’s device, with conversations, files, prompts, and data staying local instead of being sent to the cloud. Users can chat with the assistant without Wi-Fi, sign-ups, tracking, or cloud dependency, making it useful for planning trips, translating text, brainstorming ideas, analyzing data, learning new things, or getting help when internet access is unavailable. Private Mind supports chat with files, allowing users to interact with their own documents through on-device AI and intelligent retrieval without sending private material outside the device. It also includes speech-to-text, so users can speak naturally and get instant local transcriptions using Whisper. It supports multiple open-source AI models.
Learn more
Google AI Edge Gallery
Google AI Edge Gallery is an experimental, open source Android app that demonstrates on-device machine learning and generative AI use cases, letting users download and run models locally (so they work offline once installed). It offers several features including AI Chat (multi-turn conversation), Ask Image (upload or use images to ask questions, identify objects, get descriptions), Audio Scribe (transcribe or translate recorded/uploaded audio), Prompt Lab (for single-turn tasks such as summarization, rewriting, code generation), and performance insights (metrics like latency, decode speed, etc.). Users can switch between different compatible models (including Gemma 3n and models from Hugging Face), bring their own LiteRT models, and explore model cards and source code for transparency. The app aims to protect privacy by doing all processing on the device, no internet connection needed for core operations after models are loaded, reducing latency, and enhancing data security.
Learn more
