Download
Harbor is an open source, containerized toolkit designed to simplify running local large language model (LLM) environments. It combines a CLI and companion app to launch backends, frontends, and supporting services with minimal setup. With a single command, users can start preconfigured tools like Ollama and Open WebUI, enabling chat, workflows, and integrations immediately. Harbor supports multiple inference engines, including llama.cpp and vLLM, and connects them seamlessly to user interfaces. It also includes tools for web retrieval, image generation, voice interaction, and workflow automation. Built on Docker, Harbor allows services to run in isolated containers while communicating over a local network. It is intended for local development and experimentation rather than production deployment, giving developers a flexible way to explore AI systems, test configurations, and manage complex LLM stacks without manual wiring or setup overhead.
Features
- One-command setup for complete local LLM stack
- Supports multiple inference engines like Ollama, llama.cpp, vLLM
- Integrated frontends such as Open WebUI for chat and workflows
- Built-in tools for web RAG, voice (TTS/STT), and image generation
- Containerized architecture using Docker for modular services
- CLI and companion app for managing and configuring environments
Project Samples
