Moondream Integrations

Photon is Moondream’s official high-performance inference engine, designed to run vision-language models efficiently across cloud, desktop, and edge environments while delivering real-time performance for production AI systems. It is built as a custom inference layer tightly integrated with the Moondream model architecture, using optimized scheduling, native image processing, and purpose-built CUDA kernels to maximize speed and efficiency. This co-designed approach allows Photon to significantly reduce latency compared to traditional VLM setups, enabling responsive interactions on edge devices and real-time throughput on server-grade hardware. It supports deployment across a wide range of NVIDIA GPUs, from embedded systems like Jetson devices to high-end multi-GPU servers, making it adaptable for diverse operational needs. It includes production-ready features such as automatic batching, prefix caching, and memory-efficient attention mechanisms.

Lens is Moondream’s official fine-tuning service, designed to turn a general vision-language model into a highly specialized system tailored to a specific task. It provides a simple, structured workflow where users start by collecting a small dataset of images relevant to their use case, then fine-tune the model through an API using techniques such as supervised fine-tuning (SFT) or reinforcement learning, and finally deploy the customized model either through the cloud or locally with Photon. It is built around the idea that Moondream begins as a general model trained on broad, public data, and fine-tuning adapts it to understand the exact products, documents, categories, or internal information that matter to a business, significantly improving accuracy and reliability for that domain. Lens is designed for production scenarios where performance matters, enabling teams to achieve large gains in accuracy with minimal data by teaching the model to master a defined task.

Vision Agents is an open source Python framework for building low-latency voice and video AI agents with any model. It lets developers plug in LLM, speech, and vision models from more than 25 providers and ship real-time agents for telehealth, voice support, live coaching, video analysis, interactive avatars, security monitoring, sports commentary, and other multimodal applications. It is designed to help teams build agents that can listen, speak, see, process media, call tools, and respond in real time while running on Stream’s global edge network with sub-500ms latency. Developers can build a first agent in minutes, using a small Python setup with Gemini Realtime, OpenAI, Deepgram, ElevenLabs, Stream, or other supported providers. Vision Agents supports both real-time speech-to-speech models and custom STT/LLM/TTS pipelines, giving teams either the fastest path to a working voice agent or full control over speech recognition, language reasoning, text-to-speech, etc.