Download
JoyAI-VL-Interaction is an open real-time video-language interaction system built around an 8B-scale vision-first model. It is designed to watch a webcam or livestream continuously and decide whether to speak, stay silent, or delegate a harder task. Unlike turn-based assistants, it focuses on event-driven interaction where timing matters as much as answer quality. The repository releases the model, training recipe, time-aligned interaction data, and deployable system together. Its system includes inference, WebUI, ASR, TTS, and background-agent services running on vLLM-based infrastructure. It is useful for real-time monitoring, live commentary, cooking guidance, game calling, visual alerts, and other scenarios where an AI should respond at the right moment.
Features
- 8B real-time vision-language model
- Continuous video stream monitoring
- Vision-triggered proactive responses
- Speak, silent, and delegate decisions
- ASR, TTS, WebUI, and agent services
- Open model, data, recipe, and system stack
Project Samples
