TuyaOpen 1.6.0 Release Notes

TuyaOpen 1.6.0 is officially released! This update brings a major upgrade to Tuya AI APIs to V2.1, with comprehensive enhancements to core capabilities: new OPUS encoding support and Raspberry Pi 4/5 device adaptation, along with deep architectural refactoring that significantly optimizes audio/video processing and multimodal interaction performance, covering three core modules: multimodal, device MCP, and multimedia.

Additionally, AI-related APIs have been refactored and upgraded to create a more lightweight edge-side AI interaction capability library, lowering the integration barrier for developers. The supporting development toolchain has been improved, with enriched demo examples to help developers get started quickly and develop efficiently.

AI APIs Update

• Support for OPUS audio encoding format reporting, significantly improving transmission efficiency and reducing AI interaction latency
• Support for video, image, and file reporting, enabling multimodal AI-Agent model interaction
• Optimized voice interruption response speed for a more natural conversation system
• Refactored AI application component implementation framework, organizing the functionality of each module

New Features

• Agent-triggered music/story playback functionality
• Support for device MCP with four built-in tools, allowing agents to actively trigger devices (query device information/switch modes/adjust volume/take photos)
• Simple device MCP registration API, enabling developers to quickly extend hardware MCP capabilities
• Button switching between multiple chat modes (long-press conversation mode, button conversation mode, wake-up conversation mode, free conversation mode)
• Support for displaying camera-captured images on devices and sending photos to AI for multimodal analysis
• Support for custom mode registration, allowing flexible customization of voice conversation strategies
• Support for custom UI style registration, enabling developers to quickly customize their AI interaction UI/UX

Component Structure

:::txt
ai_main (Main Controller)
    ├── ai_agent (Cloud Communication)
    ├── ai_mode (Conversation Mode Management)
    ├── ai_audio (Audio Processing)
    ├── ai_ui (Interface Display)
    ├── ai_video (Video Processing)
    ├── ai_skills (Skill Processing)
    ├── ai_mcp (MCP Protocol)
    ├── assets (Resource Files)
    └── utility (Utility Classes)

New audio_player Audio Playback Service Component

Multiple Audio Stream Data Sources

• Memory buffer (MEM)
• Local files (FILE)
• Network streams (URL/HTTP)

Multiple Audio Decoding Formats

• MP3, WAV, Speex
• Opus, OGG Opus

Dual Audio Stream Playback (AI Conversation/Multimedia)

• Foreground Stream: For voice prompts, warnings, etc.
• Background Stream: For background music, etc.
• Support for preemption mode (foreground interrupts background) and mixer mode (simultaneous playback)

Audio Processing Components

• Decoder: Decodes compressed audio to PCM
• Resampling: Supports sample rate conversion
• Mixer: Supports multi-channel audio mixing
• Volume Controls: Volume control

Playback Management

• Create/add/delete playback items
• Previous/next track switching
• Loop playback (single track/playlist)
• Auto-play
• Playback controls: play/pause/resume/stop

Display Management Driver - New Features

New Frame Buffer Management

• New graphics drawing function interfaces (point drawing, matrix fill, full-screen fill)
• New image rotation interfaces (90°, 180°, 270°)

Format Conversion

• Pixel format conversion (RGB565 ↔ RGB888 ↔ RGB666, etc.)
• YUV422 to frame buffer conversion
• Monochrome conversion (supports multiple dithering algorithms)

New Graphics Accelerator (tal_dma2d) Driver

• Hardware-accelerated image format conversion, e.g., YUV422 -> RGB565/RGB888
• Hardware-accelerated 2D memory copy for image transmission and replication

Resource Management

• Multi-context support (can create multiple handles)
• Thread-safe (mutex protection)
• Hardware resource reuse (single hardware instance, multiple logical handles)

lvgl_v9 Multi-Display Device Support

• Support for dynamically adding display devices
• Support for multiple display devices working simultaneously
• Automatic association of input devices and display devices by device name

libhttp Component

• New HTTP session management functionality, supporting persistent connections and streaming reads

New Demo Examples

• New!! Development environment support for Raspberry Pi 4/5 and other SBC series, can run applications like your_chat_bot on Raspberry Pi
• Requires external audio sound card (1-2 channel microphone ADC + 1 channel speaker DAC)
• New Raspberry Pi SoC environment cross-compilation toolchain, can compile projects in (Raspberry Pi native environment/Win/Linux) environments
• Support for ASR KWS voice wake-up and 3A voice algorithms
• Dual-eye emotion application duo_eye_mood UI supports separate display and management of dual-eye animations (dual-screen independent display)
• Two SPI screens can render different content
• New switching between LVGL and Camera streams
• Provides example samples
• Completed rendering, acceleration, and scheduling optimizations on MCU

Other Fixes

• Fixed occasional Bluetooth network configuration failure issue

Development Improvements

- Support for exporting compile_commands.json to improve code navigation capabilities in editors like VSCode and Cursor

TuyaOpen 1.6.0 Release Notes

TuyaOpen 1.6.0 正式发布！本次更新重磅升级涂鸦 AI 基座至 V2.1 版本，核心能力迎来全面焕新：新增 OPUS 编码支持与树莓派4/5 设备适配，同步完成架构深度重构，大幅优化音视频处理及多模态交互性能，覆盖多模态、设备 MCP、多媒体三大核心模块。

此外，AI 相关 API 完成重构升级，打造更轻量化的端侧 AI 交互能力库，降低开发者集成门槛；配套开发工具链同步完善，丰富 Demo 示例，助力开发者快速上手、高效开发。

AI 基座更新

• 支持 OPUS 音频编码格式上报，大幅提升传输效率降低 AI 交互延时
• 支持视频、图片、文件上报，实现多模态 AI-Agent 模型交互
• 优化语音打断的响应速度，更加自然的对话系统
• 重构了 AI 应用组件的实现框架，梳理了各个模块的功能

新增功能

• 智能体触发音乐/故事播放的功能
• 支持设备 MCP，内置了四种工具，智能体可以主动触发设备（查询设备信息/切换模式/调节音量/拍照）
• 简易设备 MCP 注册 API，开发者可以快速拓展硬件 MCP 能力
• 按键切换多种聊天模式（长按对话模式、按键对话模式、唤醒对话模式、随意对话模式）
• 支持设备显示摄像头拍摄的画面，并拍照并发送给 AI 多模态分析
• 支持自定义的模式注册，语音对话策略可灵活定制
• 支持自定义的 UI 风格注册，开发者可快速定制自己 AI 交互 UI/UX

组件关系

:::txt
ai_main (主控制器)
    ├── ai_agent (云端通信)
    ├── ai_mode (对话模式管理)
    ├── ai_audio (音频处理)
    ├── ai_ui (界面显示)
    ├── ai_video (视频处理)
    ├── ai_skills (技能处理)
    ├── ai_mcp (MCP协议)
    ├── assets (资源文件)
    └── utility (工具类)

新增 audio_player 音频播放服务组件

支持多音频流数据源

• 内存缓存区（MEM）
• 本地文件（FILE）
• 网络流（URL/HTTP）

支持多种音频解码格式

• MP3、WAV、Speex
• Opus、OGG Opus

双音频流播放（AI对话/多媒体）

• 前台音频流 (Foreground-Stream)：用于语音提示、警告等
• 后台音频流 (Background-Stream)：用于背景音乐等
• 支持抢占模式（前台打断后台）和混音 Mixer 模式（同时播放）

音频处理组件

• Decoder 解码：将压缩音频解码为 PCM
• Resampling 重采样：支持采样率转换
• Mixer 混音：支持多路音频混合
• Volume Controls：音量控制

播放管理

• 创建/添加/删除播放项
• 上一首/下一首切换
• 循环播放（单曲/列表）
• 自动播放
• 播放/暂停/恢复/停止等播放控制

Display 显示管理驱动新增多种功能

新增帧缓冲管理

• 新增图形绘制功能接口(点绘制,矩阵填充,全屏填充)
• 新增图像旋转接口(90°、180°、270°)

格式转换

• 像素格式转换（RGB565 ↔ RGB888 ↔ RGB666 等）
• YUV422 到帧缓冲转换
• 单色转换（支持多种抖动算法）

新增图形加速器 (tal_dma2d) 驱动

• 用硬件加速进行图像格式转换，如 YUV422 -> RGB565/RGB888

• 硬件加速的 2D 内存拷贝，用于图像传输与复制

  资源管理

  • 多上下文支持（可创建多个句柄）
  • 线程安全（互斥锁保护）
  • 硬件资源复用（单硬件实例，多逻辑句柄）

实现 lvgl_v9 的多显示设备的功能

• 支持动态添加显示设备
• 支持多个显示设备同时工作
• 通过设备名称自动关联输入设备和显示设备

libhttp 组件

• 新增 HTTP 会话管理功能，支持持久连接和流式读取

Demo 示例新增

• New!! 开发环境支持树莓派4/5 等 SBC 系列，可在树莓派上运行 your_chat_bot 等应用
• 需要搭配外置音频声卡（1-2路麦克风 ADC + 1路喇叭 DAC）
• 新增树莓派 SoC 环境交叉编译工具链，可在（树莓派自身环境/Win/Linux）环境下编译工程
• 支持 ASR KWS 语音唤醒和 3A 语音算法
• 双眼情绪应用 duo_eye_mood 的 UI 支持双眼动画分开显示和管理（双屏异显）
• 两个 SPI 屏幕可以渲染不同内容
• 新增 LVGL 和 Camera 流之间来回切换
• 提供 example 示例
• 完成了 MCU 上面的渲染、加速和调度优化

其他修复

• 修复蓝牙配网偶现失败的问题

开发改善

• 支持导出 compile_commands.json，便于提升在 VSCode、Cursor 等编辑环境中的代码跳转能力