agent-desktop is a free, open-source computer automation tool developed by Vercel Labs that enables AI agents to interact with desktop applications on Windows, macOS, and Linux. Built with Rust, the software uses operating system accessibility APIs to identify interface elements and perform actions without relying on screenshots, image recognition, or pixel-based navigation. Agent Desktop provides structured JSON representations of application interfaces, allowing AI agents to locate buttons, menus, text fields, and other interactive elements using unique references. The platform supports automated clicking, typing, window navigation, clipboard operations, and interface inspection, enabling AI agents to execute complex desktop workflows through command-line instructions. Developers can integrate agent-desktop with AI coding assistants and automation frameworks through its command-line interface, native C library, and Model Context Protocol (MCP) server.

Features

  • AI-Powered Desktop Automation: Enables AI agents to interact with desktop applications by clicking buttons, entering text, navigating menus, managing windows, and performing automated workflows through operating system accessibility APIs.
  • Accessibility-Based Interface Recognition: Reads application accessibility trees and returns structured JSON data containing interactive elements and unique references, eliminating the need for screenshot-based element detection or pixel matching.
  • Cross-Platform Computer Control: Supports Windows, macOS, and Linux using native accessibility frameworks, including Windows UI Automation, macOS Accessibility APIs, and Linux AT-SPI.
  • Command-Line Interface and Native Library: Provides a CLI and C library for executing desktop actions, inspecting application interfaces, managing sessions, and integrating automation capabilities into custom software.
  • Model Context Protocol Integration: Includes an MCP server that exposes desktop automation commands as tools, allowing compatible AI agents and development environments to interact with desktop applications.
  • - Reliable Element Targeting and Verification: Uses snapshot-based element references, structured error reporting, interface state inspection, and post-action verification to help agents detect changes, recover from stale references, and execute workflows reliably.

Project Samples

Project Activity

See All Activity >

License

Apache License V2.0

Follow agent-desktop

agent-desktop Web Site

Other Useful Business Software
Earn up to 16% annual interest with Nexo. Icon
Earn up to 16% annual interest with Nexo.

Let your crypto work for you

Put idle assets to work with competitive interest rates, borrow without selling, and trade with precision. All in one platform. Geographic restrictions, eligibility, and terms apply.
Get started with Nexo.
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of agent-desktop!

Additional Project Details

Operating Systems

Linux, Mac, Windows

Programming Language

Rust

Related Categories

Rust Agentic AI Tool, Rust AI Agent Frameworks

Registered

2 days ago