Search Results for "gui"

UI-TARS Desktop is a graphical user interface (GUI) agent application that leverages the UI-TARS vision-language model to enable natural language control of computers. This cross-platform tool supports both Windows and macOS, allowing users to perform tasks through intuitive commands. Key features include screenshot-based visual recognition, precise mouse and keyboard control, and real-time feedback on actions.

...It provides components for instruction following, tool usage (function calling), planning, memory, RAG (retrieval augmented generation), code interpreter, etc. It ships with example applications (Browser Assistant, Code Interpreter, Custom Assistant), supports GUI front-ends, backends, server setups. Agent workflow can maintain context / memory to perform multi-turn or more complex logic over time. It acts as the backend for Qwen Chat among other use cases. Built-in Code Interpreter tool that can execute code (locally) as part of agent workflows.

PC-GUI: A lightweight desktop GUI framework for AI, natively supporting live typewriter-style streaming output like Gemini! 🎉 💡 Core philosophy: Rapid development · Minimal footprint · Native performance. We empower you to build premium desktop tools that users are willing to pay for. PC-GUI helps you meet strong market demands by building compact, powerful, commercial-grade applications with a simple and stable tech stack.

Agent S is an open-source agentic framework designed to enable autonomous computer use through an Agent-Computer Interface (ACI). Built to operate graphical user interfaces like a human, it allows AI agents to perceive screens, reason about tasks, and execute actions across macOS, Windows, and Linux systems. The latest version, Agent S3, surpasses human-level performance on the OSWorld benchmark, demonstrating state-of-the-art results in complex multi-step computer tasks. Agent S combines...

OmniParser is a comprehensive method for parsing user interface screenshots into structured elements, significantly enhancing the ability of multimodal models like GPT-4 to generate actions accurately grounded in corresponding regions of the interface. It reliably identifies interactable icons within user interfaces and understands the semantics of various elements in a screenshot, associating intended actions with the correct screen regions. To achieve this, OmniParser curates an...