Search Results for "audio gui interface"

MAI-UI is a cutting-edge open-source project that implements a family of foundation GUI (Graphical User Interface) agent models capable of interpreting natural language and performing real-world GUI navigation and control tasks across mobile and desktop environments. Developed by Tongyi-MAI (Alibaba’s research initiative), the MAI-UI models are multimodal agents trained to understand user instructions and corresponding screenshots, grounding those instructions to on-screen elements and generating sequences of GUI actions such as taps, swipes, text input, and system commands. ...

UI-TARS Desktop is a graphical user interface (GUI) agent application that leverages the UI-TARS vision-language model to enable natural language control of computers. This cross-platform tool supports both Windows and macOS, allowing users to perform tasks through intuitive commands. Key features include screenshot-based visual recognition, precise mouse and keyboard control, and real-time feedback on actions.

...Its goal is to provide programmatic access not just to standard notebook operations, but also to many capabilities that are either limited or unavailable in the web interface, making it especially useful for automation and custom pipelines. The project covers notebook management, source ingestion, conversational querying, research workflows, and sharing controls, while also enabling the generation of a wide range of study and media artifacts. These outputs include audio overviews, videos, slide decks, infographics, quizzes, flashcards, reports, data tables, and mind maps, with configurable formats and export options.

OmniParser is a comprehensive method for parsing user interface screenshots into structured elements, significantly enhancing the ability of multimodal models like GPT-4 to generate actions accurately grounded in corresponding regions of the interface. It reliably identifies interactable icons within user interfaces and understands the semantics of various elements in a screenshot, associating intended actions with the correct screen regions. To achieve this, OmniParser curates an...

AionUi is an open-source, cross-platform graphical interface that turns command-line AI tools into a unified coworking desktop for interacting with multiple local AI agents and CLI models like Gemini CLI, Claude Code, Codex, Qwen Code, and others. Instead of forcing users to work in separate terminals for each tool, AionUi automatically detects installed CLI tools and provides a central visual workspace where sessions can run in parallel, contexts are preserved, and conversations are saved...

AnyTool is an open-source universal tool-use layer for AI agents that addresses the critical problem of how autonomous agents reliably interact with external tools and environments. Rather than having each agent handle tool invocation logic on its own, AnyTool provides a standardized interface and orchestrator that intelligently selects and manages tools, reduces context overhead, and improves execution reliability across diverse capabilities like web APIs, local commands, and GUI automation. It uses progressive filtering and adaptive orchestration to ensure the right tools are retrieved efficiently and work cohesively with agents of varying complexity, scaling to thousands of tools with self-optimizing behavior. ...

Agent S is an open-source agentic framework designed to enable autonomous computer use through an Agent-Computer Interface (ACI). Built to operate graphical user interfaces like a human, it allows AI agents to perceive screens, reason about tasks, and execute actions across macOS, Windows, and Linux systems. The latest version, Agent S3, surpasses human-level performance on the OSWorld benchmark, demonstrating state-of-the-art results in complex multi-step computer tasks. Agent S combines...

npcpy is a Python-based agent framework and command-line toolkit (the NPC Shell) for developers to build, test, and integrate AI agents into their workflows, including both command-line and GUI interfaces via NPC Studio. Welcome to npcpy, the core library of the NPC Toolkit that supercharges natural language processing pipelines and agent tooling. npcpy is a flexible framework for building state-of-the-art applications and conducting novel research with LLMs. The structure of npcpy also...

ILA stands for (kind of) intelligent, learning assistant and is a speech recognition system aka voice assistant very similar to Siri, Google Now and Cortana. ILA is fully customizable and you can teach her/him/it new things by yourself like executing system commands, opening web pages, programs and apps or just some basic conversation :-) ILA runs on Java und thus is compatible to Windows, Mac and Linux. It is designed to integrate with your home enviroment and for example build up your own,...

Wumpus World Game is based on knowledgebase agents and is described in the book , Artificial Intelligence : A Modern Approach (Russel - Norvig). This is an implementation of Wumpus World (a Knowledgebase agent game) using csharp alongside prolog with a GUI. C# has the main environment of the game and the GUI The GUI was created using WPF , Windows Presentation Foundation Prolog is used as the knowledgebase of the agent of Wumpus World In order for C# to communicate with Prolog a Csharp interface is used SWI-Prolog. Author: Abdulrahman Zakrt Date: Jan.02, 2018 Requirements This implementation of Wumpus World uses prolog and is based on Dotnet and requires the following: 64 bit windows machine , Currently the game only supports 64 bit machines SWIPL(v6.6.6) 64 bit (Installed automatically by the game if not available) Latest Version of DotNet (Installed automatically by windows) How to Use In order to use the GUI only you can use the setup.exe in the SetupFiles.

An open source chess engine in C, released under the GPL. Compatible with the XBoard and UCI interface protocols to link to a GUI or other chess arbiter.

The central aim of OrkAPI is to be an interface to any kind of Ork-related data within Java. It also features a GUI that can assist in creating a move and inspecting the collected data. There are many points to extend OrkAPI with "Plugin" classes.

This project will provide an interface for using Lightweight Communications Calculus (LCC) to control agents in the Unreal Tournament enviroment. It consists of a prolog-style interperter, LCC parser/IDE,a GUI / LCC editor and interfaces to the Gamebots