Search Results for "keyboard"
Sort By:
Showing 15 open source projects for "keyboard"
View related business solutions-
Host LLMs in Production With On-Demand GPUsDeploy your model, get an endpoint, pay only for compute time. No GPU provisioning or infrastructure management required.
-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
1
Windows-MCP
MCP server enabling AI agents to control and automate Windows OS
...It focuses on native interaction with Windows UI elements rather than relying on traditional computer vision techniques, which simplifies integration and improves efficiency. It includes a set of tools that simulate user inputs like keyboard and mouse actions while also capturing the current state of windows and interfaces. It is designed to be extensible and adaptable, allowing developers to customize or expand its functionality for different automation or AI use cases. -
2
Open Interface
Control Any Computer Using LLMs
Open Interface is a cross-platform application that allows users to control their computers using large language models (LLMs). By sending user requests to an LLM backend, it determines the necessary steps and executes them by simulating keyboard and mouse inputs. The system can adjust its actions based on real-time feedback, providing a self-driving computer experience. -
3
autoMate
AI tool for automating desktop tasks via natural language input
...It combines large language models with computer vision techniques to interpret user intent and understand on-screen content, allowing it to interact with graphical interfaces similarly to a human user. autoMate follows an observe-decide-act workflow, where it analyzes the screen, plans actions, and executes them through simulated input such as mouse clicks and keyboard events. Unlike conventional RPA tools that require predefined workflows, autoMate dynamically adapts to tasks by making autonomous decisions based on the current interface state. autoMate emphasizes local execution, meaning all processing happens on the user’s machine to maintain privacy and data security. -
4
Everywhere
Context-aware desktop AI assistant that understands screen content
...It integrates with multiple large language model providers and supports various tools, enabling flexible and extensible AI-powered workflows. Everywhere features a modern design with interactive elements such as markdown rendering, keyboard shortcuts, and voice input capabilities. Additionally, the project emphasizes seamless workflow integration by operating alongside existing applications rather than requiring users to switch. -
Build Securely on AWS with Proven FrameworksMoving to the cloud brings new challenges. How can you manage a larger attack surface while ensuring great network performance? Turn to Fortinet’s Tested Reference Architectures, blueprints for designing and securing cloud environments built by cybersecurity experts. Learn more and explore use cases in this white paper.
-
5
Self-Operating Computer
A framework to enable multimodal models to operate a computer
The Self-Operating Computer Framework is an innovative system that enables multimodal models to autonomously operate a computer by interpreting the screen and executing mouse and keyboard actions to achieve specified objectives. This framework is compatible with various multimodal models and currently integrates with GPT-4o, o1, Gemini Pro Vision, Claude 3, and LLaVa. Notably, it was the first known project to implement a multimodal model capable of viewing and controlling a computer screen. The framework supports features like Optical Character Recognition (OCR) and Set-of-Mark (SoM) prompting to enhance visual grounding capabilities. ... -
6
LeWorldModel
Official code base for LeWorldModel: Stable End-to-End Joint-Embedding
...It is highly configurable through source code or configuration files, allowing users to tailor behavior, keybindings, and layouts to their preferences. le-wm is intended for users who prefer keyboard-driven workflows and a distraction-free desktop environment. Its architecture avoids unnecessary complexity, making it easy to understand, modify, and extend. -
7
Phone Harness
Let your agent control your phone
...Screenshots of the mirrored window provide visual input, while Apple's Vision framework extracts visible text and tap-ready coordinates. macOS CGEvents supply taps, long presses, drags, scrolling, keyboard input, and iPhone Mirroring shortcuts. After performing an action, the agent captures the screen again to verify the actual resulting state. The harness can be installed as an agent skill for systems such as Claude Code or Codex. Its editable helper workspace also lets an agent create missing automation functions during execution. -
8
Cradle framework
The Cradle framework is a first attempt at General Computer Control
Cradle is an open-source framework designed to enable AI agents to perform complex computer tasks by interacting with software environments in a way similar to human users. The system introduces the concept of General Computer Control, where AI agents receive screenshots as input and perform actions through simulated keyboard and mouse operations. This approach allows agents to interact with any software interface without relying on specialized APIs or predefined automation scripts. The framework integrates reasoning, planning, and memory modules that help the agent understand its environment and execute long sequences of actions. Cradle agents are capable of performing tasks across a wide variety of environments, including computer applications and video games, demonstrating the generality of the approach. ... -
9
Oasis
Inference script for Oasis 500M
Open-Oasis provides inference code and released weights for Oasis 500M, an interactive world model that generates gameplay frames conditioned on user keyboard input. Instead of rendering a pre-built game world, the system produces the next visual state via a diffusion-transformer approach, effectively “imagining” the world response to your actions in real time. The project focuses on enabling action-conditional frame generation so developers can experiment with interactive, model-generated environments rather than static video generation alone. ... -
Build Agents and Models on One PlatformGemini Enterprise Agent Platform is Google Cloud's comprehensive platform for developers to build, scale, govern, and optimize agents and models. Choose from Google's most advanced models and third-party models like Anthropic's Claude Model Family.
-
10
runprompt
Run LLM prompts from your shell
runprompt is an interactive command launcher and prompt utility that lets users bind shell commands, scripts, and workflows to quick keyboard shortcuts or natural-language queries, helping streamline repetitive terminal tasks and boost developer productivity. It functions as a lightweight, launcher-centric interface where you can type a phrase, partial command, or alias and have RunPrompt suggest or execute relevant actions instantly, reducing the need to memorize long commands or navigate complex directory structures. ... -
11
Elia
Terminal-based LLM chat tool with multi-model and local support
Elia is an open source terminal-based interface designed for interacting with large language models in a fast and efficient way. It runs entirely in the command line, offering a keyboard-driven experience that reduces the need for switching between apps. Users can chat with both proprietary models like ChatGPT and Claude, as well as local models such as Llama 3, Mistral, and Gemma. Elia stores conversations in a local SQLite database, making it easy to revisit past interactions. It supports flexible usage with inline and full-screen chat modes, along with simple configuration through a single file. ... -
12
GELab-Zero
GUI Exploration Lab. One of the best GUI agent solutions
GELab-Zero is an open-source “GUI Agent” framework aiming to automate interactions with graphical user interfaces (GUIs), combining both the agent model and all supporting infrastructure — including inference, input orchestration, and GUI automation logic — in a plug-and-play package that runs locally, without cloud dependencies. The idea is to let developers or users harness an AI agent that can simulate clicking, typing, reading UI elements, and interacting with apps in a human-like way... -
13
Universe Starter Agent
A starter agent that can solve a number of universe environments
...Its purpose is to serve as a baseline or reference implementation so researchers or developers can see how to build agents that operate in real-time, visual environments (e.g., games, browser apps) via pixel observations and keyboard/mouse actions. Under the hood, this starter agent implements a version of the A3C (Asynchronous Advantage Actor-Critic) algorithm, adapted for the specific challenges of Universe environments (e.g., network latency, VNC streaming, asynchronous observations). The repo includes modules like train.py, worker.py, model.py, a3c.py, and envs.py to support training, parallel worker management, policy/critics, and environment wrappers. -
14
Universe
Software for measuring and training an AI's general intelligence
...Universe makes it possible for any existing program to become an OpenAI Gym environment, without needing special access to the program's internals, source code, or APIs. It does this by packaging the program into a Docker container, and presenting the AI with the same interface a human uses: sending keyboard and mouse events, and receiving screen pixels. Our initial release contains over 1,000 environments in which an AI agent can take actions and gather observations. -
15
GoMad is a speech recognition system that allows you to control windows-based applications using your voice as input, instead of your mouse and keyboard.
- Previous
- You're on page 1
- Next
