Search Results for "screen"
Sort By:
Showing 29 open source projects for "screen"
View related business solutions-
MongoDB Atlas runs apps anywhereMongoDB Atlas gives you the freedom to build and run modern applications anywhere—across AWS, Azure, and Google Cloud. With global availability in over 115 regions, Atlas lets you deploy close to your users, meet compliance needs, and scale with confidence across any geography.
-
Train ML Models With SQL You Already KnowBuild and deploy ML models using familiar SQL. Automate data prep with built-in Gemini. Query 1 TB and store 10 GB free monthly.
-
1
Self-Operating Computer
A framework to enable multimodal models to operate a computer
The Self-Operating Computer Framework is an innovative system that enables multimodal models to autonomously operate a computer by interpreting the screen and executing mouse and keyboard actions to achieve specified objectives. This framework is compatible with various multimodal models and currently integrates with GPT-4o, o1, Gemini Pro Vision, Claude 3, and LLaVa. Notably, it was the first known project to implement a multimodal model capable of viewing and controlling a computer screen. -
2
Everywhere
Context-aware desktop AI assistant that understands screen content
Everywhere is a context-aware desktop AI assistant designed to interact directly with the content displayed on a user’s screen. It distinguishes itself from traditional AI tools by eliminating the need for manual input methods such as copying text or taking screenshots, instead allowing users to invoke assistance instantly through a shortcut. It can analyze on-screen information in real time and provide contextual responses, making it useful for tasks like troubleshooting errors, summarizing articles, translating text, and refining written content. ... -
3
Claude Overlay
A floating, screen-aware Claude Code chat for Windows
Claude Overlay is a Windows utility that places a screen-aware Claude Code assistant in a frameless, always-on-top chat window. It captures the active window or every monitor when a message is sent, letting Claude answer questions about visible documents, errors, dashboards, and applications. Because it runs the installed Claude Code CLI through the Agent SDK, it can also read files, execute commands, and edit open work when permissions allow. -
4
autoMate
AI tool for automating desktop tasks via natural language input
autoMate is an AI-powered local automation tool designed to enable users to control and automate their computers using natural language instructions instead of traditional scripting or rule-based systems. It combines large language models with computer vision techniques to interpret user intent and understand on-screen content, allowing it to interact with graphical interfaces similarly to a human user. autoMate follows an observe-decide-act workflow, where it analyzes the screen, plans actions, and executes them through simulated input such as mouse clicks and keyboard events. Unlike conventional RPA tools that require predefined workflows, autoMate dynamically adapts to tasks by making autonomous decisions based on the current interface state. autoMate emphasizes local execution, meaning all processing happens on the user’s machine to maintain privacy and data security. -
Ship Agents FasterGemini Enterprise Agent Platform lets you rapidly build, scale, govern and optimize production-ready agents grounded in your organization's data. The platform enables developers to build custom or pre-built agents for virtually any use case. New customers get $300 in free credits.
-
5
claude-video
Give Claude the ability to watch any video
Claude Video is an agent skill that gives Claude and compatible coding assistants the ability to analyze video content. It accepts public video URLs or local video files, then extracts the information needed to answer user questions about what happened on screen and in the audio. The workflow checks captions first, downloads only what is necessary, extracts timestamped frames, and produces a transcript through native captions or Whisper fallback. It supports different detail levels so users can trade speed, token cost, and visual coverage depending on the task. The skill is useful for summarizing videos, reviewing screen recordings, analyzing content structure, diagnosing visual bugs, and turning course material into notes. ... -
6
LeWorldModel
Official code base for LeWorldModel: Stable End-to-End Joint-Embedding
LeWorldModel is a minimalist tiling window manager designed for the X11 windowing system, focusing on simplicity, performance, and efficient use of screen space. It provides automatic window tiling behavior, organizing application windows into structured layouts without requiring manual resizing or positioning. The project emphasizes a lightweight design, minimizing resource usage while maintaining responsiveness and stability. It is highly configurable through source code or configuration files, allowing users to tailor behavior, keybindings, and layouts to their preferences. le-wm is intended for users who prefer keyboard-driven workflows and a distraction-free desktop environment. ... -
7
Open-AutoGLM
An open phone agent model & framework
Open-AutoGLM is an open-source framework and model designed to empower autonomous mobile intelligent assistants by enabling AI agents to understand and interact with phone screens in a multimodal manner, blending vision and language capability to control real devices. It aims to create an “AI phone agent” that can perceive on-screen content, reason about user goals, and execute sequences of taps, swipes, and text input via automated device control interfaces like ADB, enabling hands-off completion of multi-step tasks such as navigating apps, filling forms, and more. Unlike traditional automation scripts that depend on brittle heuristics, Open-AutoGLM uses pretrained large language and vision-language models to interpret visual context and natural language instructions, giving the agent robust adaptability across apps and interfaces. -
8
Elia
Terminal-based LLM chat tool with multi-model and local support
...Users can chat with both proprietary models like ChatGPT and Claude, as well as local models such as Llama 3, Mistral, and Gemma. Elia stores conversations in a local SQLite database, making it easy to revisit past interactions. It supports flexible usage with inline and full-screen chat modes, along with simple configuration through a single file. Installation is straightforward via pipx, and users can customize themes, system prompts, and model settings. Elia is built for developers and power users who prefer a streamlined, terminal-first workflow for working with AI models. -
9
OmniParser
A simple screen parsing tool towards pure vision based GUI agent
...It reliably identifies interactable icons within user interfaces and understands the semantics of various elements in a screenshot, associating intended actions with the correct screen regions. To achieve this, OmniParser curates an interactable icon detection dataset containing 67,000 unique screenshot images labeled with bounding boxes of interactable icons derived from DOM trees. Additionally, a collection of 7,000 icon-description pairs is used to fine-tune a caption model that extracts the functional semantics of detected elements. ... -
Fully Managed MySQL, PostgreSQL, and SQL ServerCloud SQL handles your database ops end to end, so you can focus on your app.
-
10
esp32-ai
Running a 28.9M parameter LLM on an $8 microcontroller
...This memory layout allows the device to retrieve only the embedding rows required for each token. The implementation reaches roughly 9.5 tokens per second and can display generated words on a connected screen. Trained on TinyStories, the model produces short, simple stories rather than answering questions, following instructions, or providing factual knowledge. The repository includes firmware, wiring and flashing instructions, training code, quantization experiments, ablations, and measured results. -
11
claude-real-video
Let Claude (or any LLM) actually watch a video
...It can transcribe audio, generate a clean output folder, and create a local viewer with the video, keyframe grid, and transcript. The tool can also be installed as a Claude Code skill so an agent can process videos more directly. It is useful for reviewing screen recordings, summarizing visual content, analyzing tutorials, and turning video into LLM-readable context. -
12
FlowLens MCP
Open-source MCP server that gives your coding agent
...It works together with a companion browser extension: when a user reproduces a bug or a complicated UI interaction, the extension captures a rich session log, including screen/video recording, network traffic, console logs, DOM events, storage changes, and more, and exports it. The MCP server then loads this captured “flow” and exposes it to the AI agent via the Model Context Protocol (MCP), letting the agent examine, search, filter, and reason about the session just as a human developer would, without needing the agent to re-run the flow or rely on minimal reproduction data (logs, screenshots). -
13
AppAgent
Multimodal Agents as Smartphone Users, an LLM-based multimodal agent
AppAgent is an open-source multimodal agent framework designed to enable large language models to operate smartphone applications through natural interactions with graphical user interfaces. The system allows an AI agent to interpret visual information from the screen and translate natural language instructions into actions such as tapping, swiping, and navigating between application screens. Instead of requiring backend access to application APIs, the framework interacts with apps the same way a human user would, making it compatible with a wide variety of mobile applications. AppAgent combines vision capabilities with language reasoning to understand interface elements and determine which actions are required to accomplish a task. ... -
14
OpenRecall
OpenRecall is a fully open-source, privacy-first alternative
OpenRecall is an open-source, privacy-first system designed to capture, index, and make searchable a user’s entire digital activity history, effectively acting as a personal memory layer for computing environments. It works by taking periodic screenshots of a user’s screen and applying local AI processing, including OCR and semantic analysis, to extract and structure information from both text and images. This data is then indexed into a searchable database, allowing users to retrieve past information quickly using natural language queries. Unlike proprietary alternatives, OpenRecall operates entirely locally, ensuring that all captured data remains on the user’s device and is never transmitted to external servers. ... -
15
MAI-UI
Real-World Centric Foundation GUI Agents
...Developed by Tongyi-MAI (Alibaba’s research initiative), the MAI-UI models are multimodal agents trained to understand user instructions and corresponding screenshots, grounding those instructions to on-screen elements and generating sequences of GUI actions such as taps, swipes, text input, and system commands. Unlike traditional UI frameworks, MAI-UI emphasizes realistic deployment by supporting agent–user interaction (clarifying ambiguous instructions), integration with external tool APIs using MCP calls, and a device–cloud collaboration mechanism that dynamically routes computation to on-device or cloud models based on task state and privacy constraints. -
16
OAGI Python SDK
Python SDK for the Computer Use model Lux, developed by OpenAGI
...It exposes the OAGI API in an ergonomic way, letting you trigger Lux in three main modes: Tasker for precise scripted sequences, Actor for fast one-shot tasks, and Thinker for open-ended, multi-step objectives. The SDK is designed around “computer use” as a paradigm, where the AI actually navigates interfaces, clicks, types, scrolls, and reads the screen through screenshots instead of only calling APIs. It provides high-level asynchronous agents (like AsyncDefaultAgent and AsyncActor) that encapsulate the loop of capturing screenshots, sending them to Lux, interpreting responses, and executing UI actions with PyAutoGUI. Multiple installation flavors let you choose between a minimal oagi-core package or variants that bundle desktop automation and FastAPI/Socket.IO server capabilities. -
17
Mice MX OS speech to text Voice Control
Mice speech to text with MX Cinnamon OS ISO
...The voice control program MiceStTM.py was developed to enable easy adaptation to other languages. However, only German settings are currently implemented. category: System commands comment: Screen grid trigger: Display screen (Ras.*|Grid)* terminal_command: /opt/micesttm/read-aloud/screen_grid.py & sleep 1 && xdotool search --name "screen grid" windowactivate intern_command: tts: Screen grid for the mouse click was selected. -
18
OculiX
Visual Automation IDE — automate anything you see on screen
OculiX is the evolution of SikuliX, actively maintained with the full agreement of its original creator RaiMan. Automate any desktop application using image recognition (OpenCV) and OCR (Tesseract + PaddleOCR). No access to source code or DOM required — if you can see it, you can automate it. Key features: - Guided step-by-step recorder with live code preview - Image recognition via OpenCV 4.10 - Dual OCR: Tesseract (built-in) + PaddleOCR (neural, high precision) - Local and remote... -
19
QuizSolver
AI-powered quiz solver for Windows. Free to use, easy to set up.
...Open QuizSolver.exe, paste your key, click Save Key 4. Select your quiz site from the dropdown HOW TO USE: 1. Open your quiz so the first question is visible on screen 2. Press START or F9 — the window minimizes automatically 3. The bot answers each question on its own 4. Press ESC or the floating STOP button to stop anytime Powered by Groq and OpenRouter. -
20
anchorcastapp
Free AI-powered church presentation & live sermon display app
...Features: - Live Sermon Transcription — real-time speech-to-text via Whisper AI - AI Bible Verse Detection — automatically detects and displays verses from live sermons - Song Manager — display song lyrics on projection - Media Playback — images and video on projection screen - NDI Output — stream projection over local network - Remote Control — control presentation from any phone via Wi-Fi - Timer Display — countdown and service timers - Presentation Editor — create and manage slide decks - Theme Manager — customize projection appearance - Sermon Intelligence & Analytics Free for all churches. ... -
21
Luna AI
Virtual AI anchor that combines state-of-the-art technology
Luna AI is a virtual AI streamer framework designed to power an interactive VTuber that can go live on major platforms and chat with viewers in real time. It is built around a core assistant persona called “Luna AI,” which can be driven by a wide range of large language models and platforms, including GPT-style APIs, Claude, LangChain-based backends, ChatGLM, Kimi, Ollama, and many others. The project supports multiple rendering backends for the avatar, such as Live2D, Unreal Engine (UE),... -
22
Screen Translate
An OCR translator tool made by utilizing tesseract & python-opencv
STL is an easy to use and light OCR translator tool that can be use to translate your screen. Made with python by utilizing Tesseract and opencv-python. For full view of the project you can check the Github repository: https://github.com/Dadangdut33/Screen-Translate REQUIREMENTS - Tesseract : https://github.com/UB-Mannheim/tesseract/wiki. Needed for the ocr. Install it with all the language pack. - Libretranslate (Optional for offline translation support) - Internet connection for translation if not using libretranslate # Tutorial on How To Setup https://github.com/Dadangdut33/Screen-Translate#installation-and-setup -
23
BerryNet
Deep learning gateway on Raspberry Pi and other edge devices
This project turns edge devices such as Raspberry Pi into an intelligent gateway with deep learning running on it. No internet connection is required, everything is done locally on the edge device itself. Further, multiple edge devices can create a distributed AIoT network. At DT42, we believe that bringing deep learning to edge devices is the trend towards the future. It not only saves costs of data transmission and storage but also makes devices able to respond according to the events... -
24
Douyin-Bot
Python TikTok bot
...Its original goal was playful and experimental, focused on browsing and identifying content based on computer vision and face analysis. It is best understood as a demonstration of app automation, screen capture, API-based image analysis, and scripted mobile interaction. Its main value is showing how Python can orchestrate a phone app workflow by combining ADB, screenshots, recognition logic, and automated gestures. -
25
BossSensor
Hide screen when boss is approaching
...The project uses a webcam to continuously capture images and analyze them using a face classification model trained to distinguish between the designated “boss” and other individuals. When the system detects that the trained face appears in the camera view, the program automatically triggers actions such as hiding the user’s screen or switching to a safe display. The software relies on libraries such as OpenCV, TensorFlow, and Python-based machine learning tools to perform face detection and classification. Training the system requires a dataset of labeled images representing the boss and other people so that the model can learn to differentiate between them.