Download
ESP32 iPhone Agent is an open-source hardware and software project that enables AI models to interact with a physical iPhone using an inexpensive ESP32-S3 microcontroller. It combines AirPlay screen mirroring with USB Human Interface Device emulation to provide an AI agent with visual feedback and physical input capabilities. The ESP32-S3 receives the phone's screen stream and relays visual information to an iPhone application for AI analysis. Compatible vision models interpret screenshots and generate actions such as taps, swipes, typing, and navigation. The microcontroller executes these actions through USB mouse and keyboard interfaces using iOS AssistiveTouch. Bluetooth Low Energy connects the application with the hardware controller. The system operates without jailbreaking, a Mac, or an Apple developer account. It supports OpenAI-compatible vision models for automated mobile workflows.
Features
- AI-Powered Physical iPhone Automation
- ESP32-S3 AirPlay Screen Capture
- USB HID Touch and Keyboard Emulation
- Bluetooth Low Energy Command Communication
- OpenAI-Compatible Vision Model Integration
- Computer-Free and Jailbreak-Free Operation
Project Samples
