ESP32 iPhone Agent is an open-source hardware and software project that enables AI models to interact with a physical iPhone using an inexpensive ESP32-S3 microcontroller. It combines AirPlay screen mirroring with USB Human Interface Device emulation to provide an AI agent with visual feedback and physical input capabilities. The ESP32-S3 receives the phone's screen stream and relays visual information to an iPhone application for AI analysis. Compatible vision models interpret screenshots and generate actions such as taps, swipes, typing, and navigation. The microcontroller executes these actions through USB mouse and keyboard interfaces using iOS AssistiveTouch. Bluetooth Low Energy connects the application with the hardware controller. The system operates without jailbreaking, a Mac, or an Apple developer account. It supports OpenAI-compatible vision models for automated mobile workflows.

Features

  • AI-Powered Physical iPhone Automation
  • ESP32-S3 AirPlay Screen Capture
  • USB HID Touch and Keyboard Emulation
  • Bluetooth Low Energy Command Communication
  • OpenAI-Compatible Vision Model Integration
  • Computer-Free and Jailbreak-Free Operation

Project Samples

ESP32 iPhone Agent Screenshot 1

Project Activity

See All Activity >

Categories

AI Models

License

MIT License

Follow ESP32 iPhone Agent

ESP32 iPhone Agent Web Site

Other Useful Business Software
$300 Free Credits to Build on Google Cloud Icon
$300 Free Credits to Build on Google Cloud

New customers can spin up VMs, build with AI, and query data at no cost.

Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
Start Free
Rate This Project
Login To Rate This Project

User Reviews

Be the first to post a review of ESP32 iPhone Agent!

Additional Project Details

Operating Systems

Linux, Mac

Programming Language

C

Related Categories

C AI Models

Registered

1 day ago
Report inappropriate content