$7 ESP32 board lets an AI model use a real iPhone through AirPlay and a USB mouse
A XIAO ESP32-S3 plugged into an iPhone mirrors its screen over AirPlay and taps through a USB mouse, so a vision model can run it.
A $7 Seeed XIAO ESP32-S3 plugged into an iPhone lets an AI model operate the phone the way a person would. No computer is needed while it runs, and there’s no jailbreak or paid developer account involved.
The board is both eyes and hands. It runs its own Wi-Fi hotspot and shows up in Screen Mirroring as “ESP32 Agent”, then passes the H.264 stream to an iPhone app over Bluetooth, at roughly 480×1040 and 5 fps or less. It also poses as a USB mouse and keyboard, and with AssistiveTouch on, clicks become taps and drags become swipes. The app sends screenshots to any OpenAI-compatible vision model and acts on the reply. The README shows it setting a repeating alarm in Clock.
You’ll need iOS 18, a Mac with Xcode once to flash the board and build the app, and a model endpoint. Steps take a few seconds each. The README asks for the credit “ESP32 iPhone Agent by Henry Yuan”. Code and setup guide are on Henry’s GitHub.
All credit to Henry Yuan (onehumanbeing). The full story is at the original source.
