Developer open-sources iPhone Use, saying it can let Codex operate a real iPhone

Article

Codex Can now directly operate an iPhone, with the entire toolset open-sourced

Beating AI Flash News reports that Zhong Erxin, former product manager of the Doubao desktop product, has open-sourced iPhone Use, allowing Codex to directly operate a real iPhone. Users only need to issue tasks in natural language, and Codex can open apps, read pages, tap, swipe, enter Chinese text, and search and organize lists. The entire operation can be viewed in real time in the Codex sidebar.

The project uses Appium’s WebDriverAgent (WDA), a mobile automation tool based on Apple’s XCTest testing framework. The developer first installs Xcode on the iPhone through WDA, then uses the MCP service on a Mac to send Codex’s instructions to the phone, where WDA performs actions such as tapping and text input. Jailbreaking is not required, nor is separate deployment of Appium Server.

To reduce the cost of AI repeatedly inspecting screenshots, iPhone Use prioritizes reading interface control information provided by the system, directly identifying button text, positions, and states. When a control cannot be identified or is obscured, it falls back to a screenshot and lets the model determine the tap coordinates. It also supports batch execution of consecutive operations, reducing the number of model calls. The sidebar’s real-time view is obtained through WDA’s independent screen stream, without repeatedly taking screenshots.

At present, the official iPhone Use tutorial still requires connecting the phone through USB. On first use,Xcode must be used to sign and install WDA. Apple supports wireless debugging, and WDA can also be accessed over a network, but the project has not yet provided a complete wireless connection configuration solution.

Original link https://m.theblockbeats.info/flash/371040

Key points

Beating AI reports that Zhong Erxin has open-sourced iPhone Use.
iPhone Use allows Codex to open apps, read pages, tap, swipe, enter Chinese text, and search and organize lists.
The project uses Appium’s WebDriverAgent (WDA), sending commands to the iPhone through the MCP service on a Mac.
First-time use requires connecting the iPhone through USB and using Xcode to sign and install WDA。
The project has not yet provided a complete wireless connection configuration solution.

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

In simple terms, users can tell Codex what to do on an iPhone in natural language, and Codex then performs actions such as tapping and text input through a Mac and WDA. The system prioritizes reading interface control information and uses screenshots to determine positions only when necessary.

Why it matters to readers

This gives Codex another way to operate a real iPhone interface, and also demonstrates a technical path for AI to control mobile applications.

First-time users need to prepare a Mac, an iPhone, a USB connection, and Xcode, and complete the signing and installation of WDA.

Risks and unknowns

The evidence consists only of media reports; whether the project can be reliably reproduced has not been independently verified in the evidence.
Although wireless debugging is mentioned, the project has not yet provided a complete wireless connection configuration solution.
Compatibility across different iOS versions and devices is not stated in the evidence.

Developer open-sources iPhone Use, saying it can let Codex operate a real iPhone

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center