Give any agent an iOS environment it can see, control, and learn. Phone Use turns native apps into reliable, reusable tools.
██████ ██ ██ ██████ ███ ██ ███████ ██ ██ ███████ ███████ ██ ██ ██ ██ ██ ██ ████ ██ ██ ██ ██ ██ ██ ██████ ███████ ██ ██ ██ ██ ██ █████ █████ ██ ██ ███████ █████ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██████ ██ ████ ███████ ██████ ███████ ███████
natural-language phone control for agents
TASKSave this phone's model and iOS version to a note titled “Device Info.”
DEVICEiPhone 17 Pro· iOS Simulator
01observeSettings → Aboutrunning02captureModel Name · iOS Versionqueued03openNotesqueued04writeDevice Infoqueued05verifynote exists with both valuesqueuedWHAT PHONE USE DOES
It reads the interface, takes an action, checks the result, and remembers the route so the same task gets faster next time.
OBSERVE
A compressed accessibility tree gives the agent labels, values, roles, and geometry. Vision is there when the interface needs it—not as the only source of truth.
root····ACT + VERIFY
Taps are aimed at real elements and checked against the next screen. If nothing changed, the agent knows immediately and can recover instead of drifting.
targetCell “General”
actiontap · ref_04
0x91c4MAP + COMPILE
App Cartographer saves screens and transitions as a durable graph. Successful errands compile into skills that replay with zero model calls.
$ phone-use skill run save-device-info
✓ postcondition matched
MADE FOR AGENTS. NICE FOR DEVELOPERS.
Start in the terminal, embed the typed SDK, or hand the same runtime to any MCP client. The behavior underneath stays consistent.
Explore the documentation$ npm i -g phone-use
$ phone-use goto Settings About
✓ About, 2 taps
$ phone-use ask "iOS Version"
iOS Version = 26.1COMPLETED RUN
A completed task returns its tool calls, inputs, results, and timing. The trace artifact keeps the matching interface trees and captured frames for deeper inspection.
expect Ride status is RequestingiOSWORLD · REFERENCE RESULTS
133 tasks across 26 SwiftUI apps, scored against published frontier baselines.