Browse documentation

Start

Cloud

Use Phone Use

Core concepts

TypeScript SDK

Benchmark

Reference

Core concepts

Observe

Convert a native interface into compact, actionable structure.

Structured observation

bashphone-use
phone-use observe
textphone-use
App: Settings

@e1 [Button] "General" (16,74 82x44)
@e2 [StaticText] "iOS Version" (20,188 103x22)
@e3 [StaticText] "26.1" (329,188 41x22)

Accessibility first

  1. 1

    Identifier

    Prefer the stable accessibility identifier.

  2. 2

    Exact label

    Match visible text exactly.

  3. 3

    Substring

    Relax to a contained label.

  4. 4

    Fuzzy label

    Handle punctuation and spacing variants.

Vision fallback

Canvases, maps, and custom controls may expose insufficient structure. Phone Use can use screenshots, Set-of-Marks annotations, zoomed regions, and raw coordinates for those cases.