[vijaysharm]

Depends on mobile-next/devicekit-ios#109, which adds the agent side. Draft until that ships in a devicekit-ios release and agentVersionIOS is bumped. This PR leaves the version alone, since agent bumps land as their own chore PRs.

Summary

mobilecli has no way to check an app the way a VoiceOver user hears it. Labels show up in dump ui, but not what VoiceOver actually speaks or the order it visits elements in. iOS 27's XCUIDevice.voiceOverService gives both, on simulators and real devices, and mobile-next/devicekit-ios#109 exposes it from the agent.

This adds mobilecli device voiceover get | set on|off | speech | move next|previous|in|out, plus the matching device.voiceover.* JSON-RPC methods (documented in docs/openrpc.json). Android and pre-27 iOS answer with "does not support voiceover" or the agent's "requires iOS 27" error.

Risk surface

• VoiceOver stays on after mobilecli exits, and while it is on io tap focuses an element instead of activating it. The README, the --help text and the RPC description all say so, because a forgotten set on makes later taps look broken.
• set gets a 25s agent timeout instead of the default 10s. Disabling takes ~5s even when VoiceOver is already off, and a slow enable fails after ~15s with Apple's own error, which is more useful than our timeout. 25s stays under the fleet client's 30s read deadline.
• There is no "focus this element" call in Apple's API. io tap @ref does it: with VoiceOver on, the tap moves VoiceOver focus to the element without activating it, and the next speech/move starts there. Each such tap takes ~6s.
• Utterances are cut at 64 characters by Apple's API, so long labels can't be checked in full.
• No existing command changes behaviour. VoiceOver support is an optional interface on the iOS device types, so other devices need no stubs.

VoiceOver gestures through io gesture

VoiceOver also responds to gestures synthesized through io gesture. On an iPhone 13 (iOS 27.0.1) these did what VoiceOver does for a finger:

Gesture	Effect
1-finger tap, slow drag	focus moves to the element under the finger
1-finger flick left/right, up/down	previous/next element; rotor action
2-finger twist	changes the rotor setting
2-finger swipe up/down, 2-finger tap	read all from top / from focus; pause
2-finger Z scrub	back
3-finger swipe up/down, left/right	scroll (focus must be inside the scroll view); change page
3-finger tap	speaks extra detail
4-finger tap near top/bottom	first/last element

Tap-timing gestures can't be expressed yet: double tap to activate, split tap, Magic Tap, triple tap. device.io.gesture presses every finger at t=0, so for now nothing can be activated while VoiceOver is on. Follow-ups #553 and mobile-next/devicekit-ios#110 add a per-finger start time, which made all four work in a local test.

Test plan

• go test ./... -race
• go fmt ./... leaves no diff
• golangci-lint v2.13.1 adds no new findings apart from one more goconst on the cobra Use: "get" string

Behavior verification

Needs the mobile-next/devicekit-ios#109 runner installed on an iOS 27 simulator (xcrun simctl install <udid> devicekit-iosUITests-Runner.app).

D=<simulator-udid>
mobilecli device voiceover set on --device $D
mobilecli device voiceover speech --device $D
mobilecli device voiceover move next --device $D
mobilecli device voiceover set off --device $D

• CLI on an iOS 27.0 simulator. Expected: set reports the new state; speech and move return the focused element's label (e.g. "Fitness Double tap to open."); move in returns the "Navigation Style ... Groups" error; speech while off returns "not enabled"; set maybe and move up are rejected before the device is touched
• Focus a specific element. dump ui, then io tap @<ref> with VoiceOver on. Expected: speech returns that element's label, and the app does not navigate
• JSON-RPC via mobilecli server start. Expected: the same results over POST /rpc; a missing deviceId or enabled → "'X' is required"
• CLI on a real iPhone 13 (iOS 27.0.1), Calculator. Expected: the same results as on the simulator; dump ui then io tap @<ref of the 7 key> → speech returns "7" and the display does not change, because the tap focused the key without pressing it

Demo

https://github.com/user-attachments/assets/8c1521c0-f321-46e6-b115-00726a94de66
https://github.com/user-attachments/assets/17446bbf-9f96-49fb-92f3-86fcc726a36c

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • Added VoiceOver controls for supported devices and simulators: check its status, turn it on or off, retrieve the focused element’s spoken text, and move the VoiceOver cursor in supported directions.
  • Access these controls through the CLI or JSON-RPC interface.

[coderabbitai]

Warning

Review limit reached

• Run on-demand review

_{This review includes 1 billable file and costs up to $0.25.}

Or wait 46 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used all 2 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

• Source mode: Auto
• Repository: mobile-next/mobilecli
• Branch: main
• File: .coderabbit.yaml
• Commit: 83e47ca
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 4b99858e-9313-4880-9314-ac331e0d82d6

Commits

Reviewing files that changed from the base of the PR and between 4ea223a and 52f02b9.

Files selected for processing: 1

• devices/devicekit/voiceover.go

Walkthrough

The change adds VoiceOver state, speech, and cursor-movement operations to device implementations. It exposes these operations through daemon RPC methods and four device CLI subcommands, with input validation and tests.

Changes: VoiceOver controls

Layer / File(s)	Summary
Device VoiceOver operations devices/devicekit/voiceover.go, devices/ios.go, devices/simulator.go, devices/remote.go	DeviceKit and device implementations add VoiceOver state, speech, and cursor-movement operations.
VoiceOver commands and CLI commands/voiceover.go, commands/voiceover_test.go, cli/device.go, commands/registry.go	Commands validate state and direction inputs, invoke a compatible device controller, and return state or speech results. The CLI registers get, set, speech, and move subcommands. Tests cover parsing and validation.
Daemon RPC methods and validation server/server.go, server/dispatch.go, server/daemon_test.go	The daemon registers four VoiceOver methods, validates request fields, and forwards requests to commands. Tests cover missing fields and invalid directions.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant CLI
  participant Daemon
  participant VoiceOverCommand
  participant IOSDevice
  participant DeviceKitClient
  CLI->>Daemon: Send device.voiceover request
  Daemon->>VoiceOverCommand: Forward validated request
  VoiceOverCommand->>IOSDevice: Call VoiceOver operation
  IOSDevice->>DeviceKitClient: Send VoiceOver RPC
  DeviceKitClient-->>VoiceOverCommand: Return state or utterance
  VoiceOverCommand-->>CLI: Return command result

Loading

Suggested reviewers: gmegidish

Merge Risk: Low · previous review 4ea22

Remote VoiceOver changes near the agent timeout may succeed but appear failed to callers; extend this operation’s RPC deadline before merging.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 29.41% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta…	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: adding device VoiceOver controls and speech reading support.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Docstring Coverage

Explanation

Docstring coverage is 29.41% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 17 functions across 11 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches

Generate unit tests (beta)

• Create a new PR

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

CodeRabbit Verify

• Run CodeRabbit Verify to test the changes in this pull request.

Recent review info

Run configuration

• Source mode: Auto
• Repository: mobile-next/mobilecli
• Branch: main
• File: .coderabbit.yaml
• Commit: 83e47ca
• Layers: Repository YAML
• Review profile: CHILL
• Plan: Advanced
• Run ID: 2b57c827-cfa3-4699-94e1-5f7d4fe1ebb3

Commits

Reviewing files that changed from the base of the PR and between 5389b69 and 4ea223a.

Files selected for processing: 2

• commands/voiceover_test.go
• devices/remote.go

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 0 remain after this review.

[coderabbitai]

• Fix comments: Fix CodeRabbit comments on this PR

[coderabbitai]

• Fix comments: Fix CodeRabbit comments on this PR