Repository navigation
Add Android adapter with emulator and installed-device navigation (QUA-2204) - #180
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
QualityMax ReviewReviewing Max's current mission: Max is reviewing the diff from his tiny laptop. The completed review will replace this message. |
There was a problem hiding this comment.
QualityMax Review — canonical overview updated; no inline findings were created for this review.
😸 2 finding(s) auto-dismissed by the Finding Verifier
llm-path-traversalpackages/playwright/src/android.ts:240 — false positive: The file's SHA256 must exactly match the provided hash, preventing arbitrary file reads, and this is a local developer tool.llm-command-injectionpackages/playwright/src/android.ts:286 — false positive: The variable is passed via an argv array to spawn (preventing shell injection) and used as an HTTP base path for a local server, not a filesystem path, making path traversal and command injection inapplicable.
Auto-dismissals are advisory. If a dismissal looks wrong, treat the finding as blocking and request a verifier review.
✅ QualityMax Pipeline
Powered by QualityMax — AI-Powered Test Automation |
There was a problem hiding this comment.
QualityMax Review — canonical overview updated; no inline findings were created for this review.
😸 2 finding(s) auto-dismissed by the Finding Verifier
llm-path-traversalpackages/playwright/src/android.ts:240 — false positive: The file content is strictly validated against a SHA256 hash before any use, preventing arbitrary file reads via path traversal.llm-command-injectionpackages/playwright/src/android.ts:286 — false positive: The subprocess uses an argv list preventing shell injection, and the URL path is for local IPC to a locally spawned server, not a file system path.
Auto-dismissals are advisory. If a dismissal looks wrong, treat the finding as blocking and request a verifier review.
There was a problem hiding this comment.
🔵 Needs a closer look
Native cancellation and privacy-sensitive cleanup require fixes and human review of the revision-specific qualification evidence.
2 open issues
What changed in this PR
Adds an opt-in local Android adapter to the shared target framework, with deterministic resource-ID actions, fresh assertions, and runtime attribution in Go-managed receipts.
Changes:
- Introduces an owned emulator/Appium lifecycle with separately installed Android dependencies.
- Extends target selection, evidence validation, receipt models, and schemas for Android.
- Adds synthetic fixtures, qualification controls, polling regressions, and setup documentation.
Review verification was limited to targeted source inspection; Android qualification and project tests were not rerun.
| File | Description |
|---|---|
| tools/android/package.json | Pins optional Android tooling. |
| testdata/sdk/target-assertion/observation-errors.spec.ts | Tests unavailable-read classification. |
| testdata/sdk/target-assertion/deadline.spec.ts | Tests polling deadlines and cancellation. |
| testdata/sdk/target-assertion.config.ts | Configures polling regression tests. |
| testdata/sdk/android/native.spec.ts | Exercises native assertions and failure controls. |
| testdata/sdk/android.config.ts | Configures the Android qualification project. |
| testdata/android/res/values/ids.xml | Defines deterministic resource IDs. |
| testdata/android/requirements.json | Maps expected native outcomes. |
| testdata/android/MainActivity.java | Implements the synthetic transfer fixture. |
| testdata/android/AndroidManifest.xml | Declares the fixture application. |
| scripts/package-smoke.cjs | Checks Android packaging and optional imports. |
| scripts/build-android-fixture.cjs | Builds isolated fixture APKs. |
| scripts/android-smoke.cjs | Runs the native qualification matrix. |
| scripts/android-preservation.cjs | Checks preexisting-session preservation. |
| packages/playwright/test/android.test.cjs | Tests runtime metadata validation. |
| packages/playwright/src/targets.ts | Adds Android support and polling classification. |
| packages/playwright/src/reporter.ts | Emits bounded Android runtime evidence. |
| packages/playwright/src/protocol.ts | Accepts Android target identifiers. |
| packages/playwright/src/android.ts | Implements the owned local Android driver. |
| packages/playwright/src/android-runtime.ts | Defines and validates runtime metadata. |
| packages/playwright/package.json | Exposes the optional Android subpath. |
| package.json | Adds the Android smoke command. |
| internal/runner/model.go | Adds Android runtime receipt fields. |
| internal/runner/fake_adapter_test.go | Tests scratch cleanup after worker failure. |
| internal/runner/execute.go | Owns native scratch and records runtime metadata. |
| internal/runner/adapter.go | Extends adapter validation results. |
| internal/adapters/playwrightsdk/protocol.go | Validates runtime evidence and test binding. |
| internal/adapters/playwrightsdk/android_test.go | Tests rejection of invalid runtime evidence. |
| internal/adapters/playwrightsdk/adapter.go | Enables Android target planning. |
| docs/targets.md | Documents Android and polling semantics. |
| docs/contracts/run-status.schema.json | Adds runtime metadata to status schemas. |
| docs/contracts/run-result.schema.json | Adds runtime metadata to result schemas. |
| docs/contracts/receipt.schema.json | Adds runtime metadata to receipt schemas. |
| docs/android-driver.md | Documents setup, ownership, and qualification limits. |
| cmd/9l/main.go | Updates target help and validation messages. |
| cmd/9l/android_contract_test.go | Checks required schema test attribution. |
Review effort: Balanced
| // grace only decides whether to close the driver, if work is still pending. | ||
| const timer = setTimeout(() => { | ||
| this.stop('deadline', false); | ||
| this.stop(deadlineMs < this.options.deadlineMs ? deadlineReason() : 'deadline', false); |
There was a problem hiding this comment.
Android drivers now release immediately at an operation deadline, while web drivers retain the artifact-preservation grace. Added a regression for an abort-settled transport. Commit 2528df0.
|
There was a problem hiding this comment.
QualityMax Review — canonical overview updated; no inline findings were created for this review.
😸 3 finding(s) auto-dismissed by the Finding Verifier
llm-path-traversalpackages/playwright/src/android.ts:240 — false positive: The file content is strictly validated against a SHA256 hash, preventing arbitrary file reads, and the write destination is a secure temporary directory, not user-controlled.llm-command-injectionpackages/playwright/src/android.ts:268 — false positive: The subprocess is invoked using an argv list without the shell option, which inherently prevents shell injection.llm-command-injectionpackages/playwright/src/android.ts:286 — false positive: The code uses an argv list for spawn preventing shell injection, and owner is a UUID used in a local URL path, not a file path or shell command.
Auto-dismissals are advisory. If a dismissal looks wrong, treat the finding as blocking and request a verifier review.
There was a problem hiding this comment.
QualityMax Review — canonical overview updated; no inline findings were created for this review.
😸 4 finding(s) auto-dismissed by the Finding Verifier
llm-command-injectionpackages/playwright/src/android.ts:161 — false positive: The code at the specified location (line 161) creates an AbortSignal and does not invoke any shell commands, making the injection finding a false positive.llm-path-traversalpackages/playwright/src/android.ts:168 — false positive: options.appPackage is strictly validated by a regex allowing only alphanumeric characters, underscores, and dots, which prevents any path traversal characters.llm-command-injectionpackages/playwright/src/android.ts:280 — false positive: Line 280 contains no shell command or 'owner' variable; it is a simple conditional check, making the finding entirely inapplicable.llm-command-injectionpackages/playwright/src/android.ts:318 — false positive: Line 318 constructs a file path using a template literal and a SHA256 hash, not a shell command, so no command injection is possible.
Auto-dismissals are advisory. If a dismissal looks wrong, treat the finding as blocking and request a verifier review.
There was a problem hiding this comment.
🟡 Changes recommended
Device leases can be bypassed by differing TMPDIR values, allowing concurrent sessions to interfere, and Android timeout handling can leave native operations running past their deadline.
4 open issues
1 resolved since last review
Review effort: Balanced
c368d77 to
dc4402e
Compare
There was a problem hiding this comment.
QualityMax Review — canonical overview updated; no inline findings were created for this review.
😸 4 finding(s) auto-dismissed by the Finding Verifier
llm-command-injectionpackages/playwright/src/android.ts:161 — false positive: Line 161 contains no shell command, and the shell helper is not shown to be called with user input.llm-path-traversalpackages/playwright/src/android.ts:168 — false positive: options.appPackage is strictly validated by a regex allowing only alphanumeric chars, underscores, and dots, which prevents any path traversal characters like slashes or dots.llm-command-injectionpackages/playwright/src/android.ts:280 — already fixed: The flagged code describing a shell command with an owner variable at line 280 does not exist in the current revision.llm-command-injectionpackages/playwright/src/android.ts:318 — false positive: The interpolated variables are derived from Android package names or internal scratch paths, which cannot contain shell metacharacters, preventing injection.
Auto-dismissals are advisory. If a dismissal looks wrong, treat the finding as blocking and request a verifier review.
Co-authored-by: Desperado <195594+Desperado@users.noreply.github.com>
There was a problem hiding this comment.
QualityMax Review — canonical overview updated; no inline findings were created for this review.
😸 4 finding(s) auto-dismissed by the Finding Verifier
llm-command-injectionpackages/playwright/src/android.ts:161 — false positive: Line 161 is an AbortSignal call, not a shell command, and all configuration inputs are strictly validated with regexes.llm-path-traversalpackages/playwright/src/android.ts:168 — false positive: Theoptions.appPackageis strictly validated by a regex allowing only alphanumeric characters, underscores, and dots, preventing any path traversal or shell injection.llm-command-injectionpackages/playwright/src/android.ts:176 — false positive: The appPackage is passed via an argv list to adb without a shell, and line 176 is unrelated to shell commands.llm-command-injectionpackages/playwright/src/android.ts:318 — false positive: Line 318 only constructs a local file path using a SHA256 hash. It contains no shell commands, URL paths, or user-controlled 'name' parameters, making the finding entirely mismatched.
Auto-dismissals are advisory. If a dismissal looks wrong, treat the finding as blocking and request a verifier review.


Adds an opt-in
android:<app-alias>target using pinned Appium 3.8.0 / UiAutomator2 8.7.0, with an owned local emulator adapter and experimental attachment to an installed app on a physical Android device. Deterministic actions use fresh observed controls and text/state assertions; receipts bind the APK and runtime identity to the active test. This implements the M2 foundation for QUA-2204, within QUA-2202.Physical attachment verifies the installed APK digest, leases the device exclusively on this host, and rejects ambiguous controls before mutation. It preserves the installed app and its data, disables reset, and restores owned automation state during normal cleanup. A private installed-app smoke checks both the selected destination and heading across four navigation destinations. It does not submit changes.
Android dependencies remain separate from the ordinary SDK. The driver defers agent-device 0.21.31 pending privacy qualification of its automatic artifact paths. Screenshots, page source, video, logcat and raw device serials are not retained in receipts. Existing Playwright behavior stays opt-in, and native mutations get one attempt even with
--attempts 3.Validation:
deadlinewith cleanup.npm ci, Chromium installation,npm test(199 passed, no skips), ordinary SDK smoke, shared-target smoke and packed-consumer smoke passed. These include business failures and owned browser/worker timeout/cancellation.go test -race ./...(both required fixture flags enabled),go vet ./...and diff whitespace checks passed.Qualification is limited to these runtimes and recorded cases. Physical-device disconnect and outer-process hard-kill recovery, keyboard/fold transitions and other phones remain unqualified. Private device configuration and execution evidence stay local under
.context/.verifiedAssertionsremains 0 and source assertion coverage remains unknown. Semantic screen checks (M3), the live reporter (M4), the shared finance demo (M5), epic-wide real-run qualification (M6), iOS, goals and replay remain open. The later explicit semantic run is configured by user choice to use the existing Codex CLI login without API fallback; it has not run.No SDK publication, deployment, shared adb shutdown or user AVD configuration change is part of this PR.