Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions src/agent_env/drivers/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,17 +47,18 @@ with `[tools]`. `requirements.txt` installs `cua-driver==0.38.0` from PyPI, whos
`cua-driver`, the alias `aliases.toml` gives it, or by its id, `@local/agentenv-framework/drivers/cua-driver`.

On macOS (`arm64`, composed onto a VM bundle such as `macos-26`), cua-driver is CuaDriver.app, from the cua-driver-rs
0.38.0 release, its tarball checked by sha256 and its app put in `/Applications` by `install-app.sh`. Its daemon is a
0.47.10 release, its tarball checked by sha256 and its app put in `/Applications` by `install-app.sh`. Its daemon is a
direct LaunchAgent, CuaDriver.app's own executable, since macOS gives the app's TCC permissions, Accessibility, Post
Event and Screen Recording, which `[install.macos]` grants it, only to the executable launchd starts itself; its socket
is `/tmp/agentenv-cua-driver.sock`. `requirements-macos.txt` installs the protocol alone. Agents get the 62 tools
`mcp-macos.json` pins, as macOS's `cua-driver mcp` lists them; `computer-ready` checks CuaDriver.app's executable's
sha256, those tools, and that cua-driver reports Accessibility and Screen Recording granted. Its pointer's actions go in
the foreground, where cua-driver otherwise keeps from moving a person's pointer over a background app,
and a click or a drag first rests the pointer where it presses, since an app such as Tk reads where the pointer is when it
handles the press, and a button the pointer hasn't yet entered loses the click (P4, a button and a checkbox take a click). cua-driver 0.38.0 fails two of the probe's checks on macOS for defects of its own: it posts each typed
character's key-up with the A key's code (P6, typed text arrives as key events), and a drag's mouse-down after the
pointer has moved along its path (P7). `env.toml`'s `[[known_failures.macos]]` names both with that reason, so gate 3
handles the press, and a button the pointer hasn't yet entered loses the click (P4, a button and a checkbox take a click). cua-driver fails two of the probe's checks on macOS for defects of its own: it types each character
as a key-down and key-up of the A key's code carrying the character as Unicode, so the text arrives but an app reading
key events sees the A key (P6, typed text arrives as key events), and 0.38.0 posted a drag's mouse-down after the
pointer had moved along its path (P7), which 0.47.10, posting a drag's events the same way, hasn't yet shown. `env.toml`'s `[[known_failures.macos]]` names both with that reason, so gate 3
records them in the build's probe (`known_failures`) and doesn't fail the build for them; any other failure still
does. A driver's `known_failures`, by OS type, names a check by its id and name, with why. One that a probe didn't see
fail is recorded in its probe (`unused_known_failures`, with what it saw) and warned of, so a waiver doesn't outlive its
Expand Down
2 changes: 1 addition & 1 deletion src/agent_env/drivers/envs/cua-driver/driver.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ def _desktop() -> str | None:
EXECUTABLES = {
"amd64": "bf57b2f3778a0a7daa9388e241f20ce948b32a7a2f4dd102e497f34e54137934",
"arm64": "26e9eee8165e9ce7b614b5e3d68e5d963231385a82b65236406718b5ce1ff936",
"macos/arm64": "b4d974a9c1c92a700cd7be7ede987aea4a1e6835e37039ce3d628b71bd5f0ca8",
"macos/arm64": "f026e4dea3d058c7c702dfebca93853e6f9410ce76b73c06f35eae1a81bbc873",
"windows/amd64": "ffa58d88779ee219304b0c307f4cfa970fe822f830c6ecf9254eda5c29e22bf2",
"hyprland/amd64": "8dd0f74928cff484343ed65e3ac4d1dacdccc3950b8665e1102ac463af0a967b",
}
Expand Down
8 changes: 4 additions & 4 deletions src/agent_env/drivers/envs/cua-driver/env.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ services = [
[install.macos]
python = "requirements-macos.txt"
downloads = [
{ url = "https://github.com/trycua/cua/releases/download/cua-driver-rs-v0.38.0/cua-driver-rs-0.38.0-darwin-arm64.tar.gz", sha256 = "5f71191c082ca07c9dfc13b7c9a6eec60a82818c245a93534cc0eb7911471d72", to = "CuaDriver.tar.gz", arch = "arm64" },
{ url = "https://github.com/trycua/cua/releases/download/cua-driver-rs-v0.47.10/cua-driver-rs-0.47.10-darwin-arm64.tar.gz", sha256 = "40954056eeb011660abe1ff31693676a5cce6d53b2c0e6afbafb30c9b805c141", to = "CuaDriver.tar.gz", arch = "arm64" },
]
run = [{ script = "install-app.sh" }]
grants = [{ app = "/Applications/CuaDriver.app", services = ["accessibility", "post_event", "screen_capture"] }]
Expand All @@ -43,16 +43,16 @@ services = [
{ name = "driver", command = "venv/Scripts/python.exe -m agentenv_protocol.computer.serve driver.py:CuaDriver" },
]

# cua-driver 0.38.0's own defects on macOS, which gate 3 records without failing the build.
# cua-driver's own defects on macOS, which gate 3 records without failing the build.
[[known_failures.macos]]
check = "P6"
name = "typed text arrives as key events"
reason = "cua-driver 0.38.0 posts each typed character's key-up with keycode 0, the A key, so an app sees 'a' released; its pressed keys release as they should"
reason = "cua-driver 0.47.10 types each character as a key-down and key-up of keycode 0, the A key, carrying the character as Unicode, so the text arrives whole but an app reading key events sees the A key; Return, Tab and the keys computer_key presses are real keys"

[[known_failures.macos]]
check = "P7"
name = "a drag starts and ends where it says, through its waypoints"
reason = "cua-driver 0.38.0 posts a drag's mouse-down after the pointer has moved along its path, on some first probes after a boot 3 to 10 px from the start"
reason = "cua-driver 0.38.0 posted a drag's mouse-down after the pointer had moved along its path, on some first probes after a boot 3 to 10 px from the start; 0.47.10 posts a drag's events as 0.38.0 did and passed its first 13 probes, too few to call an intermittent failure fixed"

# cua-driver 0.38.0's own defect on Windows, which gate 3 records without failing the build.
[[known_failures.windows]]
Expand Down
2 changes: 1 addition & 1 deletion src/agent_env/drivers/envs/cua-driver/install-app.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ set -eu
work=$(mktemp -d)
tar -xzf CuaDriver.tar.gz -C "$work"
rm -rf /Applications/CuaDriver.app
ditto "$work/cua-driver-rs-0.38.0-darwin-arm64/CuaDriver.app" /Applications/CuaDriver.app
ditto "$work/cua-driver-rs-0.47.10-darwin-arm64/CuaDriver.app" /Applications/CuaDriver.app
rm -rf "$work"
xattr -dr com.apple.quarantine /Applications/CuaDriver.app || true
rm -f CuaDriver.tar.gz
Loading
Loading