earakely-scale · Nov 12, 2026 · Nov 12, 2026 · Nov 12, 2026 · Nov 12, 2026
@@ -139,7 +139,7 @@ and deserialized through a registry.
 | `agents/` | The `agents` bundle: the agents agent-env ships, one folder each under `agents/agents/`, with a smoke task each under `agents/tasks/`. Its Claude Code, `@local/agentenv-framework/agents/claude-code`, is the default agent. An agent's Python packages are its folder's `requirements.txt`, installed in its image, never in agent-env's own environment. A run that pulls one of its agents reads the whole bundle, so `tst/unit/bundle/shipped_bundles_test.py` parses and checks every shipped bundle. |
 | `computers/` | The `computers` bundle: the base computers agent-env ships as `computer` artifacts, `ubuntu-24.04` (XFCE on Xvfb, supervisord as the session user, settings as `AGENTENV_SETTING_<NAME>`), and `omarchy-4.5`, a disk alone built on the composing machine from Omarchy's own VM image (its `[forms.disk]` `source`) by `disk/omarchy/`'s recipe, its services systemd user units of the graphical session that start through `with-seed`, booted by the launcher as a `linux-desktop` guest (`env/envs/computer/utils/omarchy_disk.py`, on `thin_disk.py`, which Windows' disks use too), aliased through `[artifacts]`. It has no tasks either. `disk/linux/` is the Linux disk form's recipe: `boot/`, whose Dockerfile makes a disk form (a GPT disk of an EFI system partition holding one UKI, Ubuntu's kernel with a busybox initramfs whose init reads a NoCloud seed and `switch_root`s into the image's entrypoint, and an ext4 root filesystem that is a container image's, plus `/etc/agentenv/vm/image.env` from its OCI config), `tools/`, the disk tools that make it unprivileged on the builder's own platform (`mkfs.ext4 -d`), and `launcher/`, the QEMU launcher that boots one under KVM. A base's `[forms.disk]` with no image is made from its container form; the composer makes each arch's disk from its composed container, keeping the base disk's EFI system partition, keyed by the container's key, the base disk and the recipe; gate 3 probes it where a provider boots disks natively (`disk_image_platforms()`), and a deploy's first boot with KVM probes one left `unverified`; a deploy naming `form = "disk"` boots it through `create_vm(image=..., env=...)`, its settings in the seed. |
 | `browsers/` | The `browsers` bundle: the website browsers agent-env ships, today `chromium`, a `computer_app` env (Chrome for Testing at the build Playwright pins, for amd64 and arm64, set up for unattended use by managed policies in `/etc/opt/chrome_for_testing/policies/managed`, opened at boot on its `start_urls`, its profile its data). All of it, its pin, install step, policies and command, is in `envs/chromium/chromium.py`; `tst/integration/computer/chromium_app_test.py` runs it. Like `drivers`, it has no tasks. |
-| `drivers/` | The `drivers` bundle: the computer drivers agent-env ships as `computer_driver` envs, the reference driver (X11 on Linux, recording through `x11grab`) aliased `reference`, the page driver aliased `page`, a browser computer's, whose screen is the browser's active tab and which serves Playwright MCP's `browser_*` tools beside `computer_*` from a Playwright MCP it runs attached to the browser over a CDP address the relay issues it (`browser_evaluate` and `browser_run_code_unsafe` hidden by default; `scripts/upgrade_playwright_mcp.py` pins its version), and trycua's cua-driver aliased `cua-driver` (its daemon from the pinned PyPI wheel, its own instructions, tools and skill resources served to agents through `@server_instructions`, `forwarded_tools` and `@resource`, its tools beside `computer_*`, `computer/v1` on its desktop-scope calls, its skill pack leading the computer skill; `tst/integration/computer/cua_driver_test.py` runs it). It has no tasks: a bundle with neither tasks nor evals is refused unless its `aliases.toml` names something for other bundles to use. The shipped `desktop` example composes it with the `ubuntu-24.04` base into a computer env, which `tst/integration/computer/` deploys, probes, records and has an agent use. |
+| `drivers/` | The `drivers` bundle: the computer drivers agent-env ships as `computer_driver` envs, the reference driver (X11 on Linux, recording through `x11grab`) aliased `reference`, the page driver aliased `page`, a browser computer's, whose screen is the browser's active tab and which serves Playwright MCP's `browser_*` tools beside `computer_*` from a Playwright MCP it runs attached to the browser over a CDP address the relay issues it (`browser_evaluate` and `browser_run_code_unsafe` hidden by default; each call an entry in the browser's log, begun and ended with `browser/v1` `calls/begin` and `calls/end`, that its CDP commands name as `for`; `scripts/upgrade_playwright_mcp.py` pins its version), and trycua's cua-driver aliased `cua-driver` (its daemon from the pinned PyPI wheel, its own instructions, tools and skill resources served to agents through `@server_instructions`, `forwarded_tools` and `@resource`, its tools beside `computer_*`, `computer/v1` on its desktop-scope calls, its skill pack leading the computer skill; `tst/integration/computer/cua_driver_test.py` runs it). It has no tasks: a bundle with neither tasks nor evals is refused unless its `aliases.toml` names something for other bundles to use. The shipped `desktop` example composes it with the `ubuntu-24.04` base into a computer env, which `tst/integration/computer/` deploys, probes, records and has an agent use. |
 | `examples/` | The example bundles agent-env ships (`hello`, and `echo-mcp`, which needs Docker, and `desktop`, a computer env of `ubuntu-24.04` and `reference` with a probe task and an agent task). Tests run them, but they are examples for users, not fixtures. Every shipped bundle is registered under `agent_env.bundles` in `pyproject.toml`, `<name> = "<package>"` shipping `src/<package as a path>/<name>/`. |
 
 ### Training helpers

@@ -1013,22 +1013,30 @@ are the profile's template. The base class adds:
   that needs the one before takes it first, with `snapshot_env`.
   - **An action** is a CDP command through the relay (`via: "cdp"`, `who` the connection's user, its `connection`
     number, `method`, and the tab its session is on), an op a driver makes for an agent's MCP tool call (`via: "mcp"`,
-    `who` the caller's role, `tool`, `call`, `op`), or an acting op agentenv makes itself (`via: "agentenv"`). Each
+    `who` the caller's role, `tool`, `call`, `op`), an agent's MCP tool call a driver serves through a CDP user's
+    connections, such as Playwright MCP's (`via: "mcp"`, `who`, `tool`, `call`, with no `op`, between `calls/begin`
+    and `calls/end`), or an acting op agentenv makes itself (`via: "agentenv"`). A CDP command of a user with such a
+    call open names it, `for` its entry, or `for` a list of entries when several are open; the call's entry takes
+    the tab its commands acted on, and keeps what it typed only once each typing command of its own found no
+    password field focused. Each
     carries `tab`, `url`, `ok` and, failed, `error`, and is marked `acts` or not by its method or op
     (`log.acts`, `ACTING_OPS`): input, navigation, tab commands, `Runtime.evaluate` and `callFunctionOn`, DOM
     writes, faked responses, and viewport and cookie changes act; the rest read. An acting entry keeps its `params`
     (and a tool call's `args`) with what could carry a secret taken out: text typed while a password field has focus,
     as its length; cookie and header values; the values of URL query keys that name a credential; a request's body;
     and each string past 2 KB cut, with its length and sha256. A reading entry keeps none.
   - **A tab event** (`opened`, `navigated`, `closed`) says what caused it: `by: "command"` with the `entry`, `via` and
-    `who` of the command that asked for it, matched exactly by the target a new tab's command answers with and the
+    `who` of the command that asked for it, and its `for`, matched exactly by the target a new tab's command answers with and the
     loader a navigation's does; `by: "agentenv"`; `exit` or `stop`; or `by: "page"`, with the `after` of the latest
     acting entry on its tab (its opener's, for a popup), for a click that follows a link, a redirect or a page script.
   - A driver passes an agent's tool call to the browser in the `AgentEnv-Tool-Call` header (`tool_call_header`), with
     the caller's `AgentEnv-Role`; the page driver sends it on every call it makes for a tool. `data/get` writes the log
     beside `browser-state.json` as `browser-log.jsonl`, which `data/add` passes over.
 - **`browser/v1`** on its card: `chromium` (its `version`), `clock: "real"` and, alone, `window`; `log {after}`
-  (GET), framework-only, its log's lines after `after`; and
+  (GET), framework-only, its log's lines after `after`; `calls/begin {user, acts?, within_s?}`, framework-only,
+  sent with the tool-call header, which begins the entry of an agent's tool call that `user`'s CDP connections
+  serve and answers its `entry`, and `calls/end {entry, ok, error?}`, which ends it, a call not ended within
+  `within_s` (600 s by default) written as never said to have ended; and
   `record` (GET), framework-only, which returns that record with `launches`, the app server's log of each
   launch (with its cause: `boot`, `server_restart`, `launch`, `data/add` or `data/reset`), each stop it made
   and each exit it didn't, with its exit `code` or the `signal` that ended it, since the computer booted.

@@ -18,7 +18,8 @@
 from starlette.requests import Request
 from starlette.websockets import WebSocket, WebSocketDisconnect
 
-from .log import ERROR_CHARS, KEY_FIELDS, TYPING_METHODS, BrowserLog, acts, redact_text, redacted_params
+from .log import (ERROR_CHARS, KEY_FIELDS, TYPING_METHODS, BrowserLog, acts, redact_text, redacted_args,
+                  redacted_params)
 
 if TYPE_CHECKING:
     from .control import BrowserControl
@@ -213,22 +214,73 @@ def __init__(self, websocket: WebSocket, user: str, session: str, record: dict,
         self.closing = False
 
 
+class _Call:
+    """An agent's tool call a driver serves through one CDP user's connections, open from its ``calls/begin`` to its
+    ``calls/end``: its entry; the tab its commands last acted on, else the one they last read; whether a password field
+    had focus for its typing, None until a typing command of its own is checked; and when it's given up on."""
+
+    def __init__(self, begun: dict, user: str, call: dict) -> None:
+        self.begun, self.user, self.tool, self.args = begun, user, call["tool"], call["args"]
+        self.tab: Optional[str] = None
+        self.acted = False
+        self.password: Optional[bool] = None
+        self.expiry: Optional[asyncio.TimerHandle] = None
+
+    def credit(self, target: Optional[str], acting: bool, typed: bool, password: bool, alone: bool) -> None:
+        """One of its user's commands credited to it, ``alone`` or with other calls open: the tab it was on, and for
+        one that typed, whether a password field had focus there, taken as so when other calls shared it."""
+        if target and alone and (acting or not self.acted):
+            self.tab, self.acted = target, self.acted or acting
+        if typed:
+            self.password = True if password or not alone else bool(self.password)
+
+
 class Relay:
     """Each WebSocket connection given its own browser-level session over ``devtools``, its commands sent on it past the
     guard and the answers and events of its sessions sent back, so several clients share one browser; and a record of
     who connected when, and what the guard did to their commands. ``dialogs_answered`` says agentenv answers the
     browser's dialogs itself. With a ``log``, each command is an entry in it, on the tab its session is on, and
     ``control``, agentenv's own connection, tells its tab's URL, whether a password field has focus there, and what
-    causes the tab events a command brings."""
+    causes the tab events a command brings. An agent's tool call a driver serves through a CDP user's connections,
+    such as Playwright MCP's, is an entry too, which that user's commands are credited to while it's open."""
 
     def __init__(self, devtools: DevToolsPipe, *, dialogs_answered: bool = False, log: Optional[BrowserLog] = None,
                  control: Optional[BrowserControl] = None) -> None:
         self.devtools, self.dialogs_answered, self.log, self.control = devtools, dialogs_answered, log, control
         self.connections: list[dict] = []
         self._by_session: dict[str, _Connection] = {}
         self._open: set[_Connection] = set()
+        #: The tool calls open now, by their entry's sequence number.
+        self.calls: dict[int, _Call] = {}
         devtools.listen(self._event)
 
+    def begin_call(self, user: str, who: str, call: dict, *, acts: bool, within_s: float) -> int:
+        """``who``'s tool ``call``, which a driver serves through ``user``'s connections, begun as an entry in the log:
+        each of ``user``'s commands is credited to it until ``end_call`` ends it, or ``within_s`` passes and it's
+        written as never said to have ended. Its entry's sequence number."""
+        begun = self.log.begin({"via": "mcp", "who": who, "tool": call["tool"], "call": call["call"], "acts": acts})
+        opened = self.calls[begun["seq"]] = _Call(begun, user, call)
+        opened.expiry = asyncio.get_running_loop().call_later(
+            within_s, self.end_call, begun["seq"], False, f"its driver didn't say it ended within {within_s:g} s")
+        return begun["seq"]
+
+    def end_call(self, seq: int, ok: bool, error: Optional[str] = None) -> bool:
+        """The call begun as entry ``seq`` ended: its entry written with whether it worked and why not, the tab its
+        commands acted on and that tab's URL, and, acting, its arguments, what it typed kept only once every command of
+        its own that typed found no password field focused. False when no such call is open."""
+        call = self.calls.pop(seq, None)
+        if call is None:
+            return False
+        call.expiry.cancel()
+        fields: dict[str, Any] = {"tab": call.tab, "url": self.control.url_of(call.tab)
+                                  if self.control is not None and call.tab else None, "ok": ok}
+        if error:
+            fields["error"] = redact_text(error)[:ERROR_CHARS]
+        if call.begun["action"]["acts"]:
+            fields["args"] = redacted_args(call.tool, call.args, password=call.password is not False)
+        self.log.finish(call.begun, **fields)
+        return True
+
     async def serve(self, websocket: WebSocket, user: str) -> None:
         """``websocket``, a client of ``user``'s, relayed over a browser-level session of its own until either ends."""
         try:
@@ -317,17 +369,21 @@ async def _command(self, connection: _Connection, message: Any) -> None:
 
     def _begin(self, connection: _Connection, session: str, method: str, params: dict) -> Optional[_Logged]:
         """``connection``'s command, begun in the log, if there is one, on the tab ``session`` is on, or for a browser
-        command the tab it names; what it causes made known to the control."""
+        command the tab it names, ``for`` the tool calls of its user's open now; what it causes made known to the
+        control."""
         if self.log is None:
             return None
         target = connection.targets.get(session) if session != connection.session else None
         if target is None and isinstance(params.get("targetId"), str):
             target = params["targetId"]
         acting = acts(method)
         url = self.control.url_of(target) if self.control is not None and target else None
-        begun = self.log.begin({"via": "cdp", "who": connection.user, "connection": connection.number,
-                                "method": method, "acts": acting, "tab": target, "url": url})
-        return _Logged(self, begun, method, params, target, acting)
+        crediting = [call for call in self.calls.values() if call.user == connection.user]
+        action = {"via": "cdp", "who": connection.user, "connection": connection.number, "method": method,
+                  "acts": acting, "tab": target, "url": url}
+        if crediting:
+            action["for"] = crediting[0].begun["seq"] if len(crediting) == 1 else [c.begun["seq"] for c in crediting]
+        return _Logged(self, self.log.begin(action), method, params, target, acting, crediting)
 
     @staticmethod
     def _guarded(connection: _Connection, method: str, verdict: Verdict) -> None:
@@ -398,11 +454,12 @@ class _Logged:
     and its answer."""
 
     def __init__(self, relay: Relay, begun: dict, method: str, params: dict, target: Optional[str],
-                 acting: bool) -> None:
-        self.relay, self.begun, self.method, self.params, self.target, self.acting = \
-            relay, begun, method, params, target, acting
+                 acting: bool, calls: list[_Call]) -> None:
+        self.relay, self.begun, self.method, self.params, self.target, self.acting, self.calls = \
+            relay, begun, method, params, target, acting, calls
         control = relay.control
-        self._cause = cause = {"seq": begun["seq"], "via": "cdp", "who": begun["action"]["who"]}
+        self._cause = cause = {"seq": begun["seq"], "via": "cdp", "who": begun["action"]["who"],
+                               **({"for": begun["action"]["for"]} if "for" in begun["action"] else {})}
         self._landed: Optional[asyncio.Future] = None
         self._checks_password = self.password = False
         if control is None or not acting:
@@ -453,6 +510,8 @@ def finish(self, answer: dict, guarded: Optional[str]) -> None:
             fields["tab"] = self.target = opened
         if self.target and self.relay.control is not None and self.begun["action"].get("url") is None:
             fields["url"] = self.relay.control.url_of(self.target)
+        for call in self.calls:
+            call.credit(self.target, self.acting, self._checks_password, self.password, alone=len(self.calls) == 1)
         self.relay.log.finish(self.begun, **fields)
 
 

@@ -352,6 +352,26 @@ async def browser_sites_set(self, sites: list[dict]) -> dict:
         written.replace(SITES)
         return {}
 
+    @extension(BROWSER, op="calls/begin")
+    async def browser_calls_begin(self, user: str, request: Request, acts: bool = True, within_s: float = 600.0) -> dict:
+        """An agent's tool call, from the tool-call header this is sent with, which a driver serves through ``user``'s
+        CDP connections, begun as an entry in the log, by the caller's role: ``user``'s commands are credited to it
+        until ``calls/end`` ends it, or ``within_s`` passes and it's written as never said to have ended. Its entry's
+        sequence number, ``entry``."""
+        call = read_tool_call(request.headers.get(TOOL_CALL_HEADER))
+        if call is None:
+            raise InvalidArgument(f"calls/begin is sent with the tool call it begins, in its {TOOL_CALL_HEADER} header")
+        who = normalize_role(request.headers.get(ROLE_HEADER)) or DEFAULT_ROLE
+        return {"entry": self.relay.begin_call(user, who, call, acts=acts, within_s=within_s)}
+
+    @extension(BROWSER, op="calls/end")
+    async def browser_calls_end(self, entry: int, ok: bool, error: Optional[str] = None) -> dict:
+        """The tool call ``calls/begin`` began as ``entry`` ended, with whether it worked and, if not, why: its entry
+        written, with the tab its commands acted on."""
+        if not self.relay.end_call(entry, ok, error):
+            raise InvalidArgument(f"no tool call is open as entry {entry}")
+        return {}
+
     @extension(BROWSER, op="cdp")
     async def browser_cdp(self, user: str, request: Request, origin: Optional[str] = None) -> dict:
         """``user``'s CDP address once the browser's ready, the same one each time it's asked for, as the server that