[earakely-scale]

What

A website browser now keeps a network log: every request it makes and every tunnel it opens, from the proxy it goes out through. browser/v1 network {after} reads the log's lines, and data/get exports it as HAR 1.2, network.har, beside browser-log.jsonl. That is the file a network grader such as WebArena-Verified's reads. This is M8.20 (FD-4968), built to the design note: https://claude.ai/artifact/CckmYo6W6A11k1mQ6gcVFX

• Two proxies, one format (D1, D2). The shared site proxy, on a browser alone routing sites, keeps each exchange whole. The base's stand-in, which decrypts nothing, keeps each connection. Both write the same numbered JSON lines: an open line once a request has gone upstream or a tunnel has opened, and an end line, naming the same id, once it's answered and done or has failed. A proxy started again numbers after the file's last line, and ends a line cut short by a crash first.
• Where (D3, changed while building, see below). The shared proxy writes /etc/agentenv/sites/network.jsonl, opened as root, 0644, before it drops to nobody. The stand-in writes /var/log/agentenv/network.jsonl. AGENTENV_NETWORK_LOG names the file to the browser's app server: the base sets the stand-in's, and site routing sets the shared proxy's.
• What an exchange keeps (D5, D6). The request as the browser sent it: its URL, headers and body up to 10 MB. The values of the sign-in headers a site declares are kept as <removed>; none declares any until M8.35. The answer as the browser got it: status, headers (with the proxy's Location and cookie rewrites) and, for text, JSON, forms, XML and scripts, its body up to 1 MB. Any other type keeps only its size and sha256. Bodies are stored as they came, in base64; the export decodes them.
• WebSockets (D7): the 101 answer, then messages and bytes each way, counted from the frames as they pass. Control frames aren't messages.
• The HAR (D4). An entry per request and tunnel, in the order they started, since WebArena-Verified grades the last match in file order. A connection still going is marked _open. Bodies are decoded from gzip and deflate, and from br and zstd with brotli and zstandard, now in the protocol's app extra, up to 1 MB. A coding that can't be undone leaves the body as base64 with _undecoded. agentenv's own fields start with _: _tunnel, _websocket, _failed, _refused, _truncated, _sha256.
• data/add passes over network.har, as it does over browser-log.jsonl. The network op isn't itself logged as an action.

Decided while building (in the note, for review)

1. The shared proxy's log is in /etc/agentenv/sites, not /var/log/agentenv (changes D3). The session user owns /var/log/agentenv, so it could replace a file root opened there. The sites folder is root's (0755) and the file 0644: the session user reads it and can't change or replace it.
2. The log is always on. M8.22 ties it to recordings and network: false.
3. A plain-HTTP request to a live host gets a whole entry in the shared proxy, as a site's does. A tunnel to one stays connection-level.
4. HAR's queryString holds decoded values, as Playwright's HAR does.
5. A refused request's bare host becomes a URL in HAR: https://host:443/ for a CONNECT, else http://host/.

Tests

• Unit:
  • tst/unit/env/envs/test_computer_network_log.py (new) is the golden test. It runs requests through the shared proxy and reads them back as HAR:
    • a GET with a query, cookies, a sign-in header and Referer and Sec-Fetch headers;
    • a form POST answered with a redirect onto the route, a chunked JSON POST answered with gzipped JSON sent chunked, and a multipart upload;
    • a text answer past 1 MB, an image, a Latin-1 page that sets a cookie, and the http listener's 308;
    • a WebSocket's message counts, and frames counted across every cut;
    • on the forward port: a site's tunnel by its requests, a live host's tunnel and its plain request, a host that doesn't resolve, and a refusal;
    • an open entry until it ends; numbering after a restart and a cut line; mode 0644 under umask 077; no link followed; a write that fails is logged and the proxy carries on;
    • the stand-in's connections and failures, and its numbering.
  • packages/agentenv-protocol/tests/test_browser_network.py (new): reading pages, a partial line, a log begun again, the byte limit; HAR's request and response fields, every coding, cut and charset, size and sha256 alone, WebSockets, tunnels, the stand-in's plain requests, failed, refused and open entries, and order.
  • test_browser_data.py: data/get exports network.har, network reads after a number, and data/add passes over the HAR.
  • Rebased on v0.9.1515: the protocol suite and tst/unit/env pass, 3,844 tests. One failure is a Windows-install disk recipe test, in code this branch doesn't touch, which fails on its own on macOS.
• Integration (tst/integration/computer/browser_computer_test.py, local Docker on an agent-environments-admin amd64 VM):
  • New: routing sites, an agent signs in by a form, follows the redirect, saves an item as JSON and uploads a file as multipart. WebArena-Verified 1.2.3's own loader (uvx --from webarena-verified==1.2.3, since its pydantic==2.12.0 pin clashes with ours) reads the network.har that data/get exports. It sees each request's method, URL, form, JSON and multipart fields and referer, each answer's status, redirect, cookie and decoded JSON, and both page loads as navigations.
  • The live-hosts test also checks the site's exchange, the live host's plain request and tunnel, and the refusal. The open-web test checks the stand-in's numbered lines, its tunnels to github.com with their bytes, and the failure.
  • The browser suites in full, rebased on v0.9.1515, on a fresh agent-environments-admin amd64 VM (Node 22): browser_computer_test.py, website_browser_test.py, browser_gateway_test.py, chromium_app_test.py and multi_env_computers_test.py passed 57 of 58, one skipped for want of KVM. The one failure was website_browser_test.py pinning the card's browser/v1 methods without network; the follow-up commit adds it, and that test then passed.
• Chaos on the same VM (an eval module, not committed), each proxy behind a real browser computer:
  • A burst of 1,024 requests, 64 at a time, all answered: 2.4 s through the shared proxy, 4.3 s through the stand-in. Every line is whole JSON and the numbers run without a gap. A poller reading network {after} during the burst got every line once, in order.
  • Killed mid-burst (4,096 requests, 16 at a time): the shared proxy by root's kill -9, the stand-in by the session user's pkill. Each came back (keep and supervisord), and the new process numbered on from the file's last line: no number repeated, no gap, no cut line.
  • The session user reads the shared proxy's log but can't append to it or move it (Permission denied): the file is 0644 and its folder 0755, both root's.
  • A 20 MB upload through the shared proxy is kept to 10 MB and marked _truncated; the proxy's RSS stayed at 44 MB. A HAR of 1,441 entries exports in 0.6 s.

Fixed after review (ee6d2ea)

Greptile's first round, and a second review of the diff against the code before it:

• A plain-HTTP request to a live host goes both ways at once again, as the tunnel did. The first build sent the whole upload before reading any answer, which lost 100 Continue and hung on an early 413. An answer the proxy can't read as HTTP (a header without a colon, bare-LF lines, a head over 1 MB) is relayed as it came, as before, its entry marked _unparsed; the first build turned it into a 400.
• Decoding stops at the limit as it goes, for br and zstd and for request bodies too (10 MB decoded): a 317-byte br answer could decode to 200 MB at data/get. brotli>=1.2, for output_buffer_limit.
• With no network log, bodies are counted but neither kept nor hashed, so a desktop computer's site listeners don't hold 10 MB of every upload for nothing.
• The stand-in writes its open line inside the relay that writes its end line, so a reset or a bad Content-Length can't leave an entry open for good, and it turns down a bad Content-Length with a 400 before it connects.
• A line the HAR can't use is left out and counted in _unreadable, instead of failing the whole data/get; the session user owns the stand-in's log and could write one. The HAR is built off the event loop.
• A cookie's Expires is ISO 8601, as HAR has it.

Each comes with a test; the relay tests fail on the commit before. Unit: the protocol suite and tst/unit/env, 3,856 pass (the same Windows disk recipe test fails on its own on macOS).

Found in chaos, for a decision

• A request in flight when its proxy dies stays _open in the HAR for good: 2 of the shared proxy's, 12 of the stand-in's. The proxy started again could end the dead process's open ids with _failed: {"reason": "the proxy restarted"}. That's a small follow-up if you want it.
• The session user can truncate the stand-in's log, since it owns /var/log/agentenv and the stand-in runs as it, as D3 expected. Numbering carries on, but history is gone. An agent on a browser computer alone has no shell, so this matters only for code run inside the computer.

🤖 Generated with Claude Code

Confidence Score: 5/5

The latest changes appear safe to merge; the previous cookie-date export failure is fixed.

Summary

Adds proxy-backed network logs, reads them through browser/v1 network, and exports them as network.har.

• Browser proxies record requests, tunnels, and connection results.
• The browser exposes its network log as readable lines and a HAR file.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Browser --> Proxy[Shared proxy or stand-in]
  Proxy --> Site
  Proxy --> Log[Numbered open and end lines]
  Log --> Read[browser/v1 network]
  Log --> Export[data/get]
  Export --> HAR[network.har]

Loading

_{Reviews (3) · Last reviewed commit: "fix(browser): a cookie expiry past UTC's..." · Reviewed by Greptile}

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	pypi/​zstandard@​0.25.0

View full report