[earakely-scale]

What and why

M8.12 (FD-4959): the gateway routes sites by their real names.

Until now, a site behind the gateway's /website/<name>/ route always saw its internal address as its Host, shop-website-backend:8000. A site that builds its links and redirects from Host, as many real apps do, then sent chromium to an address chromium can't reach. For example, /login redirected to https://shop-website-backend:8000/home.

Sites built on agentenv-protocol's WebsiteEnvironment already take their address from X-Forwarded-Host, so they never showed this. Any other server did.

Design note: M8.12: Gateway site routes by real names, with decisions 1a, 2a and 3a. It isn't settled until you've reviewed it.

What changes

• The upstream Host. The host a client reached a site at is the one the gateway already forwards: a computer site proxy's AgentEnv-Site-Root origin, else a front's X-Forwarded-Host, else the request's Host.
  • When that host is one of the site's names, it also goes upstream as Host. Case is ignored and the port is kept.
  • Any other host leaves Host the internal address, so a forged one never reaches the site.
  • X-Forwarded-* is unchanged (decision 2a).
  • WebSockets on the route follow the same rule. websockets writes Host from the URI, so the URI names the site and the TCP connection goes to the upstream.
  • /svc/<name>/ is unchanged.
• The site's names. They are the names its env stores, handed to the gateway as WEBSITE_NAMES, else the ones its card declares.
  • The card's are the website/v1 domains, read with the card the route already reads.
  • Most @website(domains=...) sites declare their names only on the card, which is also where a computer's site table takes them from.
  • A *. name matches one label more, as the site proxy matches it.
• list_website_urls follows the run's browser (decision 1a).
  • When the multi env's browser is a browser computer, each synthetic site is listed at its first name: https://shop.test/, or http:// for a .localhost name. The deploy names those sites in WEBSITE_LISTED_BY_NAME.
  • When the browser is playwright-mcp, or for a site deployed alone, sites keep their internal addresses, which is all playwright-mcp can reach.
• HEAD already worked, since Starlette adds it to any route that takes GET. A test now covers it.

Decided while building, for your review

1. Names from the card. A site whose env stores no names takes the ones its card declares.
2. The gateway works out the listed address, from those names, so card-only sites list by name too. The deploy only says which sites are listed by name.
3. A site with no names anywhere is unchanged. It is listed at its internal address and keeps its internal Host. A computer reaches such a site at a .localhost fallback name that depends on the run's other sites, which the gateway can't see. This is a known limitation.

How it was tested

• Unit:
  • the Host rule:
    • over HTTP: 14 cases, among them a site name, HEAD, case and port, a front's X-Forwarded-Host, a wildcard, and forged origins, suffixes, ports and addresses;
    • a site with no names, and /svc/;
    • names taken from the card;
    • over a real WebSocket.
  • list_website_urls: by name, .localhost, wildcard-only, card-only, and not listed.
  • WEBSITE_NAMES parsing.
  • The compose lines.
  • Which browsers list by name.
  • Forwarded hosts that can't be read: an unbalanced bracket, non-ASCII, a control character.
  • The whole unit tier on macOS: 13,022 passed.
    • One test failed: the Windows disk recipe test, which fails the same way on main on this Mac.
    • One file was skipped: test_docker_image_github_clone.py, which hangs here on a git credential helper. Neither touches this change.
• Integration, local Docker, on agent-environments-admin VMs (amd64 and arm64). The new test runs twice: once with the site's name stored on its env and no card, once with the name only on its card.

  • The fixture tst/data/host_website is a plain Starlette server that redirects /login to https://<its Host>/home.

  • Chromium, in a multi env, opens https://shop.test/login and lands on https://shop.test/home.

  • list_website_urls through the gateway's /mcp gives ["https://shop.test/"].

  • Asked straight at the gateway's route, the site sees Shop.Test for that origin, its internal address for https://evil.example, and its internal address with no origin named. X-Forwarded-Host says where each request came from.

  • All of it holds again after a docker restart of the gateway.

  • Results:

    Tier	amd64	arm64
    The new test, both name sources	2 of 2	2 of 2
    browser_computer_test.py -k multi_env and M8.14's test	6 of 6	5 of 6, see below
    multi_env_computers_test.py	4 of 4	4 of 4
    gateway_test.py -k website (playwright-mcp, internal addresses)	4 of 4	not run, see below

  • On amd64 the tiers ran again on the final code, after the second commit: 8 of 8 site and multi env browser tests, 4 of 4 multi_env_computers_test.py, 4 of 4 gateway_test.py -k website. The third commit only names a constant.

  • On arm64, two things the harness builds only for linux/amd64 couldn't run, since the VM has no emulation, and neither is touched here:

    • the echo agent behind test_browser_connect_cdp_hands_an_agent_an_address_at_the_multi_envs_gateway_with_its_tools_hidden (tst/util/image_cache.py defaults to linux/amd64);
    • gateway_test.py's gateway image (docker build --platform linux/amd64).
• Chaos, run on the same VMs and not committed. It passed for both name sources, on both architectures:

  • Fuzz: 552 requests over raw sockets, each naming a host as Host, X-Forwarded-Host or AgentEnv-Site-Root. The hosts included:

    • good names: SHOP.TEST, Shop.Test:443, shop.test:65535;
    • bad ones: a port past 65535, a trailing dot, an empty label, a suffix, userinfo, an IP address and an IPv6 address, a path, a query, %-escapes, non-ASCII and punycode names, 64-character labels and an 8000-character value;
    • 150 random strings.

    No answer was 5xx or hung. The site saw one of its names as Host only when it was reached at that very name, and every good name came through as sent.

  • Concurrency: 300 requests, 50 at a time, each naming a good name, a forged origin or none. Each one's Host was its own.

  • Site killed: docker kill on the site's container. Its route answered 502 at once, and the gateway kept listing it by name. After docker start, the site was back in 1.2 s at its name, and a forged origin still got the internal Host.

  • Gateway restarted under traffic: of about 120 requests, the roughly 10 sent during the restart got connection resets. None of the answered ones broke the rule. Afterwards chromium still landed on https://shop.test/home, and list_website_urls still gave https://shop.test/, including for the card-only site, whose card the restarted gateway read again.

  • The fuzz first found 500s, and they were on main too. An AgentEnv-Site-Root that urlsplit can't parse (https://]x[) and a non-ASCII Host or X-Forwarded-Host, which httpx can't send upstream, each failed the request.

    • The second commit makes each of them name nothing, as an empty one does, so the request falls back to the next header and goes through.
    • It's separate in case you'd rather it go in its own PR.

Fixed on the way

• The forwarded-host 500s the chaos fuzz found (second commit, cf03c33), described above.
• From Greptile (5f63718): a WebSocket to an https upstream reached by name went to port 80 and checked the certificate against the site's name. It now defaults the port by the upstream's scheme, and verifies a wss upstream as itself.
• From a security review (db664a0): the WebSocket relay connects straight to its internal upstream (proxy=None). With the site's name in the URI, websockets' default proxy=True would otherwise follow a proxy in the gateway's environment and tunnel to that name, which is the real domain for a site named after one. No agentenv deploy sets one today.
• From Greptile (f3dfd59): proxy=None needs websockets 15, so the gateway image and the dev extra now ask for websockets>=15. The lock already resolves 17.2, so only its recorded floor changes.
• From your review (ffb4f88): a site's card is read into one SiteCard, its reserved paths and its names together, so the names are no longer a side effect of the reserved-path read.
• f66b596 merges main, keeping feat(gateway): computers follow the clock through a watch, and clock writes wait for them (FD-5471) #614's clock followers beside the site names in the gateway's entrypoint and constructor. The VM runs predate these three commits; CI covers them.

Not in scope

• The tunnel-scheme leftover (decision 3a). A person previewing a site through its gateway route on Modal gets http links. Agents aren't affected.
• The reusable per-site smoke check from FD-4959's Nov 1 scope (sign in, submit a form). The agreed outcome kept the redirect fixture instead.
• Promoting the gateway image after the release. That's your call.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; the previous WebSocket compatibility issue is fixed.

Summary

The PR lets websites receive their declared names as Host and lists named URLs when the run uses a browser computer.

• Sites receive their declared name when the gateway forwards requests.
• Browser-computer deployments list sites at their declared names.
• Unreadable forwarded hosts fall back instead of stopping the request.

_{Reviews (12) · Last reviewed commit: "Gateway: require websockets 15, the firs..." · Reviewed by Greptile}