feat(studio): bring Mecatl Studio in-repo as a Node module

[Derek2Tu]

Summary

• Bring Mecatl Studio in-repo as a normal Next.js Node module.
• Support two pure deployment modes: managed local mecated, or authenticated external MECATL_BASE_URL.
• Harden both proxy layers with server-side bearer injection, Host/Origin/CSRF gates, bounded bodies, restricted MCP egress, random loopback ports, and redacted OAuth errors.
• Replace source-regex tests with behavior tests around SSR, external auth, controller policy, gateway policy, protocol decoding, terminal errors, schedules, and unknown events.
• Remove Cloudflare/vinext and demo/voiceover residue; run with next dev / next start.
• Add a path-scoped Studio workflow, pinned Node version, npm Dependabot coverage, and high-severity audit gating.

Scope decisions

Included here: all deployment-shape, security, CI, dependency, protocol-runtime, residue-removal, and documentation blockers raised in both review passes.

Separate follow-up PRs:

• generated TypeScript bindings from the protobuf contracts (this PR adds the typed runtime seam and behavioral drift tests without introducing repo-wide TS codegen);
• Unix-domain-socket support or :0 HTTP bind-and-report in mecated;
• a designed configuration/authentication UI rather than a key-paste stopgap;
• splitting the controller and repeated modal shells as the next responsibilities land;
• a larger real-daemon/offline browser integration suite beyond the hermetic proxy/controller tests here.

Verification

• task lint
• task test (including race tests and standalone module builds)
• task build
• task docs
• task site:build
• go run ./cmd/mecademo
• npm test
• npm run lint
• npm run typecheck
• npm audit --audit-level=high (0 vulnerabilities)
• managed-mode controller smoke test with generated bearer auth and random HTTP/gRPC ports

Open-PR overlap audit

• No other open PR touches studio/** or .github/workflows/studio.yml.
• PR feat: add microVM execution environments #580 shares only top-level workflow docs, Taskfile, ADR index, architecture, and generated docs; its hunks are distinct from Studio after removing the shared opening-sentence edit.
• PRs feat: steer-while-running — inject a user message into an in-flight run (#512) #570 and feat(mcp): per-server SSE opt-out + automatic GET-hostile detection #417 touch different ADR/readiness rows plus generated llms.txt output.
• The README edit that overlapped the broad rewrite in docs: frame mecatl as a cloud-native agent kit #499 was removed from this PR.
• The Studio ADR uses unique identifier 0225; the collision with current main and feat: add microVM execution environments #580 was resolved before push.

[jbeda]

First-pass review — deployment shape & architecture notes. Haven't been through the code line-by-line yet; this is the structural feedback from a first read of the shape (worker proxy, controller, spawn path) against how we want to deploy it.

Assuming up front: the core deploy targets are (a) everything-on-localhost (single-user dev box) and (b) k8s, and in both we want to be able to split the tiers — UI local, mecated remote. Every note below serves making that split first-class. (Non-blocking nits excluded; these are the structural items.)

1. Provider credentials — don't build key-paste UI now; hard-fail to the auth file, and treat config/auth UI as its own follow-up.
Today the controller receives the OpenRouter key over HTTP, holds it in memory, and injects it into the child's env (local-controller.mjs:649,339). That's a janky copy-paste-on-startup experience and a secret in transit — and any real use of mecated already requires the operator to own their config (settings.yaml, auth.yaml), so a one-off key-entry card optimizes the wrong moment.

Proposal: drop the /openrouter credential endpoint and the env injection. Instead, when the selected provider has no credential available, fail hard with a clear, actionable message pointing at the auth file — mecated already resolves --auth-file / $XDG_CONFIG_HOME/mecatl/auth.yaml via the shared cliconfig.ProviderFlags seam (same one mecatui and mecak8s use), so the flow is "write openrouter: {api_key: …} to auth.yaml, restart," with no secret crossing the controller and no per-provider env forwarding to maintain (readProviderKeys, cliconfig.go:208-215, is deliberately the single env-var definition; hand-patching {...process.env} is the fragility it retired).

A proper config/auth UI (not editing JSON/YAML by hand, but also not a paste-a-key box) is worth doing — but as a separate, designed thing, not this PR. This PR shouldn't ship the stopgap.

2. Make the managed mecated lifecycle optional — the mecatui two-mode pattern (ADR 0087).
This is the split-UI-from-daemon enabler. mecatui already has it: pure local (embedded) vs pure connect ADDRESS (external, never probes, never embeds). Studio wants the same two pure modes:

• managed (default, current behavior): controller spawns/supervises mecated on loopback.
• external: MECATL_BASE_URL points the worker's /api/mecatl/* proxy at a hosted mecated; the controller's spawn/restart duties no-op (or the controller doesn't run at all — its provider-config duties are the managed-mode half).

For external mode, the pre-configured-daemon route is right: the hosted mecated owns its provider creds at its own launch (auth file / env, the normal way); Studio just connects and the credential/config surface hides or disables. No SetProviderCredential RPC — that can come later if ever needed.

3. Auth on the /api/mecatl/* proxy is the open gap, and the UI/daemon split forces it.
mecated has the full edge machinery (static --auth-token bearer + OIDC Validator + rate limiting, shared across gRPC and HTTP). Studio uses none of it: the worker forwards requests verbatim (worker/index.ts:47), and the controller never passes --auth-token to its child. Today that's papered over by all-loopback; the moment MECATL_BASE_URL is off-host (external mode, k8s) the chain is unauthenticated end-to-end. Minimum viable: the proxy injects Authorization: Bearer $MECATL_AUTH_TOKEN server-side (config, not browser-supplied), and managed mode passes a generated --auth-token to its child — which also closes the "any local process / stray browser tab can drive 127.0.0.1:8081" hole on localhost.

4. Don't hardcode mecated's port in managed mode — pick a random free port now; UDS or :0-and-report as follow-ups.
The proxy targets hardcoded 127.0.0.1:8081 (worker/index.ts:44, and the controller's readiness probe at local-controller.mjs:368). A fixed port collides with a second Studio instance, a hand-run mecated, or anything squatting 8081. For this PR (no mecated changes): the controller should pick a random free port and pass it to both the child (--http-addr 127.0.0.1:<port>) and the proxy. Two cleaner follow-ups, both requiring small mecated changes, so deliberately not this PR:

• Unix domain socket — Node's fetch (undici) supports a socketPath dispatcher, but mecated's HTTP listener is TCP-only (main.go:1822-1829); only mecatui's embedded gRPC server uses a unix socket, in-process. A --http-socket listener would give the single-machine case no port at all + filesystem permissions. Follow-up PR.
• Bind :0 and report the picked port — mecated logs the bound gRPC addr from the listener (grpcLis.Addr().String(), main.go:1815) but the HTTP line just echoes the configured string (main.go:1822), so --http-addr 127.0.0.1:0 would log :0, not the real port. Follow-up: net.Listen explicitly for HTTP (as gRPC already does), log lis.Addr(), then Serve(lis); the controller already tails child stderr (local-controller.mjs:349-352) and can parse the stable line. Follow-up PR.

Until one of those lands, a controller-picked random free port is the right no-server-change behavior.

5. The Cloudflare Worker runtime is the wrong hosting shape for both targets.
worker/index.ts depends on CF bindings (ASSETS, IMAGES, ExecutionContext, vinext/server/*); the image-optimization endpoint is meaningless off-Cloudflare, and for localhost you're running a Node dev server emulating a worker to serve Next — three layers of pretend. The actual proxy logic is ~40 lines. Dropping the workers runtime for Next rewrites() in next.config.ts (which can own both /api/mecatl/* and /api/mecatl-control/* proxying) deletes worker/, vinext, .openai/hosting.json, and a chunk of the lockfile; the same next start container then works behind a k8s ingress unchanged. Looks like leftover standalone-repo hosting residue, same class as the D1/Drizzle wiring already dropped.

6. Known gap (author-flagged): the studio CI job is described in ADR 0110 + AGENTS.md as existing but is not in .github/workflows/ci.yml. Should land before/with merge.

Happy to turn any of these into a concrete patch sketch if useful.

[jbeda]

Second-pass review — multi-agent panel findings. A review panel (spec adherence, repo standards, and specialist domain reviewers — security, architecture, DevOps, library-reuse, duplication) went over the diff against origin/main. Condensed findings below; happy to drill into any of these or pair on fixes.

Ship-blockers

1. CSRF on the controller — any web page can drive it (CWE-352 / OWASP A01). local-controller.mjs:390-679 never checks Host/Origin and parses any content type. A no-cors POST from any page open in the user's browser is CORS-safelisted and executes without a preflight. Concrete state-changing exploits, zero user interaction:

   • POST /mcp (:655) connects an attacker-controlled MCP gateway → forged tool results / prompt injection into the agent's Bash/Edit tools → RCE in the user's repo.
   • POST /openrouter (:646) swaps the provider key for the attacker's → every prompt/completion visible via the attacker's OpenRouter account logs.
   • POST /model-router (:626) writes YAML to disk; POST /restart repivots the daemon.
     Fix: Host/Origin allowlist (loopback only) + a CSRF gate on state-changing POSTs. Pinning Host also closes DNS-rebinding reads of /status and rebind-driven POST /approve against mecated on :8081 — one fix, three holes. This compounds the no-auth-on-proxy finding from the first-pass comment: the controller is a second, richer unauthenticated attack surface on the same loopback.

2. SSRF via /mcp-proxy/* (CWE-918 / OWASP A10). local-controller.mjs:411-441 forwards arbitrary method/body/headers to a caller-chosen gateway.url (loopback HTTP permitted, :657-658), with an unbounded request body. Reachable after the CSRF above or by tricking the operator. Treat gateway.url as untrusted: restrict egress, cap the body, don't allow loopback HTTP from a browser-driven request.

3. The wire-protocol↔view seam is typeless, and the test suite certifies drift instead of catching it. The browser folds raw session.Event SSE into state via ad-hoc loose types (page.tsx:711-805); ScheduleRow decoding hand-rolls the proto JSON encoding (:563-587). ADR 0110's "a breaking HTTP/SSE change owes a Studio update in the same PR" is a social rule with no mechanical backing — a proto field rename sails through green. Meanwhile tests/rendered-html.test.mjs pins source regexes (/event\.result\?\.stop === "error"/), so an honest refactor fails the suite while a protocol change passes it — the inverse of the repo's guard-test discipline. Suggest generating TS types from contracts/proto/mecatl/v1 (buf TS codegen, the same proto mecatui renders from) so drift is a tsc error, and moving the suite's invariant assertions toward behavior/types rather than source text. Related: unknown event types are silently dropped by applyEvent's fallthrough — provider.route, no_progress, new team events would be invisible in the web client where the TUI surfaces them.

Docs claim more than the diff delivers

• CI job: ADR 0110:50-54, AGENTS.md:85, and PRODUCTION-READINESS.md:45 ("✅ … + CI") all describe the studio CI job in present tense, but no .github/workflows/ change is in the diff. Either the job lands in this PR (YAML below, matching house conventions) or the docs get scoped to "CI job follows."
• Memory panel: user-docs/what-you-get/studio.md:42 and docs/architecture.md:307-309 claim a read-only view of both memory stores, but page.tsx:1405-1406 renders project memory as "Enabled, not yet inspectable" (no /v1/memory endpoint exists). Docs should match shipped behavior.
• "Starter-template residue dropped": disabled, not removed — studio/.openai/hosting.json ships {"d1":null,"r2":null}, vite.config.ts:9,17-22 reads it with chatkit-template names, and build/sites-vite-plugin.ts:18-43 copies it plus a drizzle/ dir that doesn't exist in the tree. Delete the plugin, its registration, and the file (~75 lines + 2 files).
• Scope creep (~600 lines, unclaimed): the demo/voiceover tooling — playwright.demo.config.mts, scripts/record-demo.sh, scripts/add-voiceover.sh, scripts/narration-script.mjs, tests/demo/* (a five-act walkthrough referencing a Stacklok-staging gateway URL) — plus the @playwright/test devDep. Suggest moving to a studio/demo/ overlay or dropping; it's starter/marketing residue in the same class as what was already dropped.

Mechanical / smaller

• Pin Node in-repo (studio/.nvmrc + node-version-file in CI); package.json declares only a floor.
• Add npm to .github/dependabot.yml (/website + /studio) — 570-package tree, no vuln automation; consider npm audit --audit-level=high in the job.
• yagni: react-server-dom-webpack in devDependencies — no import anywhere.
• shrink: extract a <Modal> wrapper (six repeated a11y shells, page.tsx:1235-1485); a jsonError(res, code, msg) helper (eight sites in local-controller); delete the one-caller fetchWithTimeout wrapper (:167-169).
• stdlib: readStream SSE framing → TextDecoderStream + async iteration (page.tsx:807-839); relativeTime → Intl.RelativeTimeFormat (:1532-1539).
• Token-echo risk: local-controller.mjs:287 logs raw OAuth token-endpoint error bodies — a malicious endpoint can reflect the access_token into stderr. Don't echo raw upstream OAuth responses.
• Judgement call: local-controller.mjs is four responsibilities in one 700-line file (supervisor + secrets + OAuth client + YAML writer). Fine today; worth splitting into controller/ modules before the next responsibility lands.

Proposed CI job (matches house conventions — SHA-pinned, persist-credentials: false, mirrors the user-docs job)

  studio:
    name: Studio (build/test/lint/typecheck)
    runs-on: ubuntu-24.04
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
        with:
          persist-credentials: false

      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
        with:
          node-version: '22'
          cache: 'npm'
          cache-dependency-path: studio/package-lock.json

      - name: Studio checks
        working-directory: studio
        run: |
          npm ci
          npm test
          npm run lint
          npm run typecheck

Verified clean along the way: the not-a-Go-module boundary (go.work / depguard / DAG / task test untouched), no engine/internal imports, workspace resolution (no absolute paths), studio:dev requiring bin/mecated, studio:stop teardown order, the OAuth dance itself (state + PKCE S256 sound, exact-match loopback redirect, HTTPS-validated endpoints, no XSS in the callback page), four of the five studio/CLAUDE.md invariants holding in code, lockfile committed with full integrity hashes, and demo scripts cleanly quarantined out of npm test.

[Derek2Tu]

@jbeda Thanks for both review passes. I addressed the ship/deployment blockers and rebased the branch onto current main.

Addressed in this PR

• Removed the OpenRouter key-paste endpoint/UI and child-env secret injection. Managed mode now uses normal auth.yaml resolution and reports an actionable startup error; external mode leaves provider ownership with the remote daemon.
• Added pure managed/external modes. MECATL_BASE_URL, MECATL_AUTH_TOKEN, MECATL_WORKSPACE, and MECATL_STUDIO_PUBLIC_ORIGIN configure external deployment; local spawn/config controls are absent or disabled there.
• Added server-side bearer injection. Managed mode generates a daemon token; browser-supplied authorization is never forwarded.
• Removed fixed listener collisions: managed HTTP selects a free loopback port and unused gRPC binds to loopback :0.
• Removed Cloudflare/vinext/worker hosting and starter/demo residue. Studio now uses normal Next route handlers plus next dev / next start.
• Added Host/Origin checks and a non-simple request gate at both Next and controller boundaries, covering CSRF and DNS rebinding.
• Hardened MCP proxying with an unguessable internal path, bounded bodies, JSON content-type checks, credential-in-URL rejection, HTTPS-only remote egress, and explicit loopback-only HTTP opt-in.
• Stopped logging raw OAuth token-endpoint response bodies.
• Added a typed runtime protocol seam and behavior tests. Terminal errors and schedules are decoded structurally, and unknown/non-rendered events surface as notices rather than disappearing.
• Replaced source-regex tests with hermetic SSR/proxy/security/protocol behavior tests.
• Corrected memory, CI, provider-auth, deployment, and architecture docs; removed the demo claim/residue mismatch.
• Pinned Node via .nvmrc, added path-scoped SHA-pinned Studio CI, npm Dependabot entries for /website and /studio, high-severity audit gating, and removed unused dependencies. Audit is clean.
• Applied the small stdlib/refactor items: TextDecoderStream, Intl.RelativeTimeFormat, shared jsonError, and removal of the one-call timeout wrapper.

Deliberately separate follow-ups

• Protobuf-generated TypeScript bindings. This PR creates the typed runtime seam and drift-sensitive behavioral coverage; adding a repo-wide TS proto toolchain should be reviewed independently.
• Unix socket support or HTTP :0 bind-and-report in mecated.
• A designed auth/config UI (the paste-a-key stopgap remains removed).
• Controller/module and repeated modal extraction before adding more responsibilities.
• A broader real-daemon browser integration layer beyond the hermetic tests here.

Verification

task lint, task test (race + standalone modules), task build, task docs, task site:build, go run ./cmd/mecademo, Studio build/tests/lint/typecheck, npm audit, and a managed-controller smoke test all pass locally. The new Studio Actions job is running now.

Open-PR overlap

I re-audited every open PR after the final push. No other PR touches studio/** or .github/workflows/studio.yml. #580 shares only top-level integration/docs files at distinct hunks; #570/#417 share different ADR/readiness rows plus generated llms.txt; and I removed the README edit that overlapped #499. The ADR was moved to unique ID 0225 to avoid the current-main/#580 collision.

[Derek2Tu]

Review follow-up: changes made from both comment passes

This is the complete mapping from the PR feedback to the current branch.

First-pass deployment and architecture feedback

1. Remove the provider key-paste UI — done. The /openrouter controller endpoint, browser key form, and child-process credential injection are gone. Managed mode uses the standard auth.yaml resolution path and returns an actionable startup error; external mode leaves provider credentials with the remote daemon.
2. Pure managed and external modes — done. Managed mode supervises a local daemon. Setting MECATL_BASE_URL selects external mode, skips the local controller lifecycle, disables local configuration mutations, and uses MECATL_WORKSPACE for new sessions.
3. Authenticate the Studio-to-daemon proxy — done. Next injects MECATL_AUTH_TOKEN server-side and strips browser-supplied authorization. Managed mode generates a random bearer token and gives it only to the child daemon and controller proxy.
4. Remove fixed-port collisions — done for this PR. Managed HTTP selects a free loopback port; unused gRPC binds to loopback :0. Unix sockets and HTTP :0 bind-and-report remain separate daemon changes.
5. Replace Cloudflare/vinext hosting — done. The Worker, vinext/Vite adapter, .openai/hosting.json, and related dependencies are removed. Studio now uses ordinary Next route handlers and next dev / next start.
6. Add Studio CI — done. .github/workflows/studio.yml is path-scoped, SHA-pinned, uses studio/.nvmrc, and runs install, build/behavior tests, lint, typecheck, and high-severity audit.

Second-pass ship blockers

1. Controller CSRF and DNS rebinding — done. Both Next and the controller enforce allowed Host/Origin policy. Controller mutations also require a non-simple internal request header, and browser-visible proxy routes do not expose controller secrets.
2. MCP proxy SSRF and unbounded bodies — done. The internal MCP route uses an unguessable path, bounded bodies, JSON content-type validation, credential-in-URL rejection, HTTPS-only remote targets, and an explicit loopback-only HTTP opt-in.
3. Typeless protocol/view seam and source-regex tests — addressed in scope. studio/lib/protocol.ts defines the runtime event/schedule seam; terminal failures and schedules are decoded structurally; unknown or non-rendered event kinds surface as notices. Tests now exercise SSR, authenticated external proxying, controller policy, gateway policy, and protocol behavior rather than matching source text. Generated protobuf TypeScript bindings are intentionally a separate codegen/toolchain PR.

Documentation and residue findings

• CI claims: corrected and backed by the new workflow.
• Memory claims: docs now distinguish the inspectable user-model index from project memory, which is enabled but not inspectable.
• Starter residue: removed the hosting manifest, Vite adapter, Worker, and stale template wiring.
• Demo/voiceover scope: removed the Playwright demo suite, recording/narration scripts, voiceover helper, staging references, and Playwright dependency.
• Deployment/auth/provider docs: updated for managed/external mode, standard auth files, server-side bearer injection, public-origin configuration, and the actual ownership boundary.
• ADR collision: Studio moved to unique ADR 0225 after rebasing onto current main.

Mechanical and dependency feedback

• Added studio/.nvmrc and CI node-version-file usage.
• Added npm Dependabot coverage for /website and /studio.
• Added npm audit --audit-level=high; the lockfile currently reports 0 vulnerabilities.
• Removed unused react-server-dom-webpack and the Cloudflare/Vite/demo dependency cone.
• Added a shared jsonError helper and removed the one-call fetchWithTimeout wrapper.
• Replaced manual SSE decoding with TextDecoderStream and relative-time formatting with Intl.RelativeTimeFormat.
• Stopped logging raw OAuth token-endpoint response bodies.
• Controller-module splitting and repeated modal extraction remain follow-up refactors, before adding more responsibilities.

Verification and overlap audit

• Local verification passed: repository lint, race tests, builds, docs generation/link checks, docs site, offline demo, Studio behavior tests/lint/typecheck/audit, and managed-controller smoke.
• GitHub Actions is fully green, including Studio, repository race tests, lint, perf, kind e2e, docs, build, and vulnerability checks.
• No other open PR touches studio/** or .github/workflows/studio.yml.
• feat: add microVM execution environments #580 shares only distinct top-level integration/docs hunks; feat: steer-while-running — inject a user message into an in-flight run (#512) #570 and feat(mcp): per-server SSE opt-out + automatic GET-hostile detection #417 share different ADR/readiness rows plus generated llms.txt; the broad README overlap with docs: frame mecatl as a cloud-native agent kit #499 was removed from this PR.

[jtenniswood]

#618 proposes replacing this PR's UI with the Atrium workspace while carrying this PR's infrastructure forward — the piece-by-piece mapping:

Carried near-verbatim (same behavior, new paths under studio/src/):

• lib/server-proxy.ts → src/lib/server-proxy.ts (origin trust, header allowlists, bearer injection, external-mode 409 policy; extended with server-side workspace injection)
• lib/controller-security.mjs → src/lib/controller-security.mjs (unchanged, with its truth-table tests)
• scripts/local-controller.mjs + scripts/dev-local.mjs → unchanged apart from one import path
• lib/protocol.ts → src/lib/protocol/ (split into events/sessions/schedules; the feat(studio): back the chat list with the daemon's session store #615/feat(studio): let the Schedules panel author and audit schedules #616 decoder designs re-landed on top)
• tests/rendered-html.test.mjs → same hermetic harness, extended (workspace-injection + offline-503 tests)
• .github/workflows/studio.yml, the dependabot entry, studio/Taskfile.yml + root include, and the ADR posture (re-landed as ADR 0228/0229 — 0225/0227 were taken on main meanwhile)

Replaced: app/page.tsx and the single-page UI, superseded by the five-surface Atrium workspace.

This PR's deferred follow-ups that remain open in #618's ADR: generated TS proto bindings, daemon-side listener work, a designed auth/config UI, the controller module split, and the real-daemon browser test layer.

If #618 lands, this PR can close as superseded — leaving that call to its author. 🤖