diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index bfec15e..e961f04 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -41,8 +41,8 @@ jobs: node-version: "24" - name: Extension unit tests # The gate itself lives in a script, so the pull-request workflow (test.yml) runs exactly what - # this does. It discovers every extensions/*/test/*.test.js and modules/*/test/*.test.js, and - # fails if it finds none. + # this does. It discovers every extensions/*/test/*.test.js, modules/*/test/*.test.js and + # web/test/unit/*.test.js, and fails if it finds none. run: ./scripts/test-extensions.sh - name: Open VSX still signs with a key this release trusts # The app verifies extensions against Open VSX's signing key, and the key is pinned in the app diff --git a/.gitignore b/.gitignore index 82616f0..ce8da70 100644 --- a/.gitignore +++ b/.gitignore @@ -38,3 +38,8 @@ node_modules/ .DS_Store *.log out/ + +# LevelCode in your browser: the release directory scripts/build-web.mjs writes, and the bundle's +# generated assets module (web/ai-extension/build.mjs removes it after bundling). +/dist-web/ +/web/ai-extension/assets.generated.js diff --git a/CLAUDE.md b/CLAUDE.md index d50a766..444ca6b 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -42,8 +42,9 @@ extensions/levelcode-sync/ 'levelcode' auth provider that lights up the bu extensions/levelcode-updater/ notify-only update checker (polls the update feed; never auto-applies) modules/extension-signature/ Open VSX signature verifier the built app loads as @vscode/vsce-sign, + the signing keys it trusts patches/levelcode-core.patch our core source edits, applied on bootstrap -scripts/ bootstrap.sh, apply-branding.mjs, run-dev.sh, editor-identity.mjs, extension-signature.mjs, build-macos.sh, make-dmg.sh, make-icon.sh; atom (CLI launcher) + install-level.sh +scripts/ bootstrap.sh, apply-branding.mjs, run-dev.sh, editor-identity.mjs, extension-signature.mjs, build-macos.sh, make-dmg.sh, make-icon.sh, build-web.mjs; atom (CLI launcher) + install-level.sh tools/ dependency-free reference servers: sync-server (/v1 Settings-Sync), update-server (/api/update feed) +web/ LevelCode in a browser tab: page entry + callback, scratch-workspace extension, layout, themes + chat skin, browser build of levelcode-ai, dev server, tests (docs/WEB.md) vscode/ GITIGNORED upstream Code-OSS checkout (generated) ``` @@ -111,6 +112,39 @@ module for that slot. Full account + runbooks: `docs/EXTENSION-SIGNATURES.md`. - Its tests live in `modules/extension-signature/test/` — outside `extensions/`, whose `test/` folders ship in the app. `scripts/test-extensions.sh` discovers `modules/*/test/*.test.js` too. +## LevelCode in a tab (`web/`) — read `docs/WEB.md` first + +The same `extensions/levelcode-ai` source runs in a browser tab (web-worker extension host). Rules that keep +the desktop app exactly as it was: + +- **A workspace read or write goes through `extensions/levelcode-ai/host.js`**, never `fs` on a workspace path: + in a tab the workspace is whatever the editor's file-system provider says, and every read is asynchronous. + On the desktop each `host` function does what the call site did before, behind `await`. +- **Nothing outside `host.js` asks "am I in a browser?"** — a call site asks for a capability (`host.caps.shell`, + `.mcpStdio`, `.ripgrep`) or calls a function that already knows. A new tool that needs a shell/disk/process is + listed in `NEEDS_SHELL` (agent.js) so the browser neither offers it nor promises it. +- The browser build is `web/ai-extension/` (esbuild + shims for `fs`/`path`/`os`/`crypto`/`child_process`). + Strings that are true on a Mac and false in a tab are rewritten by `web/ai-extension/copy.js`, which fails the + build when the desktop text it expects is gone — change both together. `manifest.js` drops the commands and + settings a tab cannot honour. +- `web/main.js` is a plain ES module with a **top-level `await`**: a `const` declared lower in the file is not + initialised until that await finishes (this cleared a saved session once). Declare constants above the + `try`, and use function declarations for helpers it calls. +- **The look is web-only and lives in `web/`** (theme package, `chrome.css`, the chat skin, the layout commands; + `docs/WEB.md` "The look and the layout"). The chat skin is cut into the browser build's **copy** of + `media/chat.html` by `web/ai-extension/skin.js`; never edit the desktop file for it (its suites pin the page). + Every skin rule is under `body.lc-web`. An extension's `configurationDefaults` outrank the embedder's, so + `levelcode-themes`' pinned default theme is stripped from the staged copy (`withoutConfigurationDefaults`). + `node web/test/e2e-look.mjs --dist dist-web --stub-port ` is the check; look at its screenshots (`--shots`). + The embedder's command API is the module's `commands` (`api.commands.executeCommand`): `create()` returns a + disposable, so `workbench.commands` is `undefined` — a helper that used it ran inside a `try` and silently did + nothing until a browser test asked for its effect. +- The session store (`createSecretStorage`) belongs to every tab: no cache, one-key read-modify-write under a + Web Lock. The refresh token is rotated on use; do not reintroduce a start-up snapshot. +- Build: `node scripts/build-web.mjs --account ` → `dist-web/`; serve with `node web/serve.mjs --dist dist-web`. + Tests: the unit gate (`scripts/test-extensions.sh` includes `web/test/unit/*.test.js`) and the end-to-end checks in + `web/test/e2e*.mjs` (headless Chrome, stand-in backend; Node 22+). What was not run is listed in `docs/WEB.md`. + ## Toolchain (hard requirements — these bit us) - **Node = `vscode/.nvmrc` (currently 24.15.0)**. Older majors fail to compile native modules. @@ -242,7 +276,7 @@ big-file mode badge. Files: extension.js + fileOps/lineOps/columnOps/encodingEol is not in the editor until you load it: `./scripts/run-dev.sh --extensionDevelopmentPath=/extensions/levelcode-ai` (from the main checkout; the dev extension replaces the built-in one). Uncommitted work is not "on the branch" — checking the branch out somewhere else gets none of it. Run it in the editor before telling anyone to try it. -- Commit `extensions/`, `modules/`, `patches/`, `branding/`, `scripts/`, `docs/`, `PLAN.md`, `CLAUDE.md`. Never commit `vscode/`. +- Commit `extensions/`, `modules/`, `patches/`, `branding/`, `scripts/`, `web/`, `docs/`, `PLAN.md`, `CLAUDE.md`. Never commit `vscode/` or `dist-web/`. - `extensions/levelcode-ai/diagram/` modules listed in `bundle.FILES` are pasted INTO a script block in `chat.html`. They must never contain the text of a script tag or an HTML comment opener — not even in a comment — or the block ends early; `bundle.js` refuses to build if one does. Keep them dependency-free and free of `require('vscode')`/`fs`. diff --git a/README.md b/README.md index d999fa0..22bb894 100644 --- a/README.md +++ b/README.md @@ -36,7 +36,8 @@ LevelCode is a **clean overlay on top of Code-OSS**, not a vendored copy of the - `branding/` — the LevelCode identity (`product.overlay.json`) + icons, deep-merged onto upstream `product.json`; the extension gallery wired to **Open VSX**. - `modules/` — what ships in the app outside an extension: the Open VSX signature verifier and the signing keys it trusts. - `patches/` — the small set of core source patches (each tagged `// [LevelCode]`). -- `scripts/` — bootstrap / run-dev / build / dmg / icon. +- `scripts/` — bootstrap / run-dev / build / dmg / icon, and `build-web.mjs` (LevelCode in a browser tab). +- `web/` — LevelCode in your browser: the same workbench and AI extension as a static site you can host; sign in with a LevelCode account or use your own key. See [`docs/WEB.md`](docs/WEB.md). - `tools/` — dependency-free reference servers (Settings-Sync feed, update feed). - `PLAN.md`, `docs/`, `CLAUDE.md` — the vision, roadmap, and repo map. diff --git a/docs/WEB.md b/docs/WEB.md new file mode 100644 index 0000000..7121071 --- /dev/null +++ b/docs/WEB.md @@ -0,0 +1,437 @@ +# LevelCode in your browser + +The same workbench, the same `levelcode-ai` extension, in a browser tab: sign in with a LevelCode +account and chat with the agent against the gateway, or bring a provider key. There is nothing to +install and no server of ours that runs code — the editor is a **static directory**. + +This page is the engineering reference: how it fits together, how to build and serve it, what the +host has to provide, the security model, and — kept separate on purpose — what has and has not been +run. For the product's own page and copy see the account site (`/web`, `/docs/browser`). + +- [What it is, and what it is not](#what-it-is-and-what-it-is-not) +- [How it fits together](#how-it-fits-together) +- [The look and the layout](#the-look-and-the-layout) +- [Signing in](#signing-in) +- [Build and run](#build-and-run) +- [Deploying](#deploying) — origins, headers, backend settings +- [Security model](#security-model) +- [Where state lives](#where-state-lives) +- [Keeping the desktop and the browser in step](#keeping-the-desktop-and-the-browser-in-step) +- [Tests](#tests) — and [what was not run](#what-was-not-run) +- [Decisions that are the owner's](#decisions-that-are-the-owners) + +## What it is, and what it is not + +A Code-OSS **web client** (the `vscode-web-min-ci` build of the pinned checkout) with a +**web-worker extension host**, plus LevelCode's own extensions loaded as built-ins. + +| Works in a tab | Stays in the Mac app | +| --- | --- | +| The workbench: explorer, editor tabs, themes, search, Quick Open | A terminal | +| Chat, inline completion, the agent: read / search / create / edit / delete files | `run_command`, background commands, auto-verify commands | +| Keep / Undo review of the agent's edits | MCP servers that start local processes | +| Sign in with a LevelCode account (gateway), **or** your own provider key | Installing extensions (no marketplace in the page) | +| A scratch workspace saved in the browser (every modern browser) | Importing another editor's settings, Settings Sync, the update check | +| A folder from your computer (File System Access API: Chrome, Edge) | The Notepad++ pack and the customization pack (not loaded in the page), Agent Sketch | + +The browser build offers the agent **only the tools it can honour**: with no shell the shell tools +are withheld from the model rather than offered and failing, and the system prompt is derived from +the same lines so it never promises one (`host.caps`, see below). + +## How it fits together + +``` + account site (levelcode.ai) editor origin (any static host) + ┌───────────────────────────┐ tab goes to ┌──────────────────────────────────────────────┐ + │ /ai/login (sign-in) │◀─────────────────│ index.html ← build-addressed /_//… │ + │ /api/levelcode/v1/… │──── ?code= ─────▶│ main.js embedder: config, callbacks, │ + │ auth/exchange │ /callback.html │ secrets, workspace, commands │ + │ account/models │ │ callback.html hands the result back (storage)│ + │ ai/chat/completions │◀──── fetch ──────│ ┌─ workbench (Code-OSS web client) ────────┐ │ + └───────────────────────────┘ (CORS) │ │ web-worker extension host (iframe) │ │ + │ │ levelcode-ai esbuild bundle + shims │ │ + provider you chose ◀──── fetch (BYOK) ──────│ │ levelcode-web scratch workspace │ │ + │ │ chat webview (own origin, see below) │ │ + │ └──────────────────────────────────────────┘ │ + └──────────────────────────────────────────────┘ +``` + +| Path | What it is | +| --- | --- | +| `scripts/build-web.mjs` | One command → `dist-web/`: the client, the extensions, `index.html` with the configuration baked in, `_headers`, an nginx fragment, `build.json`. Changes nothing outside `--out`. | +| `web/main.js`, `index.html`, `boot.css` | The page. Plain ES module, no build step: reads `#levelcode-web-config`, creates the workbench, supplies the URL-callback provider, the secret store, the workspace provider and the `levelcode.web.openAuthUrl` command. | +| `web/callback.html`, `callback.js` | The page a sign-in returns to. | +| `web/lib/config.mjs`, `headers.mjs`, `extensions.mjs` | The configuration, the HTTP policy and the extension staging — shared by the dev server and the release, so what is tested is what ships. | +| `web/ai-extension/` | The browser build of `levelcode-ai`: `build.mjs` (esbuild, `platform: browser`), `shims/` (`fs`, `path`, `os`, `crypto`, `child_process`, `process`), `manifest.js` (the desktop manifest minus what a tab cannot do), `copy.js` (the few strings that are true on a Mac and false in a tab). | +| `extensions/levelcode-ai/host.js` | The one place the desktop/browser difference lives in the extension (see below). | +| `web/workspace/` | The `levelcode-web` extension: the **scratch workspace** (`levelcode-scratch:` file system in IndexedDB, with file and text search), and the commands to open a local folder or the scratch. | +| `web/theme/levelcode-web-theme/`, `web/chrome.css` | The two colour themes (dark and light) and the stylesheet that tidies the workbench's chrome. See [the look](#the-look-and-the-layout). | +| `web/ai-extension/skin.js`, `skin/chat.css`, `skin/chat.js` | The chat's browser skin, cut into the browser build's copy of `media/chat.html` at build time. | +| `web/serve.mjs` | Dev / end-to-end server. Serves a build (`--dist`) with its own `_headers`, or the pieces (`--static`, `--extensions`). | +| `web/test/` | Unit tests (`unit/`) and the end-to-end checks (`e2e*.mjs`) with a hand-written CDP driver and a stand-in backend. | + +### The extension runs unchanged — through `host.js` + +`extensions/levelcode-ai` is the **same source** on both. Its Node-only assumptions are handled in +two places only: + +1. **`web/ai-extension/shims/`** — the build swaps `fs`, `path`, `os`, `crypto`, `child_process` + for small in-memory/IndexedDB stand-ins for the extension's *private* state (sessions, memory, + images). `path` and `crypto` are compared against Node's over many inputs. +2. **`extensions/levelcode-ai/host.js`** — the user's *workspace* is not in the shim and must never + be: in a tab it is whatever the editor's file-system provider says, and every read is + asynchronous. `host` gives `readText`, `readBytes`, `stat`, `exists`, `isBinary`, `findFiles`, + `withReads`, `openAuth` and the capability flags `caps.shell`, `caps.mcpStdio`, `caps.ripgrep`. + On the desktop each function does exactly what the call site did before, behind `await`. + +The two rules that keep the desktop app exactly as it was: + +- **On the desktop every `host` function does what the call site did before** — the same `fs` + call, the same error. +- **Nothing outside `host.js` asks "am I in a browser?".** A call site asks for a capability + (`host.caps.shell`) or calls a function that already knows. + +## The look and the layout + +The first thing a visitor sees is the chat, not an IDE: a centred greeting with four starters and one rounded +composer, with the list of conversations docked on its left when there is room, and the Explorer and the files one +click away in the status bar. It is made **without forking the workbench** — every piece is a setting, a built-in +extension, a stylesheet or a build-time edit of the browser build's own copy of a file. + +| Piece | Where | What it does, and what it depends on | +| --- | --- | --- | +| Colour themes | `web/theme/levelcode-web-theme/` (staged as a built-in extension by `build-web.mjs`) | "LevelCode Web Dark" / "LevelCode Web Light": neutral chrome with the editor one step lighter, hairline borders, one violet accent. They carry the `agentsPanel.*` keys the floating cards use. Follows the system (`window.autoDetectColorScheme` + the two preferred-theme settings). The syntax colours are One Dark's and One Light's with every one under 4.5:1 on the editor moved in lightness only, hue kept (the light constants darkened with them, so numbers stay apart from class names); `theme.test.js` holds every syntax colour to 4.5:1. The editor paints no token *backgrounds*, so none is set (`invalid` is red text). | +| First paint | `initialColorTheme` in `web/lib/config.mjs`, picked by `pickInitialTheme` in `web/main.js` | The colours painted for the half second before the theme extension has loaded, so a dark visitor does not see a light flash. | +| Default theme | `withoutConfigurationDefaults` in `web/lib/extensions.mjs` | An extension's `configurationDefaults` **outrank** the embedder's. `levelcode-themes` pins its own default theme, so the staged copy of it has that key removed (its two themes stay selectable). The desktop source is untouched. | +| Panels, activity bar | `workbench.experimental.modernUI`, `workbench.activityBar.location: top`, `workbench.layoutControl.type: toggles` | Code-OSS's own switches. `modernUI` is **experimental**: if a bump renames or drops it the editor is still fine, only plainer (`e2e-look.mjs` still passes its theme and layout checks; its screenshots are the thing to look at). | +| Chrome | `web/chrome.css`, linked by `main.js` **after** the workbench's stylesheet | Hides the activity-bar entries for source control, run and extensions (this edition has none of them), the Outline and Timeline headers, the keyboard-layout status item and the title bar's secondary-side-bar toggle. The selectors are workbench class names and aria labels: when a bump stops matching them `e2e-look.mjs` fails rather than the icons quietly coming back. Their **menus, palette entries and shortcuts are the workbench's own** and still lead to empty views (Run and Debug, Extensions, Source Control, an empty Terminal panel). | +| The workbench's own chat | `hideBuiltInChat` in `main.js` | Code-OSS registers its Copilot Chat view on the web even with `chat.disableAIFeatures`, and opened it fills the window with a dead composer ("No default agent contributed"). Its `when` clause is gated by `chatAccountPolicyGateActive`, which nothing else reads; `main.js` sets it (and `chatSetupDisabledInWorkspace`) through the embedder's `commands.executeCommand('_setContext', …)` — the **module's** `commands`: `create()` returns a disposable, not the workbench — and again after five seconds, because the gate's own contribution writes its key again then. `e2e-look.mjs` opens the bar and fails on a "Chat" tab. | +| Editor padding | `editor.padding.top: 24` | `modernUI` fades the top 24 px of every editor into the ground, line one included. | +| No page-level `color-scheme` | `index.html`, `boot.css` | A page that declares `color-scheme: dark light` made every stock webview (the Markdown preview) paint an opaque *white* page under a dark theme. The chat page paints its own ground (`body.lc-web { background: … }`) for the same reason. | +| Welcome page | `web/ai-extension/copy.js` | The desktop opens its walkthrough once, in front of everything. In a tab the chat is the first thing there is; the walkthrough stays in the Command Palette. | +| The chat page | `web/ai-extension/skin.js` + `skin/chat.css` + `skin/chat.js` | Three exact insertions into the **embedded copy** of `media/chat.html`: a style block, `class="lc-web"` on ``, a nonce'd script. Every rule is under `body.lc-web`; the script sends only messages the page already sends. The build **fails** if chat.html stops having the places the skin is cut into. The desktop's `chat.html` is never edited for this (the desktop suites pin it). | + +**Layouts.** `levelcode.web.layout` is `chatFirst` (the chat alone, side bars closed, conversations on its +left — the way a chat app is used) or `split` (Explorer, files on the left, the chat docked on the right — the +way an editor is used). The status bar item at the left and **LevelCode: Layout: Switch** change the setting +(there is no default key binding: `Cmd+Alt+L` is Find in Selection on macOS) where it takes effect — the workspace's +value when `.vscode/settings.json` has one, since that outranks the user's, the user's otherwise; one listener +applies it, so Settings does the same. The first start in a workspace is `chatFirst`. The chat panel is not restored by a reload, so a +`split` layout docks the chat again at start — without revealing the Explorer or moving the focus (that is done +once per workspace, `workspaceState`), so an Explorer the visitor closed stays closed. The start-up and a switch wait +for the chat's tab to exist (`onDidChangeTabs`, 2.5 s) rather than reading the tab model at a guessed time. Only +existing workbench commands are used, so what the visitor then arranges by hand is an ordinary layout. Files that are +open when going back to `chatFirst` stay open as tabs behind the chat: a group that still has files (a split file +editor, say) is joined into the chat's (`workbench.action.joinAllGroups`, which moves them in inactive), and that one +group is unlocked, because a group the chat was docked in is locked and would send the next file into a new split. + +**Widths.** The chat page decides by **its own width**, not the window's: from 1000 px it docks the +conversation list as a 264 px rail; below 760 px it compacts; in between the list is the dialog it is on the +desktop. On a touch device (no hover) the rail's card actions are always shown, as in the dialog, and the second +line gives way; with a mouse they appear on hover or focus. Enter and Space on a card resume it (the desktop's +card is a `div` with a click handler only; the skin adds the keys). In `split` the chat is a narrower column, so it will usually be in the dialog form. The *workbench* is +not responsive: below about 760 px the Explorer, tabs and status bar are the desktop's, and a phone is not a +supported size for this edition yet. + +Constraints worth knowing before changing it: + +- The chat webview's CSP has no `font-src`: the skin uses the system font stack only. +- A group that is emptied by moving the chat out of it is closed, so `split` opens a file in the first group + **before** it moves the chat — the README, or an empty untitled document when there is none and the chat is the + group's only tab (`applyLayout`, pinned by `workspace.test.js`). +- The move and join commands act on the **active editor**, and `levelcode.ai.focus` returns before the reveal has + reached the tab model. The window is read only once the chat is in front (until then an editor that was in front + of it, such as a settings editor in a group of its own, still counts as a group). Each move or join then runs only + when the tab model says the chat is the active editor, read immediately + before it (after waiting up to 1.5 s for that); otherwise it is skipped, as is every later one in that arrangement, + and so is one that a newer layout request has overtaken. A skipped move leaves the window as it was; a guessed one + would move the file beside the chat. +- The rail asks the page for its conversations with `listSessions`; it must never call `openSessions()`, + which seeds the list with sample conversations when nothing has arrived yet. +- Text drawn in the theme's accent uses `textLink.foreground`, not `focusBorder` (which is translucent and a line + colour: 2.2–2.7:1 as text). The footer row is not dimmed with `opacity`: that dims its popovers as well. +- `chat.html`'s own palettes for the conversation list and the account card (`--cc-*`, One Dark) are re-pointed at + the theme by the skin. + +## Signing in + +The extension's existing PKCE flow, with the browser's pieces in place of the OS's: + +``` +1 extension accountSignIn(): verifier → SecretStorage; builds + /ai/login?redirect_uri=&code_challenge= +2 extension host.openAuth(uri) → command levelcode.web.openAuthUrl (registered by main.js) +3 main.js checks the address is /ai/…, notes where the tab is (sessionStorage), location.assign +4 account already signed in there → mints a one-time code bound to the challenge, 302 to + /callback.html?vscode-reqid=N&vscode-scheme=levelcode + &vscode-authority=levelcode.levelcode-ai&vscode-path=%2Fauth%2Fcallback&code=… + (not signed in → the login page first) +5 callback checks it is a sign-in to levelcode.levelcode-ai/auth/callback, writes localStorage + "vscode-web.url-callbacks[N]" (dated, the code only), goes back to where the tab was +6 main.js at start-up, once the workbench is listening, delivers a fresh result as + levelcode://levelcode.levelcode-ai/auth/callback?code=… to the URL service +7 extension handleAuthCallback: a sign-in is pending (a verifier is stored) → + POST /api/levelcode/v1/auth/exchange {code, verifier} → tokens +``` + +Why the tab leaves instead of opening a pop-up: a window opened from a chain that began in a +webview is the first thing a strict browser (Safari, a phone) refuses. Leaving the tab and +returning needs no permission. The PKCE verifier lives in SecretStorage, so it survives the +navigation — and where the browser lets nothing persist (no IndexedDB or WebCrypto, or a refused +write), the tab's secrets ride through that one navigation in sessionStorage and are taken back, +and removed, when the editor starts again. The tab comes back to the address it left: a folder or +workspace opened from the computer is opened again, not the scratch workspace. + +- **`?signin=1`** (the account site's "Open LevelCode" link) makes step 1 happen by itself, once: + `main.js` removes the parameter, then delivers `levelcode://…/launch`, which focuses the chat and + starts the sign-in only when there is no session. +- **A result is delivered only if it is fresh** (written in the last two minutes — the one-time + code lives for a minute) and is removed either way. One left by a callback page whose editor + tab is gone is delivered when the editor next starts. +- **`callback.html` is a page anyone can link to**, and what it leaves is delivered at the next start. + So it passes on a LevelCode sign-in and nothing else: for `levelcode.levelcode-ai/auth/callback` + only, and only the one-time `code` — never a token in the link (the old flow, which the desktop + still accepts) or the rest of the query. The extension then takes a code only while a sign-in of + its own is pending (`host.caps.unaskedSignIn` is off in a browser): a link nobody asked for cannot + sign the user in to someone else's account. +- **`levelcode.web.openAuthUrl` is an open-redirect if it is loose**, and any extension in the page + can call it. It goes only to `/ai/…`. +- The address handed over is `encodeURI(uri.toString(true))` — the string the editor's own opener + produces — so the server sees the same address however the sign-in was started. The backend + decodes `redirect_uri` until stable (three rounds at most) and then applies the exact rule in + `Levelcode::EditorCallback`. + +## Build and run + +Needs the bootstrapped Code-OSS checkout (`scripts/bootstrap.sh`, Node 24.x as the pin requires). + +```bash +# the release: one static directory (about a minute for the client, then bundling) +node scripts/build-web.mjs --account https://levelcode.ai --out dist-web + +# …serve it exactly as a static host would, with its own _headers +node web/serve.mjs --dist dist-web # http://127.0.0.1:8800 +``` + +| `build-web.mjs` flag | Meaning | +| --- | --- | +| `--account ` (`LEVELCODE_WEB_ACCOUNT`) | The LevelCode Cloud origin: where sign-in happens, and where the gateway is. Baked into `index.html`. | +| `--api-url ` (`LEVELCODE_WEB_API_URL`) | Only when the API is not on the account origin (production has one host for both): a local account site with the backend behind a tunnel, say. The gateway must be `https`. | +| `--webview-origin 'https://{{uuid}}.view.example.com'` (`LEVELCODE_WEB_WEBVIEW_ORIGIN`) | Self-host the webview origin. Without it the page uses Code-OSS's CDN at this build's own commit. Needs wildcard DNS and TLS — see below. | +| `--ext-host-origin 'https://{{uuid}}.ext.example.com'` (`LEVELCODE_WEB_EXTHOST_ORIGIN`) | Run the extension host on an origin of its own. Optional while no third-party code is loaded. | +| `--static ` | Use a prebuilt client (skips the gulp build). | +| `--id ` | Override the build id (default: a hash of everything the build put under `/_//`, staged first and named after it). | + +**Dev server** (`web/serve.mjs`): `--static --extensions --extension-names +levelcode-ai,levelcode-web --account `, plus `--product ` and `--config-defaults +` to override the product configuration. The extension is built with +`node web/ai-extension/build.mjs --out --vscode `. + +**Release layout.** `index.html` and the callback page are at the root, un-cached. Everything +else is under `/_//` and cached for a year (`immutable`): `static/` (the client), `extensions/`, +`main.js`, `boot.css`. A cold first load fetched about 20 MB uncompressed (5 MB gzip) over 147 +files from the editor's origin, plus the webview host page from the CDN; the whole directory is +about 195 MB, most of it grammars and extensions the page does not request until they are used. +(Measured as the size of the files the page asked for, not as bytes on the wire.) + +## Deploying + +It is a static site. What the host must supply: + +### Three origins, and which is optional + +| Origin | Default | Why it is separate | +| --- | --- | --- | +| **Editor** — `index.html`, `callback.html`, `/_/…` | the host you deploy to | The only origin that holds the session. | +| **Webview** — where the chat panel (and every webview) runs | `https://{{uuid}}.vscode-cdn.net`, Code-OSS's CDN at this build's own commit | Webview content must not share the editor's origin. **Each webview needs its own subdomain**, so a self-hosted origin has to be a wildcard (`*.view.example.com` with a certificate for it); a single host fails with "Expected '' as hostname or subdomain". | +| **Extension host** — the iframe the worker runs in | the editor's own origin | Isolating it (`--ext-host-origin`, a wildcard likewise) puts third-party extension code on a different origin than the session. Not needed while only LevelCode's extensions load. When it is on, **every** `fetch` an extension makes carries that origin, so the API's CORS must allow it. | + +### Headers + +`dist-web/_headers` (Netlify, Cloudflare Pages) and `dist-web/deploy/nginx.conf` carry the same +policy; `web/serve.mjs --dist` reads the same file, so the end-to-end checks run under it. The +Content-Security-Policy for the editor is the tightest the workbench allows: + +- `script-src 'self' 'unsafe-eval' blob:` — Monaco and the TextMate/WASM engines compile code at runtime; +- `connect-src 'self' https: http://localhost:* http://127.0.0.1:*` — a provider key is used from the tab, to the + provider the user chose; loopback is the one place a browser allows plain http from an https page, for a + model server on the user's own machine (an OpenAI-compatible endpoint through the custom provider — *not run*); +- `frame-src 'self' [] data:`; `object-src 'none'`, `base-uri 'none'`, + `form-action 'none'`, `frame-ancestors 'none'`. + +The callback page has `default-src 'none'` and only its own script and style. + +### On the account site + +Three settings on the Rails app (all default **off**; with none set nothing changes): + +| Variable | Meaning | +| --- | --- | +| `LEVELCODE_WEB_EDITOR_ORIGINS` | Comma list of origins the editor is served from, exactly (`https://editor.example.com`). A sign-in code is only ever sent to `/callback.html` with the right parameters. Also the CORS list. | +| `LEVELCODE_WEB_EDITOR_URL` | Where the account site's "Open LevelCode" goes: one of those origins and, optionally, a path. | +| `LEVELCODE_WEB_EXTENSION_HOST_ORIGINS` | Comma list (one wildcard each) of origins the extension host runs on, when isolated. CORS only — never a callback. | + +`GET /api/levelcode/v1/web_editor` answers `{ enabled, url }`; the account SPA shows its entry +points only when it is on. + +**Do not host the editor on a subdomain of the account host.** The account session cookie is +`SameSite=Lax`, which is a site boundary, not an origin one: it is sent on a request from a sibling +subdomain, and the API takes that cookie when there is no bearer token. A page that runs other +people's extensions there could cause credentialed requests (CORS stops it *reading* the answer, not +causing the effect). Use another registrable domain. + +## Security model + +- **The session is the user's tokens.** They are in SecretStorage, which in a tab is sealed with + AES-GCM under a **non-extractable** key kept in IndexedDB, and stored in localStorage. That keeps + tokens out of a localStorage dump, a backup or a copied profile. It is **not** a defence against + script running on the editor's origin, which can use the same key — so **the page loads no + third-party extension** (`extensionsGallery: null`) and nothing is installed into its origin. +- **The store belongs to every tab.** The refresh token is rotated on use; a tab that cached it + would present a token another tab had rotated away and end a session the other just renewed. Reads + go to storage each time and a change is read-modify-write of one key under a Web Lock. +- **One command, one destination** — `levelcode.web.openAuthUrl` (above). +- **A sign-in code is only handed to a listed origin, to the exact callback page**, by the backend; + the page then takes it out of the address bar (`history.replaceState`) and storage. +- **Keys of the user's own** (BYOK) are stored the same way and used from the tab to the provider the + user chose; no LevelCode call is involved (the end-to-end check asserts it). The provider must + accept requests from a web page; Anthropic's is called with its browser-access header. +- **The scratch workspace and the extension's private state are on the extension host's origin**, in + IndexedDB. With the host isolated they are not readable from the editor's page, by design. + +## Where state lives + +| What | Where | Notes | +| --- | --- | --- | +| Session tokens, BYOK keys | localStorage `levelcode-web.secrets.v1` (sealed); the key in IndexedDB `levelcode-web` → `keys` | editor origin; shared by tabs | +| Sign-in result in flight | localStorage `vscode-web.url-callbacks[N]`; sessionStorage `levelcode-web.return` | removed when read | +| Scratch workspace | IndexedDB `levelcode-scratch` → `nodes` | extension host origin; changes are broadcast to other tabs | +| Chat sessions, memory, images | IndexedDB `levelcode-ai-fs` → `files`, loaded at start | extension host origin; a change made in another tab shows after a reload | +| Editor settings, layout | the workbench's own storage | editor origin | + +Clearing the browser's site data removes all of it. A private window may discard it on close; the +secret store then falls back to memory for the session. The key the secrets are sealed with is created +once per browser profile: two tabs opened together on a fresh profile store it in one IndexedDB +transaction, so both end with the same key. + +## Keeping the desktop and the browser in step + +- **A new tool that needs a shell, a disk or a local process** declares it: add the tool name to + `NEEDS_SHELL` (or gate on `host.caps`) in `agent.js` so the browser neither offers nor promises it. + The browser system prompt is built from the same `SYSTEM_LINES`. +- **A new workspace read or write goes through `host`.** `fs` on a workspace path would work on the + desktop and quietly do nothing in a tab. `extensions/levelcode-ai/test/` and `web/test/unit/` check + the places this has bitten. +- **Copy that is true on a Mac and false in a tab** (keychain, "your machine") is rewritten by + `web/ai-extension/copy.js`, which **fails the build** when the desktop text it expects is gone. Update + the desktop and `copy.js` together. +- **`web/ai-extension/manifest.js`** removes the commands and settings a tab cannot honour. A new + command that shells out belongs in `REMOVED_COMMANDS`. +- **A Code-OSS bump** can change the web client's packaging and the webview service worker's version; + rebuild and run the end-to-end checks. The webview template names this build's own commit + (`{{quality}}/{{commit}}`) for that reason. + +## Tests + +| Check | Command | What it proves | +| --- | --- | --- | +| Unit gate | `./scripts/test-extensions.sh` | Includes `web/test/unit/*.test.js`: the shims against Node, `host.js` on both sides, `main.js`'s functions (address check, delivery, the shared secret store, `?signin=1`), the manifest, the copy rewrites, the chat skin (what it inserts and that every rule is scoped), the themes (required colours, contrast of the text pairs, every syntax colour at 4.5:1 on the editor, the page's defaults naming themes that exist) and the layout commands. Runs in CI on every PR (`test.yml`). Also run on Linux / Node 18 in a container with the network refused. | +| End to end | `node web/test/e2e.mjs --dist dist-web --stub-port ` | Real headless Chrome: boot from static files, the extension activates in the worker, sign in through the real callback page, agent tool calls (write / list / search / read / edit) against the scratch workspace, Quick Open, Keep and Undo, a reload that keeps the session and files. | +| The look | `node web/test/e2e-look.mjs --dist dist-web --stub-port ` | The skinned chat (greeting, starters, composer in the reading column, the rail at 1440 px and the dialog at 900 px), light and dark following the system with this build's own colours (also in the chat's webview), no Welcome page, no second chat, source control / run / extensions not offered, the layout switch both ways, and the docked layout docked again after a reload. Release builds only. | +| Same-tab sign-in | `node web/test/e2e-signin-tab.mjs --dist dist-web --stub-port ` | No second window; verifier survives the reload; no repeat; no sign-in for a signed-in editor; a second tab uses the same session; a stranded result is delivered, an old one dropped. | +| Bring your own key | `node web/test/e2e-byok.mjs --static … --extensions …` | The request goes from the tab to the provider with the user's key and nothing of LevelCode's. | + +The end-to-end checks use a **stand-in backend** (`web/test/stub-backend.mjs`, HTTPS with a +throwaway certificate) that applies the same redirect and CORS rules the real one must. They prove +the editor's half of the contract; the backend's half is its own specs. The hand-written CDP driver +needs Node 22+ and a Chrome (`CHROME=` to point at one). + +The checks were run against a release served with its own `_headers` (same-origin extension host, the +webview from Code-OSS's CDN), and against the development server with the webview and the extension +host on origins of their own (`--product` with `webviewContentExternalBaseUrlTemplate` and +`webEndpointUrlTemplate`). + +### Trying it yourself, against a real backend + +The repo's checks use a stand-in for the account site. To use the real one, locally: + +1. **Backend** — the thin.ly branch with the web edition, its usual dev server and https tunnel, plus + `LEVELCODE_WEB_EDITOR_ORIGINS=http://localhost:8800` and `LEVELCODE_WEB_EDITOR_URL=http://localhost:8800/`. + (The gateway refuses plain http, so the editor's API host has to be the tunnel.) +2. **Account site** — the onetime branch: `npm run dev` (it proxies to the backend), at `http://localhost:5173/ai`. +3. **Editor** (from the levelcode repo root; the build needs the bootstrapped `vscode/` checkout, and a git + worktree has none, so pass `--vscode ` there): + + ```bash + export API_URL=https://your-tunnel.example.com # the https address of your backend + node scripts/build-web.mjs --account http://localhost:5173 --api-url "$API_URL" --out dist-web + node web/serve.mjs --dist dist-web # http://localhost:8800 + ``` + +4. **What to do, and what you should see:** + - Signed out everywhere: open `http://localhost:8800/?signin=1`. The tab goes to the account site's + sign-in page ("Connect the editor to your account…"). Sign in with an email code. The tab comes back to + the editor with a clean address, a "Signed in to LevelCode." message, and the model and plan in the chat footer. + - Signed in on the account site first: open `http://localhost:5173/ai/account` and use **Open in browser**. + The editor opens, the tab flashes through the login page ("Opening LevelCode…") and is back signed in, + with nothing typed. + - Ask the agent: *Create a file named hello.txt containing: hello.* The file appears in the Explorer. + - **LevelCode: Open Folder from Your Computer…** (palette) in Chrome or Edge: pick a folder, ask the agent + to read a file of it and write another. + - A second tab of the editor opens already signed in (it shares the session). + +Look at the browser's network panel while you do it: every call to the API should be `200`, with no CORS +errors in the console. A CORS error means the editor's origin is not in `LEVELCODE_WEB_EDITOR_ORIGINS`. + +This was run once, by a hand-driven script, against Rails from the thin.ly branch in test mode (email and +Stripe stubbed, a throwaway database): email-code sign-in from the editor, sign-in from an already +signed-in browser, a gateway chat, the agent writing a file, and a folder opened through a real +`FileSystemDirectoryHandle` (the origin-private file system stands in for the OS picker). That script needs +the other repository's internals and is not part of this repo's checks. + +### What was not run + +Say these out loud rather than discover them in production: + +- **A real browser other than headless Chrome.** Safari and Firefox are untested; so are phones. +- **The look in Safari, Firefox and on a phone.** `e2e-look.mjs` and the screenshots are headless Chrome at 1440, + 1100, 900 and 420 px wide. The chat page is responsive down to 420 px; the workbench around it is not (see + [widths](#the-look-and-the-layout)). No screen-reader pass was done on the skin; the rail is a labelled + landmark at wide widths and the dialog it already was below them. +- **`workbench.experimental.modernUI` across a Code-OSS bump.** It is the workbench's own experimental flag; the + pinned checkout was the only one run. +- **The docked layout after a real browser reload.** `e2e-look.mjs` reloads through the debugger; the workbench + flushes the Explorer's visibility to its own storage on shutdown, and whether a closed tab always gets that + flush written was not measured. The layout is applied again at start (the chat docked), the side bar's state is + the workbench's. +- **The Markdown preview's content.** The pane's ground is now the theme's (it was white); in headless Chrome the + preview's body rendered no text on the old and the new build alike, and that was not investigated. +- **Known and left:** the Command Palette's "Chat: …" commands and the menus of features this edition lacks (above); + the empty Terminal panel behind the panel toggle (a terminal is the next stage, below); the chat page's + composer does not move with the soft keyboard on a phone; `levelcode.ai.chat.startLocation: none` is not + consulted by the layout. +- **The operating system's folder picker itself.** It needs a person. Everything after it — the command, the + workbench's `file` provider on a real `FileSystemDirectoryHandle`, the agent's list, read and write — is run + (`web/test/e2e.mjs` section 5), with the origin-private file system standing in for the picker. Permission + handling on a *persisted* handle after a reload (the browser asks again) is the browser's and was not run. +- **A production account site.** The real sign-in page and Rails were run locally (above), not the deployed + ones; the backend's rules are its own specs. +- **Another provider's CORS.** Anthropic's browser-access header is set; whether a given provider + answers a page's request is that provider's. +- **Production hosting:** DNS, a wildcard certificate, the CDN in front, the real headers on the real host. +- **Long sessions across the 8-hour access-token boundary in a tab** (the logic is the desktop's, + unchanged; the shared store was added for it and is unit-tested, not soaked). + +## Decisions that are the owner's + +1. **The editor's domain.** Not a subdomain of the account host (see Deploying). Which registrable + domain, and who holds it. +2. **The webview origin.** Keep Code-OSS's CDN (no setup; a dependency on Microsoft's hosting of the + published page for this build's commit) or self-host a wildcard origin. +3. **Isolating the extension host.** Worth it before any third-party extension is ever allowed; not + needed while the page loads only LevelCode's. +4. **Where the entry points point.** Today every "Open LevelCode" link carries `?signin=1`, so the + editor starts the sign-in itself. Whether visitors who are not signed in at the account site + should see the editor first instead. +5. **Support statement.** Chrome and Edge for folders; scratch workspace everywhere else; phones + unstated until run. diff --git a/extensions/levelcode-ai/agent.js b/extensions/levelcode-ai/agent.js index de92aac..4eec423 100644 --- a/extensions/levelcode-ai/agent.js +++ b/extensions/levelcode-ai/agent.js @@ -14,6 +14,7 @@ const fs = require('fs'); const path = require('path'); const cp = require('child_process'); const os = require('os'); // MCP servers need a cwd even when no folder is open +const host = require('./host'); // the machine we run on: workspace I/O is async here, and capabilities differ (browser) const providers = require('./providers/index'); const { formatVerifyFeedback, verifyOutcome, looksUnrunnable, sniffPort, sniffPreviewUrl, looksReady } = require('./verify'); const { classifyCommand, dangerLabel } = require('./commandSafety'); @@ -24,7 +25,7 @@ const { connectAll, getServer } = require('./mcpClient'); const diagramTool = require('./diagram/tool'); const diagramRepair = require('./diagram/repair'); -const SYSTEM_BASE = [ +const SYSTEM_LINES = [ "You are LevelCode's built-in autonomous coding agent. You accomplish the user's goal in their", 'workspace using the provided tools. Rules:', '- Be DECISIVE and FAST. Read only the file you are changing (plus at most 1 other if truly needed), then ACT.', @@ -44,7 +45,24 @@ const SYSTEM_BASE = [ '- You have SKILLS — short expert playbooks for common task types, listed under "Available skills" below with a name and a one-line description. When the user\'s goal clearly matches a skill\'s description (e.g. reviewing a diff, fixing one reported bug, writing tests), call use_skill with that exact name FIRST, before other tools, and follow the steps it returns. Pick at most one; if nothing clearly fits, just proceed normally. Do not mention skills to the user.', '- When the goal is finished, end with a line starting with "Done:" — one crisp sentence of what was accomplished. For a substantial run (several files, a tricky diagnosis, or a decision the user should know about), follow that line with a SHORT wrap-up in plain lines: what landed, how you verified it, anything left open or worth pushing on. For a trivial goal the "Done:" line alone is right. Then STOP (no more tools).', '- After you finish, your work is verified automatically: editor diagnostics for the files you changed (plus a verify command, if the user configured one) are checked. If problems are found you will get them back as a follow-up message — fix the ones you introduced, then finish again with "Done:". If a reported problem is clearly pre-existing and unrelated to the goal, do not chase it: note it in one line and finish.' -].join('\n'); +]; +const SYSTEM_BASE = SYSTEM_LINES.join('\n'); + +// The same prompt for a host that cannot run programs (LevelCode in the browser). Derived from +// SYSTEM_LINES rather than written out again so the two cannot drift: lines about commands go, the +// two mixed lines lose their command sentences, and one line says plainly what is missing. +const BROWSER_NOTE = '- You are running in LevelCode in the user\'s browser. There is no terminal: you cannot run commands, tests, installs or servers, and there is no run_command tool. Read, search, create, edit and delete files, and when something must be run, say so in one line and tell the user the exact command to run on their own computer.'; +function browserLines(lines) { + return lines + .filter((l) => !/run_command background:true/.test(l)) + .map((l) => l + .replace(' Only run_command still needs approval; if the user skips a command, adapt or stop.', '') + .replace('EVERY run_command, read_file and search MUST include', 'EVERY read_file and search MUST include') + .replace(' ("Run the extension unit tests", "Find the insertion point in section 10", "Read the runAgent call site")', ' ("Find the insertion point in section 10", "Read the runAgent call site")') + .replace(' run_command accepts an optional "folder" to pick which folder it runs in.', '')) + .concat([BROWSER_NOTE]); +} +const SYSTEM_BASE_HOST = host.caps.shell ? SYSTEM_BASE : browserLines(SYSTEM_LINES).join('\n'); const TOOLS = [ { name: 'list_files', description: 'List workspace files (optional glob like "**/*.js"). Excludes node_modules/.git/build dirs.', input_schema: { type: 'object', properties: { glob: { type: 'string' } } } }, @@ -60,16 +78,12 @@ const TOOLS = [ { name: 'use_skill', description: 'Load an expert playbook (SKILL.md) for a task type, chosen from the "Available skills" list in your system prompt. Returns the skill\'s step-by-step instructions as the tool result — then follow them. Read-only and instant (no approval). Call it once, early, when the goal matches a skill\'s description.', input_schema: { type: 'object', properties: { name: { type: 'string', description: 'the exact skill name from the Available skills list' } }, required: ['name'] } } ]; -// Rough token estimates (chars/4) for the static prompt segments, so the context popover can break -// down "what's filling the window" — system + tools are sent on every request, the rest is messages. -const SYSTEM_TOKENS_EST = Math.round(SYSTEM_BASE.length / 4); - /** Append an "Available skills" name+description menu to the base prompt. Bodies NEVER go here — * progressive disclosure means only name+description are ever in the prompt (~100 words for a dozen). */ function buildSystem(menu) { - if (!menu || !menu.length) { return SYSTEM_BASE; } + if (!menu || !menu.length) { return SYSTEM_BASE_HOST; } const list = menu.map((s) => '- ' + s.name + ': ' + s.description).join('\n'); - return SYSTEM_BASE + '\n\nAvailable skills (call use_skill with the name):\n' + list; + return SYSTEM_BASE_HOST + '\n\nAvailable skills (call use_skill with the name):\n' + list; } // The tools that resolve a PATH or a CWD against the workspace root. With no folder open they have // nothing to resolve against, so they are withheld from the model rather than offered and left to fail @@ -87,7 +101,12 @@ const NEEDS_ROOT = new Set([ ]); const PORTABLE_TOOLS = TOOLS.filter((t) => !NEEDS_ROOT.has(t.name)); -const TOOLS_TOKENS_EST = Math.round(JSON.stringify(TOOLS).length / 4); +// A host that cannot start programs (LevelCode in the browser) never offers the shell tools: a tool +// that is present but always errors is worse than one that is absent. The tool list is narrowed where +// a run assembles it (see `withheld` in runAgent); this is the estimate for the list that is sent. +// PORTABLE_TOOLS never contains them, since both are in NEEDS_ROOT. +const NEEDS_SHELL = new Set(['run_command', 'read_command_output']); +const TOOLS_TOKENS_EST = Math.round(JSON.stringify(host.caps.shell ? TOOLS : TOOLS.filter((t) => !NEEDS_SHELL.has(t.name))).length / 4); // The same estimate for the rootless list, and it has to exist separately rather than be derived at // call time: the plain path deliberately never re-stringifies (see toolsTokensEst below), so without a // second constant a rootless run reports the FULL schema cost for a list it never sent — about 1000 @@ -116,6 +135,7 @@ function rgPath() { return cands.find((c) => { try { return fs.existsSync(c); } catch { return false; } }) || null; } function rgSearch(term, cwd) { + if (!host.caps.ripgrep) { return scanSearch(term, cwd); } return new Promise((resolve) => { const bin = rgPath(); if (!bin || !cwd) { resolve(''); return; } @@ -131,6 +151,49 @@ function rgSearch(term, cwd) { }); } +/** + * The same search without ripgrep: list the workspace's files and scan them. Output matches rg's + * (`./path:line:text`, case-insensitive literal, 20 matches per file, files over 1 MB skipped, capped + * at 6000 characters) so the callers that prefix folder names and the model's habits need no change. + * Used where there is no binary to run (LevelCode in the browser). + */ +async function scanSearch(term, root) { + if (!term || !root) { return ''; } + const needle = term.toLowerCase(); + const out = []; + let size = 0; + let uris = []; + // Scoped to `root` BEFORE the cap: across the whole workspace, another folder's files could use up all 2,000. + try { uris = await host.findFiles(new vscode.RelativePattern(host.uriFor(root), '**/*'), FILE_EXCLUDES, 2000); } catch { return ''; } + const rootPrefix = root.endsWith('/') ? root : root + '/'; + for (const uri of uris) { + if (size >= 6000) { break; } + const abs = uri.path; + if (abs !== root && !abs.startsWith(rootPrefix)) { continue; } // another workspace folder's file + let text; + try { + const st = await host.stat(abs); + if (!st || !st.isFile || st.size > 1024 * 1024) { continue; } + const bytes = await host.readBytes(abs); + if (bytes.subarray(0, 8000).includes(0)) { continue; } // binary, as rg would skip it + text = new TextDecoder('utf-8').decode(bytes); + } catch { continue; } + const rel = './' + abs.slice(rootPrefix.length); + let hits = 0; + const lines = text.split('\n'); + for (let i = 0; i < lines.length && hits < 20; i++) { + if (lines[i].toLowerCase().includes(needle)) { + const line = rel + ':' + (i + 1) + ':' + lines[i].replace(/\r$/, ''); + out.push(line); + size += line.length + 1; + hits++; + if (size >= 6000) { break; } + } + } + } + return out.join('\n').slice(0, 6000); +} + // ---- readable line diff ---------------------------------------------------- /** LCS diff of two small line arrays → [{type:'ctx'|'add'|'del', text}]. */ function lcsDiff(a, b) { @@ -185,7 +248,8 @@ function workspaceRoot() { } function safeJoin(root, rel) { const pth = path.resolve(root, rel); - if (pth !== root && !pth.startsWith(root + path.sep)) { return null; } + const prefix = root.endsWith(path.sep) ? root : root + path.sep; // a root of "/" is its own prefix + if (pth !== root && !pth.startsWith(prefix)) { return null; } return pth; } /** Resolve a model-supplied workspace-relative path to an absolute path, multi-root aware. @@ -197,6 +261,22 @@ function safeJoin(root, rel) { * the primary folder, then every other folder. Returns null if nowhere. * mustExist=false (create): the folder-name prefix targets that folder; else the primary. */ function resolveWorkspacePath(rel, opts) { + const r = resolveCandidates(rel, opts); + if (!r) { return null; } + if (!r.mustExist) { return r.create; } + for (const c of r.candidates) { if (c && fs.existsSync(c)) { return c; } } + return null; +} +/** The same resolution for a host that can only ask whether a file exists asynchronously. */ +async function resolveWorkspacePathAsync(rel, opts) { + const r = resolveCandidates(rel, opts); + if (!r) { return null; } + if (!r.mustExist) { return r.create; } + for (const c of r.candidates) { if (c && await host.exists(c)) { return c; } } + return null; +} +/** Where `rel` could be: the path to create it at, and the paths that count as "exists", in order. */ +function resolveCandidates(rel, opts) { const mustExist = !!(opts && opts.mustExist); const folders = workspaceFolderList(); if (!folders.length) { return null; } @@ -205,11 +285,9 @@ function resolveWorkspacePath(rel, opts) { const named = folders.length > 1 ? folders.find((f) => f.name === seg) : null; const namedAbs = named ? safeJoin(named.root, rel.slice(seg.length).replace(/^[\\/]+/, '')) : null; const primaryAbs = safeJoin(folders[0].root, rel); - if (!mustExist) { return namedAbs || primaryAbs; } const candidates = [namedAbs, primaryAbs]; for (const f of folders.slice(1)) { candidates.push(safeJoin(f.root, rel)); } - for (const c of candidates) { if (c && fs.existsSync(c)) { return c; } } - return null; + return { mustExist, create: namedAbs || primaryAbs, candidates }; } /** "file not found" help for the model — names the folders it can address in a multi-root workspace. */ function whereHint() { @@ -259,9 +337,7 @@ function inputPreview(input, redact) { } /** Cheap binary sniff — a NUL byte in the first 8KB means it isn't text we should round-trip as UTF-8. */ -function isBinaryFile(abs) { - try { const buf = fs.readFileSync(abs); const n = Math.min(buf.length, 8000); for (let i = 0; i < n; i++) { if (buf[i] === 0) { return true; } } return false; } catch { return false; } -} +function isBinaryFile(abs) { return host.isBinary(abs); } // Run a shell command, STREAMING its output: onChunk(text, 'stdout'|'stderr') fires as it runs (live // terminal in the chat), onExit(code, ms, how) at the end. onStart(child, stop) exposes a stop() that @@ -333,10 +409,10 @@ async function runTool(tu, ctx) { // label/path ride alongside the legacy `text` so the chat can title this action with the // model's own words ("Read the runAgent call site") and still attribute it to a file. ctx.post({ type: 'agentTool', icon: 'file', text: 'read ' + input.path, label: input.explanation || '', path: input.path, kind: 'read' }); - const abs = resolveWorkspacePath(input.path || '', { mustExist: true }); + const abs = await resolveWorkspacePathAsync(input.path || '', { mustExist: true }); if (!abs) { return 'ERROR: file not found: ' + input.path + whereHint(); } - if (isBinaryFile(abs)) { return 'ERROR: ' + input.path + ' looks like a binary file — not reading it as text.'; } - let body = fs.readFileSync(abs, 'utf8').replace(/^/, ''); // drop BOM so old_str matches cleanly + if (await isBinaryFile(abs)) { return 'ERROR: ' + input.path + ' looks like a binary file — not reading it as text.'; } + let body = (await host.readText(abs)).replace(/^/, ''); // drop BOM so old_str matches cleanly if (body.length > 100 * 1024) { body = body.slice(0, 100 * 1024) + '\n…(truncated)…'; } return body; } @@ -362,10 +438,10 @@ async function runTool(tu, ctx) { return 'Plan updated (' + done + '/' + todos.length + ' done).'; } if (tu.name === 'edit_file') { - const abs = resolveWorkspacePath(input.path || '', { mustExist: true }); + const abs = await resolveWorkspacePathAsync(input.path || '', { mustExist: true }); if (!abs) { return 'ERROR: file not found: ' + input.path + whereHint() + ' (use write_file to create it)'; } - if (isBinaryFile(abs)) { return 'ERROR: ' + input.path + ' looks like a binary file — refusing to edit it as text.'; } - const cur = fs.readFileSync(abs, 'utf8'); + if (await isBinaryFile(abs)) { return 'ERROR: ' + input.path + ' looks like a binary file — refusing to edit it as text.'; } + const cur = await host.readText(abs); const oldStr = String(input.old_str || ''); if (!oldStr) { return 'ERROR: old_str is empty.'; } const res = applyStringEdit(cur, oldStr, String(input.new_str || '')); @@ -379,12 +455,12 @@ async function runTool(tu, ctx) { if (tu.name === 'write_file') { const abs = resolveWorkspacePath(input.path || ''); if (!abs) { return 'ERROR: path is outside the workspace' + whereHint(); } - const existed = fs.existsSync(abs); + const existed = await host.exists(abs); let newStr = String(input.content || ''); if (existed) { - if (isBinaryFile(abs)) { return 'ERROR: ' + input.path + ' looks like a binary file — refusing to overwrite it.'; } - try { if (fs.statSync(abs).size > 100 * 1024) { return 'ERROR: ' + input.path + ' is large (>100KB) and you only saw the first 100KB on read. Use edit_file for targeted changes — a full write_file risks dropping content you did not see.'; } } catch { /* */ } - const raw = fs.readFileSync(abs, 'utf8'); const eol = raw.includes('\r\n') ? '\r\n' : '\n'; newStr = newStr.replace(/\r\n/g, '\n').replace(/\n/g, eol); + if (await isBinaryFile(abs)) { return 'ERROR: ' + input.path + ' looks like a binary file — refusing to overwrite it.'; } + try { const st = await host.stat(abs); if (st && st.size > 100 * 1024) { return 'ERROR: ' + input.path + ' is large (>100KB) and you only saw the first 100KB on read. Use edit_file for targeted changes — a full write_file risks dropping content you did not see.'; } } catch { /* */ } + const raw = await host.readText(abs); const eol = raw.includes('\r\n') ? '\r\n' : '\n'; newStr = newStr.replace(/\r\n/g, '\n').replace(/\n/g, eol); } const ok = await ctx.applyEdit({ path: input.path, exists: existed, proposed: newStr }); if (!ok) { return 'ERROR: could not write ' + input.path + ' (path may be read-only or in conflict).'; } @@ -393,9 +469,9 @@ async function runTool(tu, ctx) { return 'Applied edit to ' + input.path + ' (pending the user\'s Keep/Undo review).'; } if (tu.name === 'delete_file') { - const abs = resolveWorkspacePath(input.path || '', { mustExist: true }); + const abs = await resolveWorkspacePathAsync(input.path || '', { mustExist: true }); if (!abs) { return 'ERROR: file not found: ' + input.path + whereHint(); } - try { if (fs.statSync(abs).isDirectory()) { return 'ERROR: ' + input.path + ' is a directory — delete_file removes a single file.'; } } catch { /* */ } + try { const st = await host.stat(abs); if (st && st.isDirectory) { return 'ERROR: ' + input.path + ' is a directory — delete_file removes a single file.'; } } catch { /* */ } // Deletion is in the autopilot danger set — ask even when autopilot runs everything else silently. // (In manual mode nothing gates here: the edit is applied and reviewed with Keep/Undo as before.) if (ctx.autopilot && typeof ctx.approve === 'function') { @@ -678,12 +754,15 @@ function resetContextAnnounce() { lastContextSig = ''; } async function setupMcp(ctx, wsFolders, dbg) { const empty = { tools: [], routes: null }; const cfg = ctx.mcp || {}; + // MCP servers here are local processes spoken to over stdio. A host that cannot start programs has + // none to offer; saying nothing is right, since nothing is configured that it could have run. + if (!host.caps.mcpStdio) { return empty; } try { - const { servers, problems } = loadServerConfig({ + const { servers, problems } = await host.withReads((readFile) => loadServerConfig({ settings: cfg.servers, folders: wsFolders, - readFile: (abs) => { try { return fs.readFileSync(abs, 'utf8'); } catch { return null; } } - }); + readFile + })); for (const p of problems) { dbg('mcp.config', p); } if (!servers.length) { return empty; } @@ -780,7 +859,7 @@ async function runAgent(ctx) { // Multi-root: name every workspace folder so the model addresses them by prefix from turn one. const wsFolders = workspaceFolderList(); const multiRootNote = wsFolders.length > 1 - ? '\n\nWorkspace folders (multi-root — prefix paths with the folder name): ' + wsFolders.map((f) => f.name).join(', ') + '. The first folder ("' + wsFolders[0].name + '") is the default for unprefixed paths and run_command.' + ? '\n\nWorkspace folders (multi-root — prefix paths with the folder name): ' + wsFolders.map((f) => f.name).join(', ') + '. The first folder ("' + wsFolders[0].name + '") is the default for unprefixed paths' + (host.caps.shell ? ' and run_command.' : '.') : ''; // Rootless: say so plainly. Without this the model sees a tool list with no read_file and improvises — // answering about files it cannot see, or apologising for a limit it cannot name. Telling it WHY the @@ -796,13 +875,13 @@ async function runAgent(ctx) { // Autopilot: act decisively and self-verify rather than pausing. Commands run without approval (the // host still gates the danger set — deletion, sudo, force-push, remote|shell, publish, system writes), // so the model should lean on verification, not on asking, when it's unsure. - const autopilotNote = ctx.autopilot + const autopilotNote = ctx.autopilot && host.caps.shell ? '\n\nAUTOPILOT IS ON. Work end-to-end without pausing for confirmation. Your run_command calls execute immediately (only irreversible ones — deleting files, sudo, force-push, piping a remote script to a shell, publishing — still ask the user). Do NOT call ask_user for anything you can reasonably decide; pick a sensible default and proceed. When you are unsure whether a change is correct, do not stop to ask — verify it: run the build/tests/linters via run_command and read editor diagnostics, then fix and re-verify until clean, and only then move on. Prefer doing and checking over asking.' : ''; // Project rules: fold a repo's own AGENTS.md (or CLAUDE.md / .cursorrules) into the cached system // block so the agent follows the project's conventions from turn one. Read once per run — an edit is // picked up on the next run. - const rules = loadProjectRules(wsFolders, (abs) => { try { return fs.readFileSync(abs, 'utf8'); } catch { return null; } }); + const rules = await host.withReads((readFile) => loadProjectRules(wsFolders, readFile)); // Project memory: a tight, verify-first digest of what earlier sessions achieved (built by extension.js // from the per-project journal). Rides the SAME cached-system channel as project rules — always-on but // small — so a new session's first reply is continuous, not amnesiac. It is untrusted context like the @@ -859,6 +938,8 @@ async function runAgent(ctx) { if (ctx.recallSessions) { extras.push(RECALL_TOOL); } if (rich) { extras.push(diagramTool.RENDER_DIAGRAM); if (ctx.diagramsStubbed) { extras.push(diagramTool.GET_DIAGRAM); } } if (extras.length) { tools = tools.concat(extras); } + // Withhold what this host cannot do: no shell, no run_command, no read_command_output. + if (!host.caps.shell) { tools = tools.filter((t) => !NEEDS_SHELL.has(t.name)); } const baseTools = extras.length ? builtins.concat(extras) : builtins; // built-ins + extras; MCP is the rest // Recomputed only when MCP or a host-gated extra actually contributed tools, so the plain path keeps the module // constant and pays nothing for a feature it isn't using — but there are now TWO plain paths, and the @@ -904,7 +985,7 @@ async function runAgent(ctx) { let feedbackUsed = 0, verifySeq = 0; async function runVerifyOnce() { const id = 'verify-' + (++verifySeq); - const cmd = ((ctx.verify && ctx.verify.command) || '').trim(); + const cmd = host.caps.shell ? ((ctx.verify && ctx.verify.command) || '').trim() : ''; ctx.post({ type: 'verifyRun', id, command: cmd }); let ran = false, exitCode = 0, cmdTail = ''; if (cmd) { @@ -929,7 +1010,7 @@ async function runAgent(ctx) { if (!v || !v.enabled) { return false; } if (ctx.signal.aborted) { return false; } if (!(ctx.editCount > 0)) { return false; } // no edits this run → nothing to verify - const cmd = (v.command || '').trim(); + const cmd = host.caps.shell ? (v.command || '').trim() : ''; const touchedCount = ctx.getTouchedUris ? ctx.getTouchedUris().length : 0; if (!cmd && !touchedCount) { return false; } // nothing to check const max = Math.max(0, v.maxRounds || 0); @@ -1174,4 +1255,4 @@ async function runAgent(ctx) { } } -module.exports = { resetContextAnnounce, runAgent, makeDiff, resolveWorkspacePath, saysDone }; +module.exports = { resetContextAnnounce, runAgent, makeDiff, resolveWorkspacePath, resolveWorkspacePathAsync, saysDone }; diff --git a/extensions/levelcode-ai/extension.js b/extensions/levelcode-ai/extension.js index b4e7e13..9eff512 100644 --- a/extensions/levelcode-ai/extension.js +++ b/extensions/levelcode-ai/extension.js @@ -14,6 +14,7 @@ const fs = require('fs'); const os = require('os'); const cp = require('child_process'); const crypto = require('crypto'); +const host = require('./host'); // workspace I/O + what this machine can do (browser vs desktop) const providers = require('./providers/index'); const catalog = require('./providers/catalog'); const { resolveGateway } = require('./providers/gateway'); @@ -919,6 +920,37 @@ function rgFiles(term, cwd) { }); } +/** + * Files whose content contains each term, for every term at once. ripgrep where the editor has one; + * a bounded scan of the workspace's text files where it does not (LevelCode in the browser). Returns + * one array of absolute paths per term, in order. + */ +async function contentHits(terms, dir) { + if (host.caps.ripgrep) { return Promise.all(terms.map((kw) => rgFiles(kw, dir))); } + const hits = terms.map(() => []); + if (!dir) { return hits; } + const prefix = dir.endsWith('/') ? dir : dir + '/'; + let uris = []; + // Scoped to `dir` BEFORE the cap: across the whole workspace, another folder's files could use up all 400. + try { uris = await host.findFiles(new vscode.RelativePattern(host.uriFor(dir), '**/*'), '{**/node_modules/**,**/.git/**,**/out/**,**/dist/**,**/*.map,**/*.min.*}', 400); } catch { return hits; } + const needles = terms.map((t) => t.toLowerCase()); + const deadline = Date.now() + 4000; // the same budget rgFiles gives ripgrep + for (const uri of uris) { + if (Date.now() > deadline) { break; } + const abs = uri.path; + if (!abs.startsWith(prefix)) { continue; } + try { + const st = await host.stat(abs); + if (!st || !st.isFile || st.size > 1024 * 1024) { continue; } + const bytes = await host.readBytes(abs); + if (bytes.subarray(0, 8000).includes(0)) { continue; } + const text = new TextDecoder('utf-8').decode(bytes).toLowerCase(); + needles.forEach((n, i) => { if (text.includes(n)) { hits[i].push(abs); } }); + } catch { /* unreadable: not a hit */ } + } + return hits; +} + /** Scope retrieval to the active file's top-level sub-project, so unrelated sibling trees * (e.g. a vendored source dump) don't pollute results. Returns the search dir + rel prefix. */ function activeProjectScope() { @@ -926,14 +958,17 @@ function activeProjectScope() { const fallback = { dir: folders.length ? folders[0].uri.fsPath : '', prefix: '' }; if (!aiConfig().get('chat.scopeToActiveProject', true)) { return fallback; } const ed = vscode.window.activeTextEditor; - if (!ed || ed.document.uri.scheme !== 'file') { return fallback; } + if (!ed || (!host.isBrowser && ed.document.uri.scheme !== 'file')) { return fallback; } const wsFolder = vscode.workspace.getWorkspaceFolder(ed.document.uri); if (!wsFolder) { return fallback; } const rel = path.relative(wsFolder.uri.fsPath, ed.document.uri.fsPath); if (!rel || rel.startsWith('..')) { return fallback; } const top = rel.split(path.sep)[0]; const topPath = path.join(wsFolder.uri.fsPath, top); - try { if (!fs.statSync(topPath).isDirectory()) { return fallback; } } catch { return fallback; } + // A first segment is a directory when the file sits below it. The desktop asks the disk, as it + // always did; a browser workspace cannot be asked synchronously and the path says the same thing. + if (host.isBrowser) { if (rel.indexOf(path.sep) < 0) { return fallback; } } + else { try { if (!fs.statSync(topPath).isDirectory()) { return fallback; } } catch { return fallback; } } return { dir: topPath, prefix: top + '/' }; } @@ -965,9 +1000,9 @@ async function gatherAutoContext(question, allFiles) { // 1. CONTENT search via ripgrep (primary signal — finds files by what's inside them). const terms = keywords.slice(0, 6); - const hits = await Promise.all(terms.map((kw) => rgFiles(kw, scope.dir))); + const hits = await contentHits(terms, scope.dir); for (let i = 0; i < terms.length; i++) { - for (const abs of hits[i]) { bump(vscode.Uri.file(abs), contentWeight(terms[i])); } + for (const abs of hits[i]) { bump(host.uriFor(abs), contentWeight(terms[i])); } } // 2. filename matches (scoped). @@ -1060,7 +1095,10 @@ function recordDiagramStat(ev) { const diagramFolders = () => (vscode.workspace.workspaceFolders || []).map((f) => ({ name: f.name, root: f.uri.fsPath })); /** This conversation's diagrams. One for the life of the extension; reset with the conversation. */ const diagrams = createDiagrams({ - resolveLink: (link) => diagramLinks.resolveLink(link, diagramFolders()), + // A diagram's links are checked as it is drawn, and that cannot wait. Where files can only be looked at + // asynchronously (LevelCode in a browser) they are checked against the files the extension has indexed + // (buildFileIndex); the desktop reads the disk, as before. The click checks the file itself. + resolveLink: (link) => diagramLinks.resolveLink(link, diagramFolders(), { fs: host.knownFilesView([...workspaceFiles.values()].map((u) => u.path)) || undefined }), onStat: recordDiagramStat }); /** True once a diagram's spec has left the model's context (a compaction, or a resume that did not fit) — @@ -1083,11 +1121,11 @@ async function openDiagramLink(id, nodeId) { const node = record && record.spec ? record.spec.nodes.find((n) => n.id === String(nodeId)) : null; if (!node || !node.link) { dbg('diagram.link.unknown', { id }); return; } // Checked again at the moment of the click: the file may have moved, or become a link, since it was drawn. - const r = diagramLinks.resolveLink(node.link, diagramFolders()); + const r = await host.withStats((fs) => diagramLinks.resolveLink(node.link, diagramFolders(), { fs })); if (!r.ok) { vscode.window.showWarningMessage('LevelCode: that link cannot be opened — ' + r.reason + '.'); dbg('diagram.link.refused', { reason: r.reason }); return; } recordDiagramStat({ type: 'link' }); try { - const uri = vscode.Uri.file(r.abs); + const uri = host.uriFor(r.abs); const doc = await vscode.workspace.openTextDocument(uri); let line = Number.isInteger(node.link.line) ? node.link.line : null; if (node.link.symbol) { @@ -1881,6 +1919,7 @@ async function saveMcpLaunchTrust(store) { * broken configuration it exists to explain. */ function mcpOverview() { + if (!host.caps.mcpStdio) { return { configured: 0, running: 0, awaitingTrust: 0, servers: [], problems: [] }; } try { const cfg = aiConfig(); const folders = (vscode.workspace.workspaceFolders || []).map((f) => ({ name: f.name, root: f.uri.fsPath })); @@ -2282,8 +2321,8 @@ async function agentFlow(text, imageBlocks) { applyDelete: (req) => review.applyDelete(req), // file deletions: apply-then-review verify: verifyCfg, // M5 auto-verify settings touched: runTouched, // agent.js adds each edited abs path here - getTouchedUris: () => [...runTouched].map((p) => vscode.Uri.file(p)), - getNewDiagnostics: () => collectNewDiagnostics([...runTouched].map((p) => vscode.Uri.file(p)), diagBaseline, verifyCfg.includeWarnings), + getTouchedUris: () => [...runTouched].map((p) => host.uriFor(p)), + getNewDiagnostics: () => collectNewDiagnostics([...runTouched].map((p) => host.uriFor(p)), diagBaseline, verifyCfg.includeWarnings), signal: abort.signal }); } finally { @@ -2341,7 +2380,7 @@ async function attachImagePaths(paths) { const mt = { '.png': 'image/png', '.jpg': 'image/jpeg', '.jpeg': 'image/jpeg', '.gif': 'image/gif', '.webp': 'image/webp' }[ext]; if (!mt) { vscode.window.showWarningMessage(path.basename(fsPath) + ' is not an image LevelCode can read.'); continue; } - const buf = await fs.promises.readFile(fsPath); + const buf = Buffer.from(await host.readBytes(fsPath)); // Guard before the bytes cross into the webview: a 200MB file would otherwise be // base64-ed onto the message bus before anything got a chance to refuse it. if (buf.length > 25 * 1024 * 1024) { @@ -3532,10 +3571,16 @@ async function handleAuthCallback(uri) { try { const q = new URLSearchParams(uri.query || ''); const code = q.get('code'); + const verifier = ctx ? await ctx.secrets.get(ACCOUNT_VERIFIER_KEY) : null; + // Where the callback is a page anyone can link to (LevelCode in a browser), a link nobody asked for must not + // sign the user in to someone else's account: only a code that answers a sign-in this editor started is taken. + if (!host.caps.unaskedSignIn && (!code || !verifier)) { + dbg('account.callback', { ignored: code ? 'no sign-in pending' : 'not a sign-in code' }); + return; + } // Preferred: PKCE code → exchange for tokens using the verifier we stashed at sign-in. if (code) { const endpoint = cloudApiUrl(); // token exchange is a BACKEND (Rails) call, not the website - const verifier = ctx ? await ctx.secrets.get(ACCOUNT_VERIFIER_KEY) : null; try { const res = await fetch(endpoint + '/api/levelcode/v1/auth/exchange', { method: 'POST', diff --git a/extensions/levelcode-ai/host.js b/extensions/levelcode-ai/host.js new file mode 100644 index 0000000..766634e --- /dev/null +++ b/extensions/levelcode-ai/host.js @@ -0,0 +1,272 @@ +/*--------------------------------------------------------------------------------------------- + * LevelCode — AI: the machine this extension runs on. + * + * The extension was written for a Node extension host: the user's project is a directory it can + * read with `fs`, and programs can be started. LevelCode also runs in a browser tab (docs/WEB.md), + * where the project is whatever the editor's file system provider says it is, every read is + * asynchronous, and there is no shell. This module is the one place that difference lives. + * + * Two rules keep the desktop app exactly as it was: + * 1. On the desktop every function here does what the call site did before — the same `fs` + * call, the same error — only behind `await`. + * 2. Nothing outside this file asks "am I in a browser?". Call sites ask for a capability + * (`caps.shell`) or call a function that already knows. + * + * Paths are the absolute strings the extension has always used for workspace files + * (`Uri.fsPath` of a workspace folder, joined). In the browser they are URI paths, and `uriFor` + * maps one back to the workspace folder it belongs to, so the scheme (`file`, `levelcode-scratch`) + * is never guessed. + *--------------------------------------------------------------------------------------------*/ +// @ts-check +'use strict'; + +const vscode = require('vscode'); +const nodeFs = require('fs'); +const path = require('path'); + +/** True in the browser build (web/ai-extension/entry.js sets the flag before anything else loads). */ +const isBrowser = !!globalThis.__LEVELCODE_BROWSER_HOST__; + +/** What this host can do. Tools and prompt text that need a capability are withheld without it. */ +const caps = Object.freeze({ + /** Start programs: run_command, background commands, verify commands. */ + shell: !isBrowser, + /** Start MCP servers as local processes (stdio). */ + mcpStdio: !isBrowser, + /** Search file contents with the ripgrep binary the editor ships. */ + ripgrep: !isBrowser, + /** + * Take a sign-in result that no sign-in of this editor asked for: the old token-in-the-link form, or a code + * with no sign-in pending. The desktop does what it always did. In a browser the callback is a page of the + * editor's own site that anyone can link to, so only a code answering a sign-in this browser started counts. + */ + unaskedSignIn: !isBrowser, +}); + +const TEXT_SNIFF_BYTES = 8000; + +/* ----- paths and URIs ------------------------------------------------------------------------ */ + +function folders() { return vscode.workspace.workspaceFolders || []; } + +/** + * The URI for a workspace path. On the desktop that is `Uri.file`, as ever. In the browser the + * workspace folder that contains the path supplies scheme and authority; a path inside none of + * them falls back to `Uri.file`, which is right for a folder opened from the user's computer. + * @param {string} abs + * @returns {vscode.Uri} + */ +function uriFor(abs) { + if (!isBrowser) { return vscode.Uri.file(abs); } + let best = null; + for (const f of folders()) { + const root = f.uri.path; + const prefix = root.endsWith('/') ? root : root + '/'; + if (abs === root || abs.startsWith(prefix)) { + if (!best || root.length > best.uri.path.length) { best = f; } + } + } + return best ? best.uri.with({ path: abs }) : vscode.Uri.file(abs); +} + +/** + * Is this document one of the user's own files — the kind inline completion is offered in? On the desktop a + * `file` or an `untitled` document, as ever. In the browser also a document inside a workspace folder, whatever + * its scheme: the scratch workspace's files are `levelcode-scratch:` ones. + * @param {vscode.Uri} uri + */ +function isUserDocument(uri) { + if (uri.scheme === 'file' || uri.scheme === 'untitled') { return true; } + if (!isBrowser) { return false; } + return folders().some((f) => { + const root = f.uri.path.endsWith('/') ? f.uri.path : f.uri.path + '/'; + return f.uri.scheme === uri.scheme && (f.uri.authority || '') === (uri.authority || '') && (uri.path === f.uri.path || uri.path.startsWith(root)); + }); +} + +/* ----- reading and probing ------------------------------------------------------------------- */ + +/** @param {string} abs @returns {Promise} rejects when the file is missing or unreadable. */ +async function readBytes(abs) { + if (!isBrowser) { return nodeFs.promises.readFile(abs); } + return vscode.workspace.fs.readFile(uriFor(abs)); +} + +/** @param {string} abs @returns {Promise} UTF-8 text; a leading BOM is left for the caller to strip. */ +async function readText(abs) { + if (!isBrowser) { return nodeFs.promises.readFile(abs, 'utf8'); } + return new TextDecoder('utf-8').decode(await vscode.workspace.fs.readFile(uriFor(abs))); +} + +/** @param {string} abs @returns {Promise} null when it cannot be read — never throws. */ +async function readTextOrNull(abs) { + try { return await readText(abs); } catch { return null; } +} + +/** + * @param {string} abs + * @returns {Promise<{isFile: boolean, isDirectory: boolean, size: number}|null>} null when absent. + */ +async function stat(abs) { + try { + if (!isBrowser) { + const s = await nodeFs.promises.stat(abs); + return { isFile: s.isFile(), isDirectory: s.isDirectory(), size: s.size }; + } + const s = await vscode.workspace.fs.stat(uriFor(abs)); + return { isFile: (s.type & vscode.FileType.File) !== 0, isDirectory: (s.type & vscode.FileType.Directory) !== 0, size: s.size }; + } catch { return null; } +} + +/** @param {string} abs @returns {Promise} */ +async function exists(abs) { return (await stat(abs)) !== null; } + +/** A NUL byte in the first 8 KB means it is not text we should round-trip as UTF-8. Unreadable counts as text. */ +async function isBinary(abs) { + try { + const buf = await readBytes(abs); + const n = Math.min(buf.length, TEXT_SNIFF_BYTES); + for (let i = 0; i < n; i++) { if (buf[i] === 0) { return true; } } + return false; + } catch { return false; } +} + +/** + * Run a loader that reads files synchronously (project rules, MCP config) against files this host can + * only read asynchronously. The loader runs once to learn which paths it asks for, those are read, + * and it runs again with the answers. A path the loader only asks for after seeing another file's + * content is picked up by the next pass (at most four). A path never read in time is absent. + * + * On the desktop the loader simply runs once against `fs`, as before. + * @template T + * @param {(readFile: (abs: string) => string|null) => T} run + * @returns {Promise} + */ +async function withReads(run) { + if (!isBrowser) { + return run((abs) => { try { return nodeFs.readFileSync(abs, 'utf8'); } catch { return null; } }); + } + /** @type {Map} */ + const known = new Map(); + let result; + for (let pass = 0; pass < 4; pass++) { + const asked = new Set(); + result = run((abs) => { + if (known.has(abs)) { return known.get(abs); } + asked.add(abs); + return null; + }); + if (asked.size === 0) { break; } + await Promise.all([...asked].map(async (p) => { known.set(p, await readTextOrNull(p)); })); + } + return result; +} + +/* ----- checking paths synchronously --------------------------------------------------------- */ + +/** + * The part of Node's `fs` a synchronous path check uses — `statSync` and `realpathSync` — answered from `lookup`. + * In the browser nothing is a symbolic link, so a path's real path is the path. + * @param {(abs: string) => ({isFile: boolean, isDirectory: boolean}|null)} lookup + */ +function statView(lookup) { + const missing = (p) => Object.assign(new Error('ENOENT: no such file or directory, ' + p), { code: 'ENOENT' }); + return { + statSync(p) { + const s = lookup(String(p)); + if (!s) { throw missing(p); } + return { isFile: () => s.isFile, isDirectory: () => s.isDirectory }; + }, + realpathSync(p) { if (!lookup(String(p))) { throw missing(p); } return String(p); }, + }; +} + +/** + * Run a check written against `fs.statSync` / `fs.realpathSync` (a diagram's code links: diagram/links.js) where + * files can only be looked at asynchronously. Like withReads: the check runs once to learn which paths it asks + * about, those are stat'd, and it runs again with the answers (at most four passes). + * On the desktop it simply runs once against `fs`, as before. + * @template T + * @param {(fs: { statSync: Function, realpathSync: Function }) => T} run + * @returns {Promise} + */ +async function withStats(run) { + if (!isBrowser) { return run(nodeFs); } + /** @type {Map} */ + const known = new Map(); + let result; + for (let pass = 0; pass < 4; pass++) { + const asked = new Set(); + result = run(statView((abs) => { + if (known.has(abs)) { return known.get(abs); } + asked.add(abs); + return null; + })); + if (asked.size === 0) { break; } + await Promise.all([...asked].map(async (p) => { known.set(p, await stat(p)); })); + } + return result; +} + +/** + * The same view over files already known to exist, for a check that cannot wait for the file system (a diagram's + * links are checked as it is drawn). Folders are the files' parents. Null on the desktop: the check reads the disk. + * @param {Iterable} files absolute paths + */ +function knownFilesView(files) { + if (!isBrowser) { return null; } + const set = new Set(files); + const dirs = new Set(); + for (const f of set) { for (let d = f.slice(0, f.lastIndexOf('/')); d; d = d.slice(0, d.lastIndexOf('/'))) { dirs.add(d); } } + dirs.add('/'); + return statView((abs) => (set.has(abs) ? { isFile: true, isDirectory: false } : dirs.has(abs) ? { isFile: false, isDirectory: true } : null)); +} + +/* ----- sending the user to sign in ------------------------------------------------------------- */ + +/** Registered by the page (web/main.js); it takes the tab to the account site and the sign-in comes back to it. */ +const OPEN_AUTH_COMMAND = 'levelcode.web.openAuthUrl'; + +/** + * Open the account site's sign-in page. The desktop hands it to the system browser, as ever. A browser + * tab leaves the page for it: a pop-up opened from a chain that began in a webview is the first thing a + * strict browser (Safari, a phone) refuses, and the sign-in returns to the editor in the same tab. The + * page's entry script supplies the command that navigates; without it this falls back to openExternal. + * + * The string handed over is the one the editor's own opener would have opened — `encodeURI(uri.toString(true))` + * (src/vs/editor/browser/services/openerService.ts) — so the server sees the same address whichever way + * the sign-in was started, as it has from the desktop app all along. + * @param {vscode.Uri} uri + */ +async function openAuth(uri) { + if (isBrowser) { + try { + const known = await vscode.commands.getCommands(true); + if (known.includes(OPEN_AUTH_COMMAND)) { + await vscode.commands.executeCommand(OPEN_AUTH_COMMAND, encodeURI(uri.toString(true))); + return; + } + } catch { /* fall through to the pop-up */ } + } + await vscode.env.openExternal(uri); +} + +/* ----- finding files ------------------------------------------------------------------------- */ + +/** + * `vscode.workspace.findFiles`, which is what the editor's own search service answers. The browser + * build uses it too: LevelCode's scratch workspace registers a file search provider. + * @param {vscode.GlobPattern} glob a string, or a vscode.RelativePattern to search under one folder + * @param {string} exclude + * @param {number} max + */ +function findFiles(glob, exclude, max) { + return vscode.workspace.findFiles(glob, exclude, max); +} + +module.exports = { + isBrowser, caps, + uriFor, isUserDocument, readBytes, readText, readTextOrNull, stat, exists, isBinary, withReads, withStats, knownFilesView, findFiles, openAuth, + // For tests. + _path: path, +}; diff --git a/extensions/levelcode-ai/inlineComplete.js b/extensions/levelcode-ai/inlineComplete.js index e758eaa..04636e9 100644 --- a/extensions/levelcode-ai/inlineComplete.js +++ b/extensions/levelcode-ai/inlineComplete.js @@ -10,6 +10,7 @@ const vscode = require('vscode'); const { sendWithAuthRetry } = require('./providers/authRetry'); +const host = require('./host'); const PREFIX_MAX = 2500; // chars of context before the cursor (smaller = faster first token) const SUFFIX_MAX = 1000; // chars of context after the cursor @@ -98,7 +99,7 @@ function registerInlineComplete(context, deps) { async provideInlineCompletionItems(document, position, ctx, token) { const cfg = aiConfig(); if (!cfg.get('completions.enabled', true)) { return null; } - if (document.uri.scheme !== 'file' && document.uri.scheme !== 'untitled') { return null; } + if (!host.isUserDocument(document.uri)) { return null; } // file and untitled; in a browser, the workspace's own schemes too // Debounce: if the user keeps typing, this token is cancelled and we bail. const debounce = Math.max(0, cfg.get('completions.debounce', 150)); diff --git a/extensions/levelcode-ai/providers/anthropic.js b/extensions/levelcode-ai/providers/anthropic.js index 31c2103..8ce94ce 100644 --- a/extensions/levelcode-ai/providers/anthropic.js +++ b/extensions/levelcode-ai/providers/anthropic.js @@ -13,7 +13,11 @@ const { readLines } = require('./sse'); const ANTHROPIC_URL = 'https://api.anthropic.com/v1/messages'; function anthropicHeaders(apiKey) { - return { 'content-type': 'application/json', 'x-api-key': apiKey, 'anthropic-version': '2023-06-01' }; + const h = { 'content-type': 'application/json', 'x-api-key': apiKey, 'anthropic-version': '2023-06-01' }; + // From a browser the call is cross-origin, and Anthropic answers CORS only to a request that says it + // knows the key is being used from a page (web/ai-extension/entry.js sets the flag; the desktop never does). + if (globalThis.__LEVELCODE_BROWSER_HOST__) { h['anthropic-dangerous-direct-browser-access'] = 'true'; } + return h; } /** diff --git a/extensions/levelcode-ai/reviewSession.js b/extensions/levelcode-ai/reviewSession.js index c23ecb3..968f4f4 100644 --- a/extensions/levelcode-ai/reviewSession.js +++ b/extensions/levelcode-ai/reviewSession.js @@ -16,7 +16,8 @@ const vscode = require('vscode'); const fs = require('fs'); const path = require('path'); -const { makeDiff, resolveWorkspacePath } = require('./agent'); +const { makeDiff, resolveWorkspacePathAsync } = require('./agent'); +const host = require('./host'); const PERSIST_KEY = 'levelcode.ai.pendingReviews'; @@ -108,18 +109,18 @@ function registerReview(context, post, dbg, recordTouch) { if (!workspaceRoot()) { return false; } // Multi-root aware — MUST resolve exactly like the agent's tool layer (same resolver), or an // edit the agent computed against folder N would be applied into folder 0. - const abs = resolveWorkspacePath(req.path || '', { mustExist: !!req.exists }); + const abs = await resolveWorkspacePathAsync(req.path || '', { mustExist: !!req.exists }); if (!abs) { return false; } - const uri = vscode.Uri.file(abs); + const uri = host.uriFor(abs); const key = uri.toString(); const proposed = String(req.proposed != null ? req.proposed : ''); - const existsNow = fs.existsSync(abs); // re-check at apply time (TOCTOU vs the agent's earlier check) + const existsNow = await host.exists(abs); // re-check at apply time (TOCTOU vs the agent's earlier check) // Snapshot the pre-run content ONCE, from DISK (the true baseline — never a dirty buffer). let snapshot; if (pending.has(key)) { snapshot = pending.get(key).snapshot; } else if (existsNow) { - try { snapshot = fs.readFileSync(abs, 'utf8').replace(/^/, ''); } + try { snapshot = (await host.readText(abs)).replace(/^/, ''); } catch { return false; } // can't read the file we're about to edit → abort (never a destructive empty-snapshot edit) } else { snapshot = ''; } @@ -179,13 +180,13 @@ function registerReview(context, post, dbg, recordTouch) { async function applyDelete(req) { if (!workspaceRoot()) { return false; } // Multi-root aware — same resolver as the agent's tool layer (see applyEdit). - const abs = resolveWorkspacePath(req.path || '', { mustExist: true }); + const abs = await resolveWorkspacePathAsync(req.path || '', { mustExist: true }); if (!abs) { return false; } - const uri = vscode.Uri.file(abs); + const uri = host.uriFor(abs); const key = uri.toString(); let snapshot; if (pending.has(key)) { snapshot = pending.get(key).snapshot; } - else { try { snapshot = fs.readFileSync(abs, 'utf8').replace(/^/, ''); } catch { return false; } } + else { try { snapshot = (await host.readText(abs)).replace(/^/, ''); } catch { return false; } } // Checkpoint hook: the file EXISTED (created=false) → per-turn Restore recreates it from this snapshot. if (recordTouch) { try { recordTouch(key, snapshot, false); } catch (e) { /* checkpoint capture must never break editing */ } } const edit = new vscode.WorkspaceEdit(); @@ -239,7 +240,7 @@ function registerReview(context, post, dbg, recordTouch) { const fr = pending.get(key); if (!fr) { return; } const edit = new vscode.WorkspaceEdit(); - const gone = !fs.existsSync(fr.uri.fsPath); + const gone = !(await host.exists(fr.uri.fsPath)); if (fr.exists && gone) { // the agent DELETED this file → recreate it from the snapshot edit.createFile(fr.uri, { ignoreIfExists: true }); @@ -333,11 +334,11 @@ function registerReview(context, post, dbg, recordTouch) { vscode.window.onDidChangeActiveTextEditor(() => updateActiveUi()), vscode.commands.registerCommand('levelcode.ai.review.keepActive', () => { const k = activePendingKey(); if (k) { keepFile(k, 'kept'); } }), vscode.commands.registerCommand('levelcode.ai.review.undoActive', () => { const k = activePendingKey(); if (k) { undoFile(k); } }), - vscode.languages.registerCodeLensProvider({ scheme: 'file' }, lensProvider), + vscode.languages.registerCodeLensProvider({ scheme: host.isBrowser ? '*' : 'file' }, lensProvider), vscode.workspace.registerTextDocumentContentProvider(SNAPSHOT_SCHEME, snapshotProvider), vscode.window.onDidChangeVisibleTextEditors(() => { for (const fr of pending.values()) { decorate(fr); } }), vscode.workspace.onDidChangeTextDocument((e) => { - if (e.document.uri.scheme !== 'file') { return; } + if (!host.isBrowser && e.document.uri.scheme !== 'file') { return; } // a browser workspace has its own schemes const key = e.document.uri.toString(); if (!pending.has(key)) { return; } if (expected.get(key) === e.document.getText()) { return; } // our own apply — ignore @@ -361,18 +362,35 @@ function registerReview(context, post, dbg, recordTouch) { ); // Rehydrate pending reviews from a previous window (so a reload mid-review keeps Keep/Undo working). - for (const s of (context.workspaceState.get(PERSIST_KEY, []) || [])) { - try { - const uri = vscode.Uri.parse(s.uri); - let current = ''; - try { current = fs.readFileSync(uri.fsPath, 'utf8').replace(/^/, ''); } catch { /* file gone */ } - const span = changedSpan(s.snapshot, current); - const { add, del } = countDiff(makeDiff(s.snapshot, current)); - pending.set(s.uri, { uri, rel: s.rel, exists: s.exists, snapshot: s.snapshot, range: new vscode.Range(span.start, 0, span.end, 0), add, del, appliedText: current }); - expected.set(s.uri, current); - } catch { /* */ } + const saved = context.workspaceState.get(PERSIST_KEY, []) || []; + function rehydrate(readCurrent) { + for (const s of saved) { + try { + const uri = vscode.Uri.parse(s.uri); + const current = readCurrent(uri, s); + const span = changedSpan(s.snapshot, current); + const { add, del } = countDiff(makeDiff(s.snapshot, current)); + pending.set(s.uri, { uri, rel: s.rel, exists: s.exists, snapshot: s.snapshot, range: new vscode.Range(span.start, 0, span.end, 0), add, del, appliedText: current }); + expected.set(s.uri, current); + } catch { /* */ } + } + if (pending.size) { for (const fr of pending.values()) { decorate(fr); } refreshLenses(); updateActiveUi(); } + } + if (!host.isBrowser) { + rehydrate((uri) => { try { return fs.readFileSync(uri.fsPath, 'utf8').replace(/^/, ''); } catch { return ''; /* file gone */ } }); + } else { + // A browser workspace can only be read asynchronously: read every pending file, then rehydrate. + void (async () => { + const current = new Map(); + for (const s of saved) { + try { current.set(s.uri, ((await host.readTextOrNull(vscode.Uri.parse(s.uri).fsPath)) || '').replace(/^/, '')); } catch { /* */ } + } + rehydrate((uri, s) => current.get(s.uri) || ''); + // The chat may have asked for the cards (resync, on `ready`) while these reads were still out, and was + // sent none. Send them now; a chat that is not listening yet asks again when it is. + if (pending.size) { resync(); } + })(); } - if (pending.size) { for (const fr of pending.values()) { decorate(fr); } refreshLenses(); updateActiveUi(); } /** Re-send all pending review cards + state to the webview (called when the chat view becomes ready). */ function resync() { diff --git a/extensions/levelcode-ai/test/diagramHost.test.js b/extensions/levelcode-ai/test/diagramHost.test.js index 833c1b6..efabe0e 100644 --- a/extensions/levelcode-ai/test/diagramHost.test.js +++ b/extensions/levelcode-ai/test/diagramHost.test.js @@ -114,7 +114,7 @@ const HOST = ['clientRender', 'openDiagramLink', 'diagramMarkdown', 'exportDiagr // eslint-disable-next-line no-new-func const makeHost = new Function('env', [ "'use strict';", - 'const { vscode, fs, path, os, catalog, sessionMemory, diagrams, diagramLinks, diagramExport, diagramText, diagramTool, diagramBundle, diagramAscii, aiConfig, dbg,', + 'const { vscode, host, fs, path, os, catalog, sessionMemory, diagrams, diagramLinks, diagramExport, diagramText, diagramTool, diagramBundle, diagramAscii, aiConfig, dbg,', ' recordDiagramStat, agentFlow, findCompactionCut, estimateMsgTokens, meterModel, prepProviderRequest, authRetry, providers, checkpoints, ctx,', ' sessionsManager, currentContextLimit, post, replaySessionExpired, postContextFiles, refreshSessions, focusChatView } = env;', "const COMPACT_SYSTEM = 'system', COMPACT_INSTRUCTIONS = 'Summarize:\\n';", @@ -134,7 +134,11 @@ function boot(over) { const settings = Object.assign({ 'diagrams.enabled': true }, over && over.settings); const diagrams = createDiagrams({ resolveLink: (link) => diagramLinks.resolveLink(link, [{ name: 'app', root: ws }]), onStat: (ev) => stats.push(ev) }); const env = { - vscode, fs, path, os, catalog, sessionMemory, diagrams, diagramLinks, diagramExport, diagramText, diagramTool, diagramBundle: bundle, + vscode, + // The host module (host.js) as the desktop sees it: workspace paths are files on disk. + host: { isBrowser: false, caps: { shell: true, mcpStdio: true, ripgrep: true, unaskedSignIn: true }, uriFor: (abs) => vscode.Uri.file(abs), + withStats: async (run) => run(fs), knownFilesView: () => null }, + fs, path, os, catalog, sessionMemory, diagrams, diagramLinks, diagramExport, diagramText, diagramTool, diagramBundle: bundle, diagramAscii: (over && over.ascii) || ascii, aiConfig: () => ({ get: (k, d) => (k in settings ? settings[k] : d) }), dbg: (label, data) => logs.push({ label, data }), diff --git a/modules/extension-signature/test/extensionSignature.test.js b/modules/extension-signature/test/extensionSignature.test.js index c3a666c..1d4f19d 100644 --- a/modules/extension-signature/test/extensionSignature.test.js +++ b/modules/extension-signature/test/extensionSignature.test.js @@ -1083,7 +1083,8 @@ function moduleTrusting(keys) { }); await test('the gate runs this suite, and a release asks the registry and the built app', () => { - assert.match(read('scripts', 'test-extensions.sh'), /^for t in extensions\/\*\/test\/\*\.test\.js modules\/\*\/test\/\*\.test\.js; do$/m); + // One of the loop's patterns, wherever it sits: the gate also discovers other trees (web/test/unit). + assert.match(read('scripts', 'test-extensions.sh'), /^for t in (?:\S+ )*modules\/\*\/test\/\*\.test\.js(?: \S+)*; do$/m); const release = read('.github', 'workflows', 'release.yml'); const gate = position(release, 'run: ./scripts/test-extensions.sh', 'release.yml'); const asked = position(release, 'run: node scripts/extension-signature.mjs registry', 'release.yml'); diff --git a/scripts/build-web.mjs b/scripts/build-web.mjs new file mode 100644 index 0000000..19611c1 --- /dev/null +++ b/scripts/build-web.mjs @@ -0,0 +1,180 @@ +#!/usr/bin/env node +// Build LevelCode in your browser: one static directory, dist-web/, that any static host can serve. +// +// node scripts/build-web.mjs [--vscode vscode] [--out dist-web] [--account https://levelcode.ai] [--api-url ] +// [--webview-origin 'https://{{uuid}}.view.example.com'] +// [--ext-host-origin 'https://{{uuid}}.ext.example.com'] +// [--static ] [--id ] +// +// What it does, in order (docs/WEB.md has the reasoning): +// 1. builds Code-OSS's web client from the bootstrapped checkout (gulp vscode-web-min-ci, about a +// minute), or takes a prebuilt one with --static; +// 2. bundles levelcode-ai for the web worker extension host (web/ai-extension), copies the scratch +// workspace extension (web/workspace) and every declarative built-in (grammars, themes, icons); +// 3. names the build after a hash of everything it staged, lays that out under the build-addressed +// prefix (/_//...), renders index.html with the configuration baked in, and writes the HTTP +// policy for the host (_headers, nginx). +// +// It changes nothing outside --out, except that building the client from the checkout leaves gulp's own output, +// vscode-web/, next to it (pass --static to use a client built earlier and skip that). It does not deploy. +import { spawnSync, execFileSync } from 'node:child_process'; +import fs from 'node:fs'; +import path from 'node:path'; +import { createHash } from 'node:crypto'; +import { fileURLToPath } from 'node:url'; +import { webConfig, renderIndex } from '../web/lib/config.mjs'; +import { stageDeclarativeExtensions, copyExtension, withoutConfigurationDefaults } from '../web/lib/extensions.mjs'; +import { policy, toHeadersFile, toNginx } from '../web/lib/headers.mjs'; +import { hashTree } from '../web/lib/fingerprint.mjs'; + +const REPO = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); +const WEB = path.join(REPO, 'web'); + +const args = Object.fromEntries(process.argv.slice(2).reduce((a, v, i, all) => { + if (v.startsWith('--')) { a.push([v.slice(2), all[i + 1] && !all[i + 1].startsWith('--') ? all[i + 1] : 'true']); } + return a; +}, [])); +const vscodeDir = path.resolve(REPO, args.vscode || 'vscode'); +const outDir = path.resolve(REPO, args.out || 'dist-web'); +const account = String(args.account || process.env.LEVELCODE_WEB_ACCOUNT || 'https://levelcode.ai').replace(/\/+$/, ''); +// Only for a deployment whose API is not on the account origin (production has one host for both). +const apiUrl = String(args['api-url'] || process.env.LEVELCODE_WEB_API_URL || '').replace(/\/+$/, ''); +const webviewOrigin = args['webview-origin'] || process.env.LEVELCODE_WEB_WEBVIEW_ORIGIN || ''; +const extHostOrigin = args['ext-host-origin'] || process.env.LEVELCODE_WEB_EXTHOST_ORIGIN || ''; + +const log = (m) => console.log('[build-web] ' + m); +const die = (m) => { console.error('[build-web] ' + m); process.exit(1); }; +const sizeOf = (p) => { let n = 0; const walk = (d) => { for (const e of fs.readdirSync(d, { withFileTypes: true })) { const f = path.join(d, e.name); e.isDirectory() ? walk(f) : (n += fs.statSync(f).size); } }; walk(p); return n; }; +const mb = (n) => (n / 1048576).toFixed(1) + ' MB'; + +/* ----- 1. Code-OSS's web client ---------------------------------------------------------------- */ +let staticDir; +if (args.static) { + staticDir = path.resolve(args.static); + log('using the prebuilt web client at ' + staticDir); +} else { + if (!fs.existsSync(path.join(vscodeDir, 'node_modules'))) { die('no bootstrapped checkout at ' + vscodeDir + ' — run scripts/bootstrap.sh first'); } + const product = JSON.parse(fs.readFileSync(path.join(vscodeDir, 'product.json'), 'utf8')); + if (product.nameShort !== 'LevelCode') { die('the checkout is not branded (product.json nameShort is "' + product.nameShort + '") — run scripts/apply-branding.mjs'); } + log('building the web client (gulp vscode-web-min-ci) in ' + vscodeDir); + const r = spawnSync('npm', ['run', 'gulp', '--', 'vscode-web-min-ci'], { cwd: vscodeDir, stdio: 'inherit', env: { ...process.env, NODE_OPTIONS: '--max-old-space-size=8192' } }); + if (r.status !== 0) { die('the web client build failed'); } + staticDir = path.resolve(vscodeDir, '..', 'vscode-web'); +} +for (const must of ['out/vs/workbench/workbench.web.main.internal.js', 'out/vs/workbench/workbench.web.main.internal.css', 'out/nls.messages.js', + 'out/vs/workbench/services/extensions/worker/webWorkerExtensionHostIframe.html', 'out/vs/workbench/contrib/webview/browser/pre/index.html']) { + if (!fs.existsSync(path.join(staticDir, must))) { die('the web client is missing ' + must); } +} + +/* ----- 2. assemble, under a name the build does not have yet ------------------------------------- */ +// Everything under /_// is served as never changing, so the id is a hash of exactly what is there: it is put +// together first, in a staging folder, and named after it. A list of inputs to hash missed what it did not name — +// a theme, a grammar, the web client's CSS or a worker — and a changed file was then served stale for a year. +// Nothing staged depends on the id; only the page (index.html, outside it) names it. +fs.rmSync(outDir, { recursive: true, force: true }); +const staging = path.join(outDir, '_', '.staging'); +fs.mkdirSync(staging, { recursive: true }); +let root = staging; + +log('copying the web client'); +fs.cpSync(staticDir, path.join(root, 'static'), { recursive: true, filter: (src) => !/\.(js|css)\.map$/.test(src) }); +const baked = new Set(fs.existsSync(path.join(staticDir, 'extensions')) ? fs.readdirSync(path.join(staticDir, 'extensions')) : []); + +const extOut = path.join(root, 'extensions'); +fs.mkdirSync(extOut, { recursive: true }); + +log('bundling levelcode-ai for the web worker extension host'); +const r = spawnSync(process.execPath, [path.join(WEB, 'ai-extension', 'build.mjs'), '--out', extOut, '--vscode', vscodeDir, '--minify'], { stdio: 'inherit' }); +if (r.status !== 0) { die('the levelcode-ai browser build failed'); } + +log('staging the scratch workspace and the declarative built-ins'); +const scratchOut = path.join(extOut, 'levelcode-web'); +copyExtension(path.join(WEB, 'workspace'), scratchOut); +fs.writeFileSync(path.join(scratchOut, 'package.nls.json'), '{}\n'); +const extensionsSource = path.join(vscodeDir, 'extensions'); +if (!fs.existsSync(extensionsSource)) { die('no extensions/ in ' + vscodeDir + ' to take the declarative built-ins from'); } +const declarative = stageDeclarativeExtensions(extensionsSource, extOut, { exclude: [...baked, 'levelcode-ai', 'levelcode-web'] }); +// A checkout that has not had LevelCode's extensions copied in still builds; the themes are then absent. +if (!declarative.includes('levelcode-themes')) { + const themes = path.join(REPO, 'extensions', 'levelcode-themes'); + if (fs.existsSync(themes)) { copyExtension(themes, path.join(extOut, 'levelcode-themes')); declarative.push('levelcode-themes'); } +} +// The browser edition's own colour themes. The desktop's default theme is pinned by levelcode-themes' manifest +// (an extension's defaults outrank the page's), so that one is dropped here; its themes stay selectable. +const themeOut = path.join(extOut, 'levelcode-web-theme'); +copyExtension(path.join(WEB, 'theme', 'levelcode-web-theme'), themeOut); +fs.writeFileSync(path.join(themeOut, 'package.nls.json'), '{}\n'); +const desktopThemes = path.join(extOut, 'levelcode-themes', 'package.json'); +if (fs.existsSync(desktopThemes)) { withoutConfigurationDefaults(desktopThemes); } +const extensions = ['levelcode-ai', 'levelcode-web', 'levelcode-web-theme', ...declarative.sort()]; +log(`${extensions.length} extensions: levelcode-ai, levelcode-web, levelcode-web-theme and ${declarative.length} declarative (${declarative.slice(0, 6).join(', ')}, ...)`); + +// main.js and boot.css are addressed by the build id like everything else; the page names them. +for (const f of ['main.js', 'boot.css', 'chrome.css']) { fs.copyFileSync(path.join(WEB, f), path.join(root, f)); } + +/* ----- 3. identity of this build: what was staged ------------------------------------------------- */ +const id = String(args.id || hashTree(createHash('sha256'), staging, { skip: ['.DS_Store'] }).digest('hex').slice(0, 12)); +const prefix = `/_/${id}`; +root = path.join(outDir, '_', id); +fs.renameSync(staging, root); +let gitHead = 'unknown'; +try { gitHead = execFileSync('git', ['-C', REPO, 'rev-parse', '--short=10', 'HEAD'], { encoding: 'utf8' }).trim(); } catch { /* not a checkout */ } +log('build id ' + id + ' (levelcode ' + gitHead + ')'); + +log('writing the page'); + +const productConfiguration = { + // No gallery: nothing is installed from the Internet into the page's own origin. + extensionsGallery: null, +}; +if (!webviewOrigin) { + // Code-OSS publishes each stable commit's webview host pages to its CDN. product.json pins an older insider commit, + // whose service worker is a version behind this build's; the template below names this build's own commit. + productConfiguration.webviewContentExternalBaseUrlTemplate = 'https://{{uuid}}.vscode-cdn.net/{{quality}}/{{commit}}/out/vs/workbench/contrib/webview/browser/pre/'; + console.warn('[build-web] note: no --webview-origin, so the chat panel loads its host page from Code-OSS\'s CDN (*.vscode-cdn.net, this build\'s own commit).\n' + + ' That works with no setup. To serve it yourself give a wildcard origin, e.g. --webview-origin \'https://{{uuid}}.view.example.com\'\n' + + ' (each webview needs its own subdomain; see docs/WEB.md).'); +} +if (webviewOrigin) { productConfiguration.webviewContentExternalBaseUrlTemplate = `${webviewOrigin}${prefix}/static/out/vs/workbench/contrib/webview/browser/pre/`; } +if (extHostOrigin) { productConfiguration.webEndpointUrlTemplate = `${extHostOrigin}${prefix}/static`; } +const config = webConfig({ + account, + apiUrl, + extensions, + base: prefix, + staticBase: `${prefix}/static`, + extensionsBase: `${prefix}/extensions`, + productConfiguration, +}); +fs.writeFileSync(path.join(outDir, 'index.html'), renderIndex(config)); +for (const f of ['callback.html', 'callback.js', 'callback.css', 'icon.svg', 'icon-192.png', 'icon-512.png', 'manifest.webmanifest']) { + fs.copyFileSync(path.join(WEB, f), path.join(outDir, f)); +} + +/* ----- 4. the host's half ------------------------------------------------------------------------ */ +const toPattern = (o) => (o ? o.replace('{{uuid}}', '*') : ''); +const pol = policy({ account, webviewOrigin: toPattern(webviewOrigin), extHostOrigin: toPattern(extHostOrigin), base: '/_' }); +fs.writeFileSync(path.join(outDir, '_headers'), toHeadersFile(pol)); +fs.mkdirSync(path.join(outDir, 'deploy'), { recursive: true }); +fs.writeFileSync(path.join(outDir, 'deploy', 'nginx.conf'), toNginx(pol)); +const info = { + id, levelcode: gitHead, builtAt: new Date().toISOString(), account, apiUrl: apiUrl || '(the account origin)', + webviewOrigin: webviewOrigin || '(Code-OSS default: *.vscode-cdn.net)', + extHostOrigin: extHostOrigin || '(same origin as the editor)', + extensions, +}; +fs.writeFileSync(path.join(outDir, 'build.json'), JSON.stringify(info, null, 2) + '\n'); + +/* ----- 5. check what was written ----------------------------------------------------------------- */ +const html = fs.readFileSync(path.join(outDir, 'index.html'), 'utf8'); +const m = html.match(/\n'; + +const read = (name) => fs.readFileSync(path.join(__dirname, 'skin', name), 'utf8'); + +/** + * @param {string} html the desktop chat.html (after copy.js has rewritten its desktop-only words) + * @returns {string} the same page with the browser skin + * @throws if the page is not the shape this expects, or a skin file could end its own block early + */ +function skinChat(html) { + const css = read('chat.css'); + const js = read('chat.js'); + for (const [name, text] of [['chat.css', css], ['chat.js', js]]) { + if (/<\/(?:script|style)/i.test(text) || text.includes('