Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .changeset/classic-sync-in-the-tab.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
"@inkandswitch/patchwork-bootloader": patch
"@inkandswitch/patchwork": patch
---

Classic automerge-repo sync connects from the tab's own Repo instead of the automerge worker, so a worker restart no longer drops it and tabs no longer ask the replacement to connect again.

`connectClassicSync` in `@inkandswitch/patchwork-bootloader` now takes the Repo: `connectClassicSync(repo, server?)`. `setupServiceWorker()` no longer returns `connectClassicSync`. `patchwork.sw.connectClassicSync(server?)` works as before. The `ConnectClassicSyncMessage` type is gone.
6 changes: 6 additions & 0 deletions .changeset/drop-package-memory.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"@inkandswitch/patchwork-bootloader": patch
"@inkandswitch/patchwork-filesystem": patch
---

The automerge worker no longer measures package memory. Removed: the `@patchwork/package-memory` channel and its message types (`PACKAGE_MEMORY_CHANNEL`, `PackageMemoryRequestMessage`, `PackageMemoryDocument`, `PackageMemoryAcceptedMessage`, `PackageMemoryReplyMessage`), and the records a restarting worker saved for its replacement (`LOADED_CACHE`). The service worker clears that cache along with every other cache but its own. `packageDocuments` is gone from `@inkandswitch/patchwork-filesystem`; nothing else walked a package.
13 changes: 13 additions & 0 deletions .changeset/every-tab-its-own-node.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
"@inkandswitch/patchwork-bootloader": patch
"@inkandswitch/patchwork": patch
"@inkandswitch/patchwork-filesystem": patch
---

Every tab is its own Subduction node again, with its own socket to the sync server. The sync node in the automerge worker is gone. When the worker restarts (to recycle its wasm memory, for a new build, or after a trap), it takes only its resolver Repo with it. Before, every restart threw away the node every tab synced through, and each tab had to reconnect and resubscribe. The worker's resolver Repo has its own socket too.

The automerge worker still holds `AUTOMERGE_WORKER_LOCK` for as long as it lives. The lock now does one thing: it lets tabs and the service worker tell when the worker has died. A replacement no longer waits for it before it builds its Repo, answers handoffs or announces itself `online`. An instance that's restarting already stops taking handoffs and package-memory requests, so the two don't answer at once.

Removed from `@inkandswitch/patchwork-bootloader`: `getSyncRemote`, `watchDocument`, `relayKeyhive` and the `PortSyncRemote` type. Pass `subductionWebsocketEndpoints: [syncServerUrl]` to the Repo instead of `subductionRemote: getSyncRemote()`. On keyhive sites, use `new Repo(config)` directly rather than wrapping it in `relayKeyhive`. `loadKeyhiveSigner` is gone from `@inkandswitch/patchwork-bootloader/signer`. `startWorkerControl` no longer returns `holding`.

Tabs still follow package docs without loading them. `watchDocument(repo, url, onChange)`, now exported from `@inkandswitch/patchwork`, subscribes to the document through the tab's own Subduction connection and listens on the origin's heads channel. Every Repo relays the server's heads there, so the watcher hears both the server's pushes and other tabs' writes. `onChange(seen)` gets the same summary `ModuleWatcher` already compares, so a repeat costs no lookup. Subduction can't unsubscribe from a single document, so after `unwatch()` the server keeps pushing it to that tab until the connection drops.
21 changes: 21 additions & 0 deletions .changeset/hashed-worker-files.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
"@inkandswitch/patchwork": patch
"@inkandswitch/patchwork-bootloader": patch
"@inkandswitch/patchwork-e2e": patch
---

A build emits every file the automerge worker and the page load under a content-hashed name in `/assets`, served `immutable`, instead of at a fixed path: the `/packages/…` chunks, `automerge.wasm`, `subduction.wasm`, `keyhive_wasm.wasm`, the worker's second stage (`automerge-protocol-handler.js`) and `module-loader-worker.js`. A fixed path could only be revalidated, which fails offline, or cached on a guess, which after a deploy could start the worker on one build's files and the rest of another's. Now a build only ever loads its own copies, and they start offline from the HTTP cache.

Only `service-worker.js` and the automerge worker's script, `automerge-protocol-handler-worker.js`, keep their fixed paths, since the browser starts both by URL from one build to the next. The worker's script is served `max-age=0, stale-while-revalidate=31536000`, so the browser starts it from its copy when it can't check for a newer one. Everything that copy imports is hashed, so even a copy from an older build brings up one whole build. A worker that restarts because a newer tab connected fetches its script past the cache first, so the replacement starts on the newer build rather than the cached one, which would carry on serving the newer tab.

The fixed paths still work: the generated `_redirects` sends each one to its hashed name with a 302, and `vite preview` does the same, whether or not the site writes Netlify files. But a module imported both by a fixed path and by its hashed name is two modules, so code should import the packages by bare specifier (the import map points at the hashed names) and load the rest by the URLs in `@inkandswitch/patchwork-bootloader/emitted`: `AUTOMERGE_WASM`, `SUBDUCTION_WASM`, `KEYHIVE_WASM`, `AUTOMERGE_PROTOCOL_HANDLER` and `MODULE_LOADER_WORKER`. In dev these are the fixed paths.

API changes:

- `emitWasmAssets` (`@inkandswitch/patchwork-bootloader/externals`) emits the wasm under hashed names and returns each one's reference id by file name.
- `@inkandswitch/patchwork/vite` has a `hashed()` plugin, included in `patchwork()`, which swaps in the hashed names. `hashedFiles(config)` gives a build's table of them, also exported from `@inkandswitch/patchwork/vite/hashed`.
- `workers` entries say whether a worker is `hashed`. The workers no longer import the packages from `/packages/…` URLs: they link to the build's chunks like the page does.
- `buildHeaders(options, hashed?)` takes that table for the wasm preloads.
- The new `buildRedirects(hashed?)` writes the redirects. `REVALIDATED_PATHS` and `REVALIDATED_CACHE_CONTROL` name the stale-while-revalidate files and their header.

The e2e offline test waits for the wasm by the URLs the page preloads.
5 changes: 5 additions & 0 deletions .changeset/shrink-automerge-repo-patch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@inkandswitch/patchwork": patch
---

The shipped automerge-repo patch no longer adds `subductionRemote`, the option that let a Repo sync through another node; nothing uses it since every tab became its own node again. The `SubductionRemote`, `SubductionRemoteHandlers` and `SubductionRemoteSyncResult` types are gone, as are the `SubductionStorageBridge`, `INTERCEPTOR_PREFIX` and `SubductionConnections` exports. `subductionSaveThrottle` stays. A site applying the patch from `@inkandswitch/patchwork/dist/patches` should take the new one.
12 changes: 12 additions & 0 deletions .changeset/tabs-resolve-handoffs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
"@inkandswitch/patchwork-bootloader": patch
"@inkandswitch/patchwork": patch
---

The service worker asks a tab to resolve `automerge:` requests before falling back to the automerge worker. Every tab is its own node with its own Repo, so the tab that made a request can usually answer it from documents it already holds.

The service worker asks the tab that made the request first, then any other open tab (focused first, then visible). This covers requests from the module-loader worker and navigations, which have no window of their own. Only when no tab can answer does it broadcast to the automerge worker, which now handles just those requests.

The tab replies `accepted` straight away, or `unavailable` if it has no Repo yet. A tab that says nothing within 250ms is passed over. That number is provisional: the service worker logs each tab's accept latency under `patchwork:serviceworker`, to measure from. A tab that has accepted holds a Web Lock for that one request until it has replied. If the lock is released with no reply, the tab has gone (closed, crashed, navigated away) and the next candidate is asked.

New in `@inkandswitch/patchwork-bootloader`: `resolveHandoffsWith(repo)` makes this tab answer with `repo`. `@inkandswitch/patchwork` calls it with the Repo it creates; a site that passes its own Repo isn't asked and keeps resolving through the automerge worker. The resolving code the worker used is shared by both, in `resolver.ts`. The new messages are `ResolveRequestMessage`, `ResolveAcceptedMessage` and `ResolveUnavailableMessage`.
9 changes: 9 additions & 0 deletions .changeset/worker-recycle-fixes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
"@inkandswitch/patchwork-bootloader": patch
---

Fixes to how the automerge worker recycles itself to give back wasm memory:

- The service worker no longer deletes the cache a restarting automerge worker saves what it had loaded into (`patchwork-package-memory`, now `LOADED_CACHE` in the bootloader's types). It cleared every cache but its own on each activation, so the replacement never had anything to restore.
- A worker with classic sync connected recycles like any other. Tabs ask the replacement to connect classic sync again, after a restart and after the worker goes unexpectedly, so connecting it no longer stops the worker from ever recycling.
- The recycle check counts subduction's wasm memory too, not only automerge's.
5 changes: 5 additions & 0 deletions .changeset/worker-repo-on-self.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@inkandswitch/patchwork-bootloader": patch
---

The automerge worker's resolver Repo is on `self.repo` again from the moment the worker starts, for the console. The Repo is only built once something has to be resolved, so until then reading `self.repo` starts building it and gives its promise. `await repo` works either way.
131 changes: 104 additions & 27 deletions .github/workflows/bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,24 +11,15 @@ env:
MARKER: "<!-- patchwork-bench -->"

jobs:
bench:
name: benchmarks
# Say a run is under way, keeping the last results below it, marked as out
# of date, until this one replaces them.
announce:
name: mark the comment out of date
runs-on: ubuntu-latest
timeout-minutes: 120
permissions:
contents: read
pull-requests: write
# main and the PR on the same runner, one after the other: runner numbers
# drift between runs, so a PR is only ever compared with main measured
# alongside it. Only Patchwork as it ships (`patchwork` mode, whose tests
# are titled "patchwork: …" or "S5 patchwork, …"): the other modes and the
# storage adapters are alternatives the bench explores, not what merging
# changes.
steps:
# Say a run is under way, keeping the last results below it, marked as
# out of date, until this one replaces them.
- name: mark the comment out of date
uses: actions/github-script@v7
- uses: actions/github-script@v7
with:
script: |
const marker = process.env.MARKER
Expand Down Expand Up @@ -59,6 +50,24 @@ jobs:
await github.rest.issues.createComment({ owner, repo, issue_number, body })
}

# One runner per browser, so the browsers run at once without sharing a
# machine. On each, main and the PR run one after the other: runner numbers
# drift between runs, so a PR is only ever compared with main measured
# alongside it, in the same browser. Only Patchwork as it ships (`patchwork`
# mode, whose tests are titled "patchwork: …" or "S5 patchwork, …"): the
# other modes and the storage adapters are alternatives the bench explores,
# not what merging changes.
bench:
name: bench (${{ matrix.browser }})
runs-on: ubuntu-latest
timeout-minutes: 120
permissions:
contents: read
strategy:
fail-fast: false
matrix:
browser: [chromium, firefox, webkit]
steps:
- uses: actions/checkout@v4
with:
path: pr
Expand Down Expand Up @@ -89,42 +98,93 @@ jobs:
working-directory: pr
run: pnpm install --frozen-lockfile && pnpm build

- name: install chromium
- name: install ${{ matrix.browser }}
run: |
(cd main/sites/bench && pnpm exec playwright install --with-deps chromium)
(cd pr/sites/bench && pnpm exec playwright install --with-deps chromium)
(cd main/sites/bench && pnpm exec playwright install --with-deps ${{ matrix.browser }})
(cd pr/sites/bench && pnpm exec playwright install --with-deps ${{ matrix.browser }})

# A single run of a benchmark can be off by more than the PR changes it,
# so each side runs ROUNDS times, alternating, so that the runner
# slowing down partway affects both. Failed tests still record their
# rows, so every round reports whatever got measured; main failing
# doesn't fail the PR, the PR failing does, once it's been reported.
# rows, so every round reports whatever got measured. A side whose
# bench.config.ts has no project for this browser records nothing.
# Whether the PR passes is the report's call, from what's uploaded: a
# PR whose tests threw leaves `threw` beside its runs.
- name: bench main and the PR, alternating
env:
ROUNDS: 3
BROWSER: ${{ matrix.browser }}
run: |
for side in main pr; do (cd $side/sites/bench && pnpm exec vite build); done
mkdir -p runs
for round in $(seq "$ROUNDS"); do
for side in main pr; do
echo "::group::$side, round $round"
rm -f $side/sites/bench/bench-results/*.jsonl
if ! (cd $side/sites/bench && pnpm exec playwright test -c bench.config.ts --grep "patchwork[:,]"); then
if [ "$side" = pr ]; then touch pr-failed; fi
if ! (cd $side/sites/bench && pnpm exec playwright test -c bench.config.ts --project="$BROWSER" --grep "patchwork[:,]"); then
if [ "$side" = pr ]; then touch runs/threw; fi
fi
mkdir -p "runs/$side-$round"
cp $side/sites/bench/bench-results/*.jsonl "runs/$side-$round/" 2>/dev/null || true
echo "::endgroup::"
done
done

- name: compare
- uses: actions/upload-artifact@v4
if: always()
with:
name: bench-runs-${{ matrix.browser }}
path: runs/
retention-days: 7

# Every browser's runs side by side, in one comment, and the one check that
# says whether the PR passes: it fails when the PR's benchmarks threw in any
# browser, or a check (an "ok" row) that main passes failed in any round of
# the PR (see compare.ts). Main failing doesn't fail the PR.
report:
name: benchmarks
needs: bench
# Not for a run cancelled by a newer push: that one reports instead.
if: ${{ !cancelled() }}
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: sites/bench/compare.ts
sparse-checkout-cone-mode: false

- uses: actions/setup-node@v4
with:
node-version: 24

- uses: actions/download-artifact@v4
with:
pattern: bench-runs-*
path: runs

- name: compare
run: |
node pr/sites/bench/compare.ts --main runs/main-* --pr runs/pr-* > compare.md
threw=""
for browser in chromium firefox webkit; do
dir="runs/bench-runs-$browser"
if [ ! -d "$dir" ]; then
threw="$threw $browser (recorded nothing)"
elif [ -e "$dir/threw" ]; then
threw="$threw $browser"
fi
done
node sites/bench/compare.ts --main runs/*/main-* --pr runs/*/pr-* --why why.txt > compare.md || touch pr-regressed
if [ -n "$threw" ]; then
echo "$threw" > pr-failed
{ echo "**The PR's benchmarks threw in:$threw.** See the run's logs."; echo; cat compare.md; } > compare.tmp
mv compare.tmp compare.md
fi
cat compare.md >> "$GITHUB_STEP_SUMMARY"

- uses: actions/github-script@v7
if: always()
with:
script: |
const fs = require("fs")
Expand All @@ -140,7 +200,7 @@ jobs:
marker,
`## Benchmarks: Patchwork with this PR against main`,
``,
`\`${head}\` against main at \`${base}\`, Patchwork as it ships (\`patchwork\` mode), measured one after the other on the same runner ([run](${run})). Lower is better throughout. A row is in the table when every run of the PR came out on the same side of every run of main, with the medians more than 20% apart; every benchmark is below it.`,
`\`${head}\` against main at \`${base}\`, Patchwork as it ships (\`patchwork\` mode), in chromium, firefox and webkit, each on its own runner with main and the PR measured one after the other ([run](${run})). Lower is better throughout. A row is in the table when every run of the PR came out on the same side of every run of main, with the medians more than 20% apart; every benchmark is below it.`,
``,
compare,
].join("\n")
Expand All @@ -154,6 +214,23 @@ jobs:
await github.rest.issues.createComment({owner, repo, issue_number, body})
}

# Says why as error annotations, which show on the run's summary page and
# the PR's checks. The comment has the same, with the tables.
- name: fail if the PR's benchmarks failed
if: always()
run: test ! -e pr-failed
run: |
failed=0
if [ -e pr-failed ]; then
echo "::error title=benchmarks threw::The PR's benchmarks threw in:$(cat pr-failed). See that browser's bench job for the test that threw."
failed=1
fi
if [ -e pr-regressed ]; then
if [ -s why.txt ]; then
while IFS= read -r line; do
echo "::error title=check failed::${line//%/%25}"
done < why.txt
else
echo "::error title=compare failed::compare.ts exited without naming a check that failed; see the compare step."
fi
failed=1
fi
exit $failed
5 changes: 4 additions & 1 deletion core/bootloader/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@
"import": "./dist/externals.js",
"types": "./dist/externals.d.ts"
},
"./emitted": {
"import": "./dist/emitted.js",
"types": "./dist/emitted.d.ts"
},
"./externals-list": {
"import": "./dist/externals-list.js",
"types": "./dist/externals-list.d.ts"
Expand Down Expand Up @@ -109,7 +113,6 @@
},
"scripts": {
"build": "tsc && cp src/global.css dist/global.css",
"test": "vitest run",
"dev": "tsc -w --preserveWatchOutput"
}
}
7 changes: 2 additions & 5 deletions core/bootloader/src/automerge-protocol-handler-worker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,13 @@
// it starts, then imports the worker itself (automerge-protocol-handler.ts).
// Keep its imports small: none of this runs until they've all loaded.
import { AUTOMERGE_WORKER_LOCK } from "./types.js";
import { AUTOMERGE_PROTOCOL_HANDLER } from "./emitted.js";
import { importRest, startEarly } from "./worker-early.js";

// Built as a file of its own next to this one, so a variable rather than a
// literal: the bundler leaves the import alone.
const REST = "./automerge-protocol-handler.js";

// Not awaited: this script should finish evaluating now, so nothing about the
// worker waits on the import but the code that's in it.
void importRest(
startEarly(AUTOMERGE_WORKER_LOCK),
"automerge-protocol-handler-worker",
new URL(REST, import.meta.url).href
new URL(AUTOMERGE_PROTOCOL_HANDLER, import.meta.url).href
);
Loading
Loading