feat(pipeline): add the fact-record pipeline core - #474
Open
Menci wants to merge 42 commits into
Open
Conversation
The data plane is moving from an onion of nested interceptors to a list of stages over one immutable record of facts. This is the package everything else will compose; nothing in it knows what a model is. A stage declares where control may go and may declare more than one place — `return`, `through`, `into` — and there is no name for a kind of stage: the fields are the declaration. Five shapes are legal and `defineStage`'s five overloads admit exactly those, with the overload fixing what `next` accepts rather than a marker boolean or a conditional type. `transform` is sugar in the `execute` slot, two layers deep, and an omitted direction passes through unchanged — which is most of them, since today's interceptors call the continuation 107 times across 68 files and 50 of those are a bare return. Declarations are checked, never applied. After a stage hands on, in either direction and on the answering path too, the runner asserts that what it declared `provides` is there and what it declared `consumes` — and did not also declare providing — is gone. A key in both is a modify: a translation taking the source and putting the target elsewhere on the way down, a fork taking ownership of every branch's disposable and handing one onward on the way up. Nothing is removed on a stage's behalf, because a declaration that acts cannot also be checked. `compose` reasons over declarations, which are strings, so a stage written against one fact space drops into a pipeline over another with no variance question to lose. It catches what types cannot: an `into` stage that is not last, a short-circuit that does not cover what the stages above it need, a stage that can neither answer nor descend, and the load-bearing one — a key an earlier stage consumed cannot be needed below it, so a translated request cannot re-enter its own chain. The entry contract is derived from the stages and asserted when the first request arrives. Handover is enforced at runtime by one predicate at every place a value can enter the record. `Object.isFrozen` cannot be that predicate: freezing a typed array with elements throws, so the walk skips them, and a shallow builtin freeze satisfies `isFrozen` while its children stay mutable. So a WeakSet of handed-over roots is the gate and `Object.freeze` is what makes a later write throw. A handed record is frozen in place rather than copied, so a stage that hands on what it received hands on the same object. The dump is the encoding in the design record: events carrying states and never differences, an object id taken at first sight so a cycle terminates on its second visit, `$$` escaping a key that already begins with `$` (tool parameters are JSON Schema), large strings shared by value because a deep clone defeats reference sharing, and secrets stored as length, redaction and hash so a reader can see the same secret twice without ever holding one. Two things the runner does that a reader should not have to discover: the run context is threaded rather than ambient, because a module-level variable saved around an await interleaves two concurrent runs and a gateway serves concurrent requests by definition; and the disposal sweep runs in `finally`, because a stage that throws must not abandon an open upstream body — an aborted connection cannot be reused and leaves its billing unsettled. Tests carry every property, including the one nothing else guards: on a 49-message conversation with one message rewritten, 48 of 51 objects come back by identity, so a layer costs what its stage touched. A stage rewritten to rebuild unconditionally drops that to zero while every other test passes.
…ails
An adversarial review of the first commit ran five lenses over the package and
reproduced every claim it made. Nine blocking defects survived refutation, and
they shared one cause: `run` treated the answer as the only channel out, so
everything else — the events, the open bodies — was reachable only through the
success return.
· a run that 500s produced no dump at all, though a dump of a failing turn is
the one an operator most wants and the current gateway does keep it;
· every body opened below a throw was abandoned, because the sweep read a
variable that is only assigned on success — and the test written for that
case asserted the empty result as correct;
· the sweep was awaited before `run` resolved, so a streaming family could
never hand its stream back: the drain both blocked the handler and ate the
frames it was supposed to deliver;
· a body at a key a stage declared it consumes was released by nobody unless
the stage happened to fork, though ownership is a declaration and not an
arity;
· a fork released a body the stage had handed up under a different key, out
from under whoever was reading it.
Events now leave through a sink the prologue resolves, the moment they exist —
which also makes recording conditional without a mode flag, and is the seam a
live observer will attach to. What the run owns is tracked where the resource is
created rather than where it lands, so a body is known before the stack unwinds
past it. The drain is handed to the caller, because a streaming family's answer
*is* the stream; a run that threw has nothing left to hand back, so it drains
before the throw propagates.
Two encoding defects went the same way. A typed array was walked by index, so a
400 KiB image became 4.4 MB of NDJSON — an eleven-fold expansion of the value
most likely to be large, and enough to break the single-put sizing the storage
rests on. And `undefined` inside a fact vanished, which is precisely the failure
the design names when it explains why `undefined` is not a removal. Buffers,
`undefined`, `NaN`, the infinities and `bigint` each carry a tag now.
Assembly gained the two terminal checks it was missing: a last stage that can
only descend is unsatisfiable, and a stage that can only answer is where the
array ends. The handoff error names the caller as well as the target. The
handover gate covers the response direction and the answering path. `transform`
no longer repeats the runner's check with a worse message. A log line is
snapshotted where it is written, so a stored line is a state that existed.
`secret` takes a renderer whenever the value is not already a string, because
`String(bytes)` is not a rendering anybody would recognise.
The tests were rebuilt against the properties rather than against the outcome,
after the review showed several passed with the behaviour they named deleted.
Each is now checked by removing what it tests: dropping the drain, the handover
loop or the freeze each fails exactly the test that claims it.
A stage that carries both traits makes two statements, not one. The human's ruling separates them in as many words — 「如果它短路返回,会 provides 哪些东西, 此时只有 provides」 against 「如果它要透传后面 stage 的 await next() 返回的东西 ……它要 provides/consumes/needs 什么」 — and `compose` has always checked them apart, testing a short-circuit's `provides` against what the stages above need independently of the pass declaration. `defineStage` had them sharing one type parameter, so a stage could not answer with a key its descend path never carries. The first real family found it immediately: a resolver that refuses a request no upstream can serve answers with a refusal key, and passes through untouched when an upstream can — and the compiler rejected the passing-through path for not carrying a key that exists only when it did not pass through. The reference example could not have found this. Both of its both-trait stages answer with a key their descend path also carries, so the two slices coincide and the conflation is invisible. The test added here is the shape where they differ, and it fails to compile against the old signature. The bad error message was the second half of the defect: overload resolution fell through to the `into` shape and reported that `next` takes too few arguments, which is not the problem.
§1.4 listed this as the last shape it had not written, and called it "where the load-bearing typing lives". It needs nothing: a pipeline's interior is not part of its type, because `compose` takes stages whose slices are already erased, so a provider composes stages typed over its own space and exports a `Pipeline` whose type mentions only the gateway's. The proof is a provider that is a separate module with a fact space of its own. Its key is not closed over — it is created by one of its stages, travels in the record where the dump can see it, and is consumed by another before the request leaves. A caller cannot name it, because it cannot learn that it exists. Three further shapes, because the claim is about composition and not about one call: failover forking across the boundary from above it, two providers whose spaces differ reached through one seal type, and the key's absence from what the run answers with. The reference example could not have settled this. Its provider closes the credential over in a closure, so the key never enters the record and the question of a travelling foreign key never arises. This unblocks the chat migration, which is built entirely on handoffs of this kind — four protocol chains that translate into one another, each ending in a provider's sealed chain.
`Symbol.asyncDispose` is a release mechanism, and the language hands it out on
terms that do not match a gateway's in either direction. Measured on Node 24:
Symbol.asyncDispose in (async function*(){})() true every async generator
Symbol.asyncDispose in new ReadableStream() false what a body actually is
So the structural predicate was wrong twice over. It adopted every
generator-shaped fact as a run resource, which meant a stage consuming such a key
had its source released the moment it handed up — a transducer silently producing
zero frames — and a fork threw on receiving one at a key nobody declared
consuming, which is every failed-over streaming request. Meanwhile it missed the
upstream body the whole rule exists for.
`own(value, release)` is what says the run is answerable for a value. That is the
same thing `consumes` on the response side already declares, so declaration was
always the mechanism; this is the runner reading it rather than guessing.
Found by the first family that streams, which is the only way it could have been
found: the reference example's bodies are plain objects with a disposer, so both
halves of the mismatch are invisible there. The two tests added here are the two
halves — a generator the language marks and the run does not, and a stream the
language does not mark and the run does.
A stage had no way to begin work without reaching past the record for a scheduler, and anything it started that way escaped the run's own accounting. Deferral is now a property of the value: a helper marks a promise, the marked value enters the record like any other, and teardown waits for what this run began. Declared rather than sniffed. Testing for a then method would make a fact that happens to hold a promise indistinguishable from one the run owes work to, and it would be quietly awaited instead of reported. Per run and not per root, because a discarded branch's facts are private to it and a root-level sweep could never see them. Teardown has a deadline and exceeding it is an error a reader can act on, since a value that never settles would otherwise hold teardown open forever.
…a document The dump's log-line test named itself after structured fields and asserted every part of the line except them; removing the fields from the encoder left it green. Two comments pointed at design documents that are not in the repository, so a reader following them finds nothing.
…nerator
Two assertions read `Symbol.asyncDispose` off an async generator and required it
to be there. That is a fact about the host rather than about this runner, and it
is false on the host CI runs and `apps/platform-node` supports:
Node 24 Node 22
Symbol.asyncDispose in (async function*(){})() true false
Symbol.asyncDispose in new ReadableStream() false false
So the suite passed on a developer's Node 24 and failed on Node 22, where the
product behaves identically: ownership is claimed through `own()` and nothing is
ever sniffed.
The variance is worth writing down rather than deleting, because it is a second
argument for claiming. On Node 24 a structural predicate adopts every
generator-shaped fact; on Node 22 it adopts nothing; neither column is what a run
needs. A predicate whose answer depends on the host cannot be what decides which
resources a gateway closes.
What the tests assert now is the run's own answer, which is the same everywhere.
The four chat protocols were referred to by abbreviations that only work if you already know which vendor owns which shape: `responses`, `messages`, `chatCompletions` and `gemini`. `messages` in particular collides with the plural of `message`, which is also a wire field on two of those protocols, so reading a symbol was not enough to tell an endpoint from an array. Every identifier, module path, directory and internal string that names a protocol now carries its vendor and its endpoint: chatCompletions -> openaiChatCompletions / OpenAIChatCompletions / openai-chat-completions responses -> openaiResponses / OpenAIResponses / openai-responses messages -> anthropicMessages / AnthropicMessages / anthropic-messages gemini -> geminiGenerateContent / GeminiGenerateContent / gemini-generate-content Comments, docs and test names take the prose spelling of the same names: "OpenAI Responses", "Anthropic Messages", "OpenAI Chat Completions". Deliberately left alone, because the name is not ours to change: - Public URL paths and route shapes (`/v1/responses`, `/v1/messages`, `/v1/messages/count_tokens`, `:generateContent`, `supported_endpoints` values, operator path overrides). - Vendor and foreign-project strings: the OpenAI error code `responses_item_routing_unavailable`, Codex wire values such as `responses_compaction_v2`, `openresponses/openresponses` references, and every Anthropic/OpenAI/Google payload field. - Flag ids in `OPTIONAL_FLAG_IDS`; operators have them in stored config. - Database object names and persisted enum values: the `responses_items` and `responses_snapshots` tables, `api_keys.responses_retention_seconds`, their indices and triggers, the `expiration_sweeps.domain` and `spilled_files.owner_kind` values, the CHECK-constrained `source_api` and `target_api` values behind `ChatTargetApi`, and the `ModelEndpoints` keys stored in `upstreams.config_json`. Renaming a large table is not viable on the deployment target, so the schema keeps its current spelling and the code that reads it keeps the schema's words while the surrounding names become canonical. - `responses` and `Messages` where they are the plural of `response` and `message` rather than the protocol.
The mechanical pass could not decide these from the token alone, because the same spelling is a persisted value elsewhere: - `PUBLIC_DATA_PLANE_ROUTES` route ids and the API-docs entries that name them. - `PlaygroundApi` members, the records keyed by them, and their i18n labels. - `CollectKind` members in the request inspector. - The namespace aliases the OpenAI Responses event builder is imported under. - The affinity carrier domains. These are AEAD associated data on values a client echoes back, so a rename costs one turn of affinity on the deploy that lands it and then self-heals: `unwrap` treats a domain mismatch as a foreign value rather than failing the request. `ModelEndpoints` keys, `ChatTargetApi` members and the expiration-sweep domain keep their old spelling: all three are stored, and two of them are pinned by a CHECK constraint no migration is allowed to relax.
`openai-*`, `anthropic-*` and `gemini-generate-content` sort differently from the names they replaced, so `eslint --fix` moved 145 import lines. No other change.
`ModelEndpoints` was the last place in the tree that abbreviated a protocol: `chatCompletions`, `responses` and `messages` keyed the map a model declares its served endpoints with. `ChatTargetApi` mirrored those spellings, and both now use the canonical `openaiChatCompletions` / `openaiResponses` / `anthropicMessages`. The map is authoritative operator configuration, stored per model in `upstreams.config_json.models[].endpoints` and once more at `config_json.endpoints` for custom upstreams, so `0083` rewrites the keys in place. It follows `0024_structured_model_endpoints.sql`, which performed the same class of rewrite on the same column when the path array became this map. `MODEL_CATALOG_REVISION` goes to 6 so the derived catalog cache, which is not authoritative, goes cold instead of being migrated. The migration test asserts the capability rather than the JSON: it seeds a row in the old shape, runs the migrations, and reads the endpoints back through the production repository codec and the custom provider's config parser. Its negative control runs the same read with `0083` skipped and requires it to fail, so the test cannot pass for a reason other than the migration. `ChatTargetApi` is safe to rename because nothing stores it. The CHECK-constrained `source_api` and `target_api` columns behind it were dropped when `0050_perf_ttft_tpot.sql` rebuilt the performance tables around `operation`, so no live column carries those values.
The pass rewrote three classes of value it had no business renaming, each caught by a test rather than by the typechecker: - The legacy `pathOverrides` key `chat_completions` that `0044_custom_pathoverrides_slash_keys.sql` migrates from; it is stored data the migration matches on by name. - The migration filename `0065_responses_state.sql`, split because the token scanner cannot begin a match on a digit and so resumed inside the name. - Prose already qualified once, which the namespace-alias rename qualified again into "OpenAI OpenAIResponses". The control-plane `modelEndpointsSchema` also still validated the old endpoint keys, so every azure and custom upstream POST was a 400; test expectations that read `Object.keys(endpoints).sort()` move with the new sort position.
The dashboard labelled the protocols "Responses", "Messages" and "Chat Completions" in some places and "OpenAI Responses" / "Anthropic Messages" in others, in the same locale file. An abbreviation only reads if you already know which vendor owns which wire shape, and the operator looking at these labels is exactly the person who does not. Both locales move together so they stay structurally equivalent. The DeepSeek, Qwen and Kimi flag descriptions keep "Chat Completions API" — there it names those vendors' own API, not OpenAI's. Four comments that still abbreviated mid-sentence beside canonical names are rewritten to the prose form as well.
The gallery's sample rows named the protocol both ways in one control. It is dev-only copy, so this is consistency rather than behavior.
The backfill tool's fixtures build an `upstreams` row by hand, so they carry the endpoint keys directly.
An export taken before the protocols were spelled out in full carries the old endpoint keys, and restoring one has to say so. It does — the runtime validator names the key it did not understand — but nothing asserted it, and the property is easy to lose: the request schema alone would strip an unknown key silently, leaving a model that imports "successfully" and then serves nothing. Verified against the mechanism rather than the shape: disabling the endpoint check in `packages/provider/src/model-config.ts` fails this row and only this row.
Three protocol names read in full everywhere and the fourth did not: comments and test names still said "Gemini" and "Chat" where the sentence beside them already said "OpenAI Responses" and "Anthropic Messages". A peer list reading "Anthropic Messages/Gemini/Chat" asks the reader to expand two of its three entries. Prose about the protocol we serve and translate now names it in full — its payloads, fields, streams, sources, targets, routes, interceptors and envelopes. Prose about the vendor does not, because there the abbreviation is the correct name: - Google's model families (`Gemini 3 Flash`), its model catalog (`/v1beta/models`, "Gemini surfaces chat-kind models only"), its OpenAI-compat endpoint, its OAuth, its docs, and its `model:action` URL form. - `Copilot Chat`, which is the VSCode extension's product name. - `Chat` as the `ModelKind` in the dashboard's kind picker, and "Chat protocols" where it means the chat family rather than one member of it. - The DeepSeek, Qwen and Kimi "Chat Completions API" descriptions, which name those vendors' own compatible endpoints — prefixing OpenAI there would assert something false. Hyphenated compounds take the hyphenated form (`Gemini-generateContent-via-OpenAI-Chat-Completions`), and articles move with the vowel the qualified names now start with. Also lands the data-transfer import test for a pre-rename backup, which failed to compile: `UpstreamRecord.config` is a provider union rather than an open record, so the export's serialized config is what the spread has to come from.
The sweep missed `packages/protocols`, whose test names and field comments still said "Chat" for the OpenAI Chat Completions protocol, and one comment in `common/models.ts` that says Gemini generateContent has no `ModelEndpoints` key of its own — a statement about the protocol, not about Google. `ChatModelInfo`'s "Chat capability metadata" stays: that describes the chat kind, which all four protocols share.
Menci
marked this pull request as draft
August 18, 2026 18:36
Every stage already had a logger whose lines land in that stage's record when a
dump is open, but the recorded event carried an ad-hoc `{ level, message,
fields }` and stuffed the stage's name into `fields.stage`. A reader holding the
log lines alone could not say which stage wrote one without the rest of the run
to resolve `stageId` against.
The event now carries a `LogEntry` — level, context, message, fields — which is
the shape `@guiiai/logg` writes, so a sink that speaks logg takes one unchanged.
`context` is the stage's name, where logg puts the same thing. The type is
declared here rather than imported: a foundation package states the shape it
needs and never the implementation.
The ordering a reader depends on holds by construction, and is now asserted: a
line names its stage by id, entering a stage is what mints that id, and the
entering event goes out before the stage body can log. So no line can arrive
before the event that says which stage it belongs to.
The four chat protocols already carry their vendor. The five non-chat ones did
not, and two of them collide with ordinary English: `completions` is also the
plural noun, and worse, a substring of `/v1/chat/completions`, so a symbol named
`completions` could be either endpoint; `embeddings` is what an embedding
response is full of.
completions -> openaiCompletions / OpenAICompletions / openai-completions
embeddings -> openaiEmbeddings / OpenAIEmbeddings / openai-embeddings
imagesGenerations -> openaiImagesGenerations / OpenAIImagesGenerations
imagesEdits -> openaiImagesEdits / OpenAIImagesEdits
audioTranscriptions -> openaiAudioTranscriptions / OpenAIAudioTranscriptions
`rerank` keeps its spelling. It is a `ModelKind` that fans out to six vendor
wire protocols (`cohere-v1`, `cohere-v2`, `jina-v1`, `voyage-v1`, ...), so no
vendor owns it and a vendor prefix would assert something false.
The two module directories that group these endpoints, `images` and `audio`,
take the vendor as well. They are not model kinds — those are `image` and
`transcription` — but OpenAI's own API families, and leaving them bare would
have made them the only unqualified protocol directories standing beside
`openai-completions` and `openai-embeddings`. `chat/` stays: `chat` is the kind.
`ModelEndpoints` keys are authoritative operator configuration, so
`0083_canonical_endpoint_keys.sql` gains a `WHEN` branch per key rather than a
second migration over the same column — it has not shipped, and it already
rewrites the three chat keys in that map. Its test now seeds two rows, one per
family, because the validator throws on the first key it does not recognize and
a single row could only ever witness one of them; the negative control names a
chat key and a media key. `rerank` is seeded unrenamed as a guard.
`MODEL_CATALOG_REVISION` is already 6 on this branch and covers this too.
Deliberately left alone:
- Public URL paths and path fragments (`/v1/completions`, `/v1/embeddings`,
`/v1/images/{generations,edits}`, `/v1/audio/transcriptions` and their
unversioned forms), including `PassthroughServeApiName`, whose members are
public URL fragments by design, `ENDPOINT_PATHS` values, `pathOverrides`
keys, and the same set restated in nginx, Wrangler and the Vite proxy.
- `/v1/chat/completions` and every `chat/completions` spelling: that is the
other protocol's path.
- Copilot wire values: `supported_endpoints` entries, `capabilities.type`, and
the `completions` quota id with its `x-quota-snapshot-completions` header.
- Persisted, CHECK-constrained `operation` values (`text_completion`,
`embeddings`, `image_generation`, `image_edit`, `audio_transcription`) and
the `audio-transcription:` log tag that mirrors one of them.
- Shipped migrations, including `0016_custom_drop_embeddings_endpoint.sql` and
the legacy underscore `pathOverrides` keys `0044` matches on by name.
- `image_generation` and everything named after it: that is the OpenAI
Responses builtin tool, not the images endpoints.
- Ordinary English — "text completions", "alternative completions",
"fast-path completions", "embedding catalogs", `return_embeddings` (Jina's
rerank field), and the `text-embedding-*` model-id tokens the custom
provider's kind heuristic matches on.
The gallery's multiselect sample also gets its option values back: the earlier
mechanical pass rewrote `defaultSelectedOptions` without rewriting the `value`
attributes it points at, so the sample rendered with no selection.
…d one
The last abbreviations in the tree. `messages-web-search-shim` becomes
`anthropic-messages-web-search-shim`, and the three `responses-*` shims become
`openai-responses-*`. The other ten ids name a vendor dialect, a rewrite or a
usage convention rather than a protocol, and keep their spellings — the rule is
about protocol names, not about lengthening every identifier.
The dashboard already labelled all four in full ("OpenAI Responses Web Search
Shim"), so this is the id catching up to the copy an operator actually reads.
A flag id is authoritative operator configuration in two columns — per-upstream
`flag_overrides` and per-model `config_json.models[].flagOverrides` — so `0084`
rewrites the keys in place, the way `0083` did for the endpoint capability map.
A stale key fails in two directions and neither is loud: the read path takes any
string, so the toggle survives and silently stops matching, and the write path
rejects unknown ids, so the next dashboard save of that upstream fails on a key
the operator never typed.
One difference from `0083` is worth naming, because getting it wrong is
production-breaking rather than cosmetic. An override's value is a boolean, and
`json_each` hands JSON `true` back as the integer 1, so rebuilding the object
with a bare `json(value)` writes `{"flag": 1}` — which `normalizeFlagOverrides`
throws on, taking every upstream with overrides down at load. The value is
rebuilt from `json_each.type` instead. Verified by mutation: with the naive form
the migration test fails on exactly that assertion.
`target_api` is this gateway's own debug lane, not a vendor field — it carries a `ChatTargetApi`, so its value is `anthropicMessages` like every other statement of that value. Two Copilot boundary fixtures still said `messages`. Found by sweeping the field's values rather than the identifier: a string literal in a test is exactly where a rename sweep aimed at declarations passes over.
…ses it `createEncoder` is what `encodeRun` and `createRunEncoder` are built from, and it was in the package barrel with no caller anywhere — the one export of the fifty-two that names a seam rather than a capability. It is now module-private, which is what it always was in fact. Found by auditing every barrel export for consumers rather than by reading the list: the other ten values in that block each have one.
`$audit-copilot-workarounds` opens by naming the three boundary registries as
`{openai-chat-completions,messages,responses}`. Two of those directories were
renamed with everything else and the skill was not, so the audit it describes
starts by reading two paths that are not there.
Found by checking every repository path the skills name against the tree rather
than by grepping the skills for the old words — the other ten resolve.
`0083` rewrote the endpoint capability map and `0084` rewrote the flag ids. Both are the same change — protocol names spelled in full, reaching the operator configuration that stores them — and both UPDATE `upstreams`, so they are one migration over one table rather than two. The four statements stay as they were and run in order; nothing about what is rewritten changes. Both migration tests keep their own fixtures and assertions and now name the merged file. Verified by mutation: replacing the type-aware flag value rebuild with a bare `json(flag.value)` still fails the boolean assertion, so the half that would break every upstream with overrides is still instrumented after the merge.
`StatefulOpenAIResponsesStore` reads as though the protocol were called "Stateful OpenAI Responses". The protocol is `OpenAIResponses` and `Stateful` is this gateway's word for what the store does, so the qualifier goes after the name it qualifies — which is what `wrapOpenAIResponsesStatefulOutput` was already doing beside it. Seven names move: the store, its backing, its lookup, and the layered, repo and memory implementations. The implementation qualifiers stay in front of the whole phrase, where they read as what they are. Two alternatives were considered and both lose the rule this PR exists for. "OpenAI Stateful Responses" splits `OpenAIResponses` around a word that is not part of it; "Stateful Responses" abbreviates the protocol to `Responses`. Only qualifier-after keeps the canonical spelling intact and unbroken.
#472 landed on main and reshaped the same provider surfaces this branch renames: an ingress header rule became one value rather than one name, so a call takes `HttpHeaderLines` positionally instead of a `Headers` inside `opts`. Twelve conflicts, all the same shape — main's new signature, this branch's names: - `provider/anthropic-messages.ts` and its test: the filter-and-append body over header lines, under `headersForAnthropicMessagesCall`. - azure, copilot, custom, ollama and claude-code providers: `[...opts.headers]` threaded as its own argument at every call. - the two locale files and the dashboard test: main's `duplicatePassthrough` key and its wording, with `anthropicMessagesOwned` spelled in full. Three files main *added* merged without conflict and needed the rename applied by hand — git cannot see that a new file spells a protocol the old way. One substitution over-applied while doing it: `messages:` is the Anthropic Messages wire field as well as the old endpoint-map key, and the payloads in those tests carry the field. The wire form is restored; only the capability-map key is renamed.
#493 landed on main: an upstream usage counter reported as `null` now reads as an absent counter rather than erasing what an earlier event stated, and the usage types split into `UsageDelta` (nullable, what the wire carries) and `Usage` (the totals). Eight conflicts, all the rename against that reshaping. `protocols/anthropic-messages/usage.ts` is taken from main and renamed rather than patched — main rewrote most of it, and reconciling hunk by hunk would have produced something neither side wrote. The clone of `iterations` stays here: it is #476 that deletes it, not this PR, and taking it out during a merge would pull that PR's concern down into this one. Two test files gained a test from main that arrived without a conflict, because main appended where this branch had not edited. Neither the merge nor the rename can see those: a file that already existed gets no rename pass, and git has no reason to flag text it merged cleanly. Both are renamed by hand here, which is the second time this class has come up in two merges.
`26-dump-format.md` shaped the encoding so every event can be emitted the moment it happens and then said live push was designed for and not built. This is that piece: the transient carrier for one run's bytes while it is being produced, with the durable artifact remaining the writer's own separate path. It carries opaque bytes rather than records, which is what leaves an implementation free to split and merge at any byte position and makes a stream's bytes identical to the artifact's. `append` takes the offset it is writing at, because an append that failed cannot be told apart from one that landed and lost its acknowledgement — resolving by position is what makes the writer's retry the same bytes in the same places instead of a duplicated splice. `read` takes an offset for the same reason in the other direction: deployments terminate every reader unconditionally, and the offset lives in the caller, so there is no server-side cursor and no seam between backlog and live tail. The HTTP binding ships beside the interface as a matched pair, because a live body has no length and the transport cannot say whether it finished. Measured against both the Cloudflare edge and local workerd: a response body whose stream errors arrives at the client as a *clean end* — `done: true`, `curl` exits 0. So the framing carries the signal itself: each chunk behind a big-endian uint32 length, a zero length terminating, and a body that stops early detected either by the missing terminator or by ending part-way through a declared frame. A length prefix rather than a marker in the content, so nothing needs escaping and the content stays opaque. The Durable Object keeps byte segments keyed by their start offset, resolves an append inside one synchronous turn against `ctx.storage.sql`, serves readers over hibernatable sockets with each reader's offset in `serializeAttachment`, and reclaims on one rule — ended and idle. `deleteAlarm()` is called explicitly beside `deleteAll()`: on this repository's compatibility date the latter leaves a pending alarm that fires and re-creates storage, resurrecting an object that was supposed to have ceased to exist. The Node implementation is in-process and says so. §6 of the spec calls for Redis Streams; this target has no other cross-instance component — its channel broker is an in-process `EventTarget` — so a Redis-backed stream would be the only piece able to serve a read from a second instance while dump notifications still could not cross one. Whether that target becomes multi-instance is a design decision rather than something to settle by adding a required service here, and the interface is what makes the swap additive once it is settled. Nothing consumes this yet, which is the same shape `packages/pipeline` has in this PR: the contract lands with the core, and the writer that tees into it lands with the dump writer.
Not a rename change, and it rides on this PR because this is the one of the four targeting `main` and `verify` is the gate all four are judged by. Move it if you would rather it stood alone. The test spawns `tsx` three times, each paying its own loader startup, against vitest's default 5 s. It passes alone and inside a quiet suite and times out under a full `verify` — a stopwatch failing rather than the CLI. Sixty seconds is far past what the work takes and still catches a hang.
Menci
marked this pull request as ready for review
August 20, 2026 04:40
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The data plane is moving from an onion of nested interceptors to a list of stages over one
immutable record of facts. This is the first of three stacked Pull Requests and it adds the
package everything else will compose. There is no business logic in it — nothing here knows
what a model is — so nothing in the running gateway changes yet.
What a stage is
A stage declares where control may go, and it may declare more than one place. There is no
name for a kind of stage; the fields are the declaration.
throughandintoare mutually exclusive because of what they mean, so five shapes arelegal and
defineStage's five overloads admit exactly those — the overload fixes whatnextaccepts, rather than a marker boolean or a conditional type. A stage that declared no way down
is handed no continuation at all.
transformis sugar in theexecuteslot:openfor state spanning both directions, then onefunction per direction, either of which may be omitted. Omitting one is the common case —
today's interceptors call the continuation 107 times across 68 files and 50 of those are a bare
return run();.Declarations are checked, never applied
After a stage hands on — in either direction, and on the answering path too — the runner
asserts that every key it declared
providesis there and every key it declaredconsumes,and did not also declare providing, is gone. Nothing is removed on a stage's behalf: a
declaration that acts cannot also be checked, and checking is the whole of what a declaration
is for.
A key in both
consumesandprovidesis a modify, and that one rule covers bothdirections: going down it is a translation taking the source and putting the target at another
key; coming up it is a fork taking ownership of every branch's disposable and handing one of
them onward.
What assembly catches that types cannot
compose(name, stages)reasons over declarations, which are strings — so a pipeline's interioris not part of its type and a stage written against one fact space drops into a pipeline over
another with no variance question to lose.
The first is the load-bearing one: a key an earlier stage consumed cannot be needed below it,
so a translated request cannot re-enter its own chain, and assembly says so rather than a
runtime guard. The entry contract is derived from the stages and asserted when a request
arrives, at
runand at every handoff.Handover
Entering the record is a move.
Object.isFrozencannot be the gate, and both halves of thereason are load-bearing: freezing a typed array that has elements throws, so the walk skips
them and a skipped value can never satisfy
isFrozen; and a shallow builtin freeze satisfiesisFrozenwhile its children stay mutable. So aWeakSetof handed-over roots is the gate andObject.freezeis what makes a later write throw.A handed record is frozen in place rather than copied, so a stage that hands on what it
received hands on the same object — which is what the dump's folding reads.
The dump
Six events, carrying states and never differences, so a reader derives every change and the run
stores none. An object id is taken at first sight and before recursing, which is what makes
a cycle terminate on its second visit. A key that already begins with
$is written with onemore, because a tool's parameters are JSON Schema and
$schema,$defsand$refreally doarrive. Large strings are shared by value, which is the only handle left when a stage
deep-clones a payload. A secret is stored as
{length, redacted, hash}and never as itself.Two things worth reviewing closely
The run context is threaded, not ambient. A module-level variable saved and restored around
an
awaitinterleaves two concurrent runs — one dump lost, the other holding both, onestageIdissued twice — and a gateway serves concurrent requests by definition.The disposal sweep runs in
finally. A stage that throws must not abandon an open upstreambody: an aborted connection cannot be reused and leaves its billing unsettled.
Tests
54 tests plus two compile-time assertions (a
throughstage'snexttakes one argument; aprovider's own key is unreachable from a gateway-only pipeline). Among them the one property
nothing else in the system guards:
A stage rewritten to rebuild unconditionally drops that to zero while every other test passes.
What comes next
PR 2 migrates the six non-chat families, and is where
passthrough-serve.tsis deleted — thoseendpoints forward a body they have not parsed, and the architecture has no such concept. PR 3
migrates chat, which does not divide further: translation couples the four protocols, and the
two shims are elements of the interceptor array rather than things beside it.