feat(chat): migrate the chat protocols onto the pipeline - #476
Draft
Menci wants to merge 125 commits into
Draft
Conversation
…ols share The first commit of the chat migration. Four source protocols, ten translation directions between them, and one array of stages every chain runs — this is the space they run over and the first three entries of that array. The key layout says two things the context object could not. Each protocol has its own request and response key, so a translation consumes the source and provides the target — and `compose` then refuses a chain that could re-enter its own protocol, which is `ctx.targetApi === <self>` made structural. And `ingress.*` is what the client asked for, so `wantsStream` is four disjoint keys rather than one: a translated request must not inherit the target protocol's answer to a question the client never asked. That guard turns out to be two things wearing one expression. "Do not run on a re-entered request" is now structural and no stage asks it. "Which protocol was chosen" stays a live question, because a vendor normalizer genuinely needs to know — it becomes a declared fact read through `needs`, which is visibility rather than removal. So the five guards are not deleted by the migration; they change what they read. The rewrites themselves are transcribed, not redesigned: they already worked. What is new is that a stage returns the record it hands on instead of assigning to `ctx.payload`. Today's code already writes its map conditionally, which is the convention that keeps a 49-message conversation costing three objects, so the tests assert identity rather than equality — a payload no flag touched comes back as the same object, and a flag that fires but changes nothing changes nothing. Gemini is source-only. Nothing in `packages/translate/src/` targets it, so by the ruling it contributes one pipeline rather than two: a source role and no target role.
The chat interceptors were written against route.candidate, the key the selector split replaced. They reach for the live candidate only to read its enabled flags, and the selector carries those as data — so they now read what travels in the record and the tests need no candidate stub at all. The shared chat stages arrive with them: resolveChatCandidates enumerates, narrows to the wires a source protocol can reach, and orders by affinity, which can refuse outright when a turn's own state requires two upstreams at once.
The reference chat chain, and what it establishes for the three families that follow: the edge decides whether the client sees the frames or the one object they add up to, which is where the stream-to-value collection belongs — the upstream speaks SSE whatever the client asked for, so folding is the edge's own work rather than a second reading. The shared chat resolver narrows to the wires a source protocol can reach and orders what is left by affinity, which can refuse a turn whose carried state needs two upstreams at once. The interceptors run as ordinary stages between the fork and the ending. Metering reads the upstream's own usage off its own events as they pass, so a streaming turn settles from the promise its last chunk resolves rather than in the stage. The route still runs the existing surface.
The chain composed and nothing had run it. Doing so found the edge handing its SSE view up without move(), which the handover gate refuses — a stream the run never took ownership of would have escaped the record's own guarantee. The five cases are the ones only running can state: frames written out when the client asked to stream, the same frames folded into one object when it did not, the upstream's forwardable headers kept and content-length dropped, a refusal carrying the upstream's own status and words, and a dial nobody answered failing over to the next candidate rather than ending the run.
The chain declared the services affinity needs and then read the client's own payload straight out of the record, so the per-candidate rewrite was lost — a turn carrying state for one upstream would have been sent to another unchanged. The ending now asks for what affinity materialized, and records the candidate that answered so a follow-up turn comes back to it.
A chat run's three extra services are all the live half of resolution: the candidate, and the payload affinity materializes for it. Neither can enter the record — move() would freeze the provider's own models cache, and materializing is per candidate — so the resolver keeps them and the stage that dials asks back by selector, which is the shape the shared services already use. The seam splits rather than being wrapped. Building the chat context over an already-built one would have minted a second dump accumulator, and the body can only be taken once: takeRequestBody empties what it is given, so the second call would hand the dump an empty buffer. What is shared is now the options and the services, and each family builds the one context its run has.
Gemini contributes one pipeline rather than two, because nothing translates into it, and it has no wire of its own: the ending translates the turn out to Chat Completions, dials, and translates the answer back, metering on the dialect the upstream actually spoke. Only that first wire is built — the Messages and Responses ones, `:countTokens`, this family's interceptors and the affinity egress are named in the header as absent rather than left implied. Running the chain settled three things it had wrong. The stream handed to the client is moved into the record like every other fact; the turn that is translated is the one affinity materialized for the candidate, and the candidate that answered is marked for the next turn; and the metered reading is converted to billing metrics rather than cast to them, since a cast leaves a TokenUsage where UsageQuantities is declared and the usage row it writes then carries no metric at all.
A translation is two declarations: it consumes the source protocol's request key and provides the target's, and coming back it consumes the target's response key and provides the source's. Saying it that way retires the runtime test. The source key is gone below the handoff, so a stage needing it cannot be placed there and compose says so at assembly — which is ctx.targetApi === <self> deleted and made structural. And nothing below knows a translation happened: the target chain sees its own protocol's keys and only those, so one chain serves a native request and a translated one. A refusal is handed to the pair as the upstream's own bytes, with the headers that actually came back rather than a synthesized set — that is what lets a context-window error become the shape Claude Code reads for auto-compaction. An answer that arrived as a value has no pair to map it, so the handoff fails rather than handing the client another protocol's object under its own key.
…oken categories What an upstream reports is per token category and what is billed is per metric name; the two are different shapes and a cast between them typechecks. The usage row was therefore written with no metrics at all — the recorder looked for input_tokens and found input. Converting through the same function every other caller uses fixes it, and the chain now asserts the quantities rather than only that a row was written.
The chain rendered failures through the shared envelope, which is the OpenAI families' shape: a Gemini SDK reads error.status — the Google-RPC name — and would have found it undefined, and error.code carries the HTTP status the generic shape has no field for. The mapping the replaced surface used already existed next door; it is now a renderer the edge calls, so the two entry points that build this envelope agree on one definition of it.
/v1/messages served through the chain Chat Completions established: an edge that writes Anthropic's own named SSE events when the client asked to stream and reassembles them into one message when it did not, settlement above the fork, affinity-ordered candidates, and a native ending that dials callMessages. Three things the replaced serve/attempt/respond surface said are said here rather than left implied. The turn that goes out is the one affinity materialized for the candidate, and the candidate that answered is marked for the next turn. The beta flags keep their typed transport path — read off the inbound headers and dropped from what the provider is handed, so no header allowlist can admit them and no other source protocol can leak them in. And a dial that never connected is a failure value the fork moves past rather than a throw that ends the run. Only the native wire is built. The translated ones, count_tokens and this family's own interceptors are named in the header as absent rather than left implied. What the header cannot say is that a pre-upstream refusal now renders in the shared error envelope rather than Anthropic's, since renderErrorEnvelope is what the edge writes; an upstream that refused in its own words is still handed on in them.
Builds `/v1/responses`' own wire as a pipeline, on the chain Chat Completions established: the edge that writes SSE frames or the one response object, settlement above the fork, candidate narrowing ordered by affinity, failover, and an ending that dials `callResponses`. Two things this family states that its siblings do not. A provider answers with the branch it actually ran, and one of them is not a stream — a compaction is a single envelope that carries its own counts — so it rides at the response key's value arm rather than at a key of its own or lowered into synthesized frames. And the client's stream is terminated by `[DONE]` here rather than by whatever the upstream's stream ended on, which is what the transport reads to know the turn is over; the ending stops reading at the turn's terminal event, and a stream that never states one ends the run with the sentence the replaced surface used. Scope is one wire and one transport. The WebSocket entry, `/v1/responses/compact`, the two translated wires, this family's interceptors and the stored-items membrane are each a step of their own and are named in the file header rather than implied by absence. Tests run the chain: frames written out, frames folded into one response, an envelope served as itself, a refusal answered in the upstream's own status and words, and a dial nobody answered failed over to the next candidate.
… the upstream said Three families rendered a gateway-synthesized refusal through the OpenAI envelope. A Gemini client reads error.status — the Google-RPC name — and a Messages client reads a top-level type and a request_id; neither shape carries those, so both would have found the fields missing. The rule the envelope already encoded stays: an upstream that refused in its own words is handed on in them, because those words are already what its client reads. Only a refusal the gateway itself produced has no body to forward, and that is the one case where the *client's* protocol decides the shape. That distinction is now named once and each protocol supplies its own envelope. A turn whose carried state needs two upstreams at once answers 400, as every family did before the shared stage invented a 409 for it. The three chat chains assemble alongside the six that came before them.
The seam wrote an SSE comment as keepalive, which is invisible to any client by design — but Anthropic defines a ping event and its clients read one, so the two are not the same wire byte. A family that has its own idle frame now names it alongside the frames it is answering with, and Messages exports the one it writes so the route that serves that chain cannot forget it.
The interceptors rewrite the request as a fact and the ending read the payload back from the resolver, so all three stages mutated a copy nobody sent. Deleting every one of them left the suite green. Affinity's per-candidate payload now enters the record below the fork, where there is a payload to speak of — each candidate has its own, and re-running the suffix is what produces the next. Everything between that stage and the dial rewrites a fact, and the ending sends the fact. Four comments claimed the fact space deletes the ctx.targetApi guards. The record retracts that: the claim rested on splitting the interceptor array by role, which was rejected — 「这里没有 source 和 target 之分！这是同一个数组！」 — and with one array the guards remain. What changed is that what they test is declared rather than ambient, which is visibility, not deletion.
…teway made A gateway-synthesized refusal reached the client as type: 'api_error' — which says the gateway broke rather than that the request did — and Responses lost the param and code that name which field was at fault and why routing failed. A protocol's envelope carries more than a status and a sentence, and only whoever refused knows those, so the refusal now carries what it would write and the edge renders it. The upstream's own body still wins wherever there is one. Which refusal it was travels with it: a turn that cannot be routed, a model no wire reaches, and a model no upstream has are three different things to say.
Time to first token was written in one place no stage reaches, so every chat run took the neutral branch and recorded no TTFT at all. It is measured where the token is — the only place that can tell a generated frame from the envelope around it. A rate can depend on the service tier and on how much input there was. Both are selector coordinates rather than quantities, and dropping them left the pricer with an empty selector, so neither axis could ever apply. The reading now goes through the same measurement the other families use and carries both.
…ng reads The shared stages' header described a dispatch stage that picks a wire. No such stage was built — every chain dials its own protocol — so what it now describes is the payload entering the record below the fork, and what will decide a wire once the translated ones land. The selector is one statement about a candidate and was written twice; the chat resolver uses the definition the shared one already had. ChatNarrowing.source was declared and never read. A handoff declared it needs the upstream's headers and then supplied [] when they were absent, which would have hidden the assembly error that absence is.
The ending read its payload back from the resolver while everything between the fork and the dial rewrites the request as a fact, so a rewrite would have gone nowhere. Affinity's per-candidate payload now enters the record below the fork, where there is one payload to speak of, and the ending sends the fact. Time to first token was written in one place no stage on this chain reaches, so every Messages run took the neutral branch and recorded no TTFT at all. It is measured where the token is — the only place that can tell a generated frame from the envelope around it. A rate can depend on the service tier and on how much input there was. Both are selector coordinates rather than quantities, and dropping them left the pricer with an empty selector, so neither axis could ever apply. The reading now goes through the same measurement the other families use and carries both. A stream that ran out before message_stop was served as a whole answer, and frames an upstream wrote after it went out to the client. The meter now stops at the terminal event and fails a stream that never reached one, which is what the protocol's own reassembly already did for the folded shape alone. ChatNarrowing.source goes with the key it was declared under.
…e its first token The ending read its payload back from the resolver while everything between the fork and the dial rewrites the request as a fact, so a rewrite would have gone nowhere. Affinity's per-candidate payload now enters the record below the fork, where there is one payload to speak of, and the ending sends the fact. Time to first token was written in one place no stage on this chain reaches, so every Responses run took the neutral branch and recorded no TTFT at all. It is measured where the token is — the only place that can tell a generated frame from the envelope around it. The two readings this chain already had right — pricing facts beside the quantities, and a stream that fails rather than answering short — were carried by no test, which is how the other chains came to disagree with them. They have one each now. ChatNarrowing.source goes with the key it was declared under.
…tates The ending read its turn back from the resolver while everything between the fork and the dial rewrites the request as a fact, so a rewrite would have gone nowhere. Affinity's per-candidate turn now enters the record below the fork, where there is one turn to speak of, and the ending translates the fact. Time to first token was written in one place no stage on this chain reaches, so every Gemini run took the neutral branch and recorded no TTFT at all. It is measured where the token is, on the dialect the upstream spoke — the same one the usage beside it is read on. A rate can depend on the service tier and on how much input there was. Both are selector coordinates rather than quantities, and dropping them left the pricer with an empty selector, so neither axis could ever apply. The reading now goes through the same measurement the other families use and carries both. A wire that closed cleanly had not thereby finished the turn: with no finish reason on any choice, nothing that came out of the translation said the answer was over, and a client streaming was served those frames as a whole answer. The folded shape already refused them, through the protocol's own reassembly; both shapes now do, because the ending stops at the turn's terminal frame and fails a stream that ran out before one. ChatNarrowing.source goes with the key it was declared under.
The meter read to exhaustion, so an upstream that dropped mid-turn was served as a whole answer and anything it sent after its terminator was forwarded. The streaming path is where this has to be caught: nothing folds those frames, so the collector's own check never sees them. The terminator is written out before the read stops, because it is what a client reads as the end. The first test written for this passed with the guard deleted — it drove the non-streaming path, where the collector throws the same sentence for its own reasons. It now drains the stream, and both halves of the guard are covered.
# Conflicts: # packages/gateway/src/data-plane/pipeline/serve.ts
The chain existed and was tested against stub providers, but nothing reached it: the route still built a chat context and ran the onion, so every property the chain states was stated only under test. The handler is now the prologue and epilogue the other migrated families use — read the ingress, open the chat prologue against a scratchpad store, hand the payload over, and write what the run answered with. Two decisions stay at the entry because only it can make them, and both are read before any stage can rewrite what they are read from: whether the client asked to stream, which the run has to be opened knowing, and whether it asked to be shown the usage chunk metering asks the upstream for either way. A body that is not JSON never enters a pipeline at all — there is no model to resolve and no attempt to make — so it is answered in this protocol's own 400 envelope. Wiring is what states what the chain does not carry yet, and two of them are written down in failing tests rather than papered over. The affinity egress never runs, so a turn hands back no `reasoning_opaque` and the next one cannot be pinned to the upstream that served this one; `http_test.ts` says so twice and is left saying it. Six of the nine interceptors are not stages — the usage-chunk request, both usage normalizers and the three vendor normalizers — a refusal that reached no upstream leaves the streamed-usage key unwritten, and the meter reports what was billed without reporting whether the stream reached its terminator. Each of those is the chain's own to close.
The chain existed and was tested against stub providers, but nothing reached it. The generate entry is now the prologue and epilogue the other migrated families use; `/v1/messages/count_tokens` stays where it is, because it is a second operation over this protocol rather than another wire under its chain. A stream is handed to the seam with this protocol's own idle frame. Anthropic defines a `ping` event and its clients read one, so an idle connection is held open with that rather than with an SSE comment no client sees — the chain names the frame and the route passes it on. A body that is not JSON is answered in Anthropic's own envelope, and a body carrying `anthropic_beta` or `betas` is refused as before, both now recorded as the gateway refusals they are. Wiring states what the chain does not carry yet. None of this family's five interceptors is a stage, and the first of them is the one `http_test.ts` catches: Claude Code's one-token model probe is answered by dialling an upstream instead of by the gateway. The web-search shim, the billing-attribution scrub, the reasoning and role rewrites, and the affinity egress are absent with it; so is the mid-stream `error` event `respond.ts` wrote into a client's own stream. A refusal that reached no upstream leaves the streamed-usage key unwritten, and the meter reports what was billed without reporting whether the stream reached its terminator.
The chain existed and was tested against stub providers, but nothing reached it. Both generate actions are now the prologue and epilogue the other migrated families use; `:countTokens` stays where it is, because it is a second operation over this protocol rather than another wire under its chain. Gemini carries the model in the path rather than the body, so the id the run resolves against is the one the route already split off the action segment, and whether the turn streams is the action itself rather than a field. A body that is not JSON is answered in the Google-RPC envelope this protocol's clients read. Wiring states what the chain does not carry yet, and `http_test.ts` catches the first: the chain has one wire, so a candidate reachable only over Messages or Responses is dialled on Chat Completions anyway and the turn fails. None of the four Gemini interceptors is a stage, the thought-signature rewrite and the affinity egress on the way out are absent with them, and a stream that ends without a terminal event ends the run as a throw rather than as the Google-RPC envelope `respond.ts` wrote into the client's own stream. A refusal that reached no upstream leaves the streamed-usage key unwritten, and the meter reports what was billed without reporting whether the stream reached its terminator.
…sal settle at all Two defects the non-chat families had already been through, both found by wiring the routes. Every chat chain handed its usage up as a bare list, so each handler adapted it with failed: false — a stream that stopped before its terminator was recorded as a turn that produced what it said it would. The meters now report both together, which is what the seam has taken since the non-chat families moved, and the four adapters delete. A refusal never provided the streamed-usage key at all, so settlement's still-reading test read undefined and deferred a turn that had no stream to wait for: the run wrote no performance row. Refusing now says there is nothing still to read, which is the same statement an ending makes when it did not stream.
…ages Chat Completions had three of its interceptors as stages; the other three families had none, so their chains ran nothing between the fork and the ending. Gemini's thought suppression was among the missing, which is a client that never opted in being shown thought parts. Thirteen stages, one per (interceptor, protocol) pair because the payload types differ, and one rule each: the role fold, the flag reading and the key removal are written once, and a protocol contributes only the walk over its own items. Two of them speak about the response — Gemini's thought suppression and the Responses cache-token fold — and say so as a response-direction declaration, carrying what they read on the way down in the stage's own closure, because a response-side `needs` can only name what the ending provides. What the interceptor form did by mutation is a rewrite now: the record is frozen, so Gemini's three strippers cannot delete a field in place. Every rule writes conditionally, so a turn it does not touch comes back by identity and the layer costs what it actually changed. A `ctx.targetApi` guard is gone wherever array position now says the same thing. Four interceptors stay in the interceptor form. The Claude Code probe answers with the Messages family's own response facts rather than rewriting a payload, and the Responses compact shim, the Responses server-tool shim and the Messages web-search shim each drive a turn of their own. Composing the stages into each family's chain is the next step and is not here.
… chain runs it The per-rule tests say what one stage does; nothing said that the array a family's `pipeline.ts` will hold assembles at all. `compose` refuses declarations that do not line up, so running the whole array between stages that declare what the real neighbours declare — an edge that needs the answer, the headers and the billed set on the way up, and an ending that answers with all three — is that check. It also pins the one ordering that lives between stages rather than inside one: the reasoning sentinel is the gateway's canonical form, and a vendor normalizer can only put it on the wire in the vendor's shape because it runs after the stage that wrote it.
The other half of ruling 2-and-6's sub-request clause. The planning stays with the turn — what a call resolves to is the turn's decision — and what runs the backend is a run of its own, with its own prologue, its own settlement and its own record. A search settles differently from an image, and the difference is real rather than an omission. What the search backend charges is accounted per api key in units no model prices, so this run bills no entity: the row it writes names none, which is the settlement stage's own statement that a run which measured rather than generated still writes. And no `PerformanceOperation` names search, so there is no sample for the performance half to land on. What the run buys is the record — a search that ran inside a turn is legible as its own. Which is what the test asserts, because it is the only observable: one stored record on `/alpha/search`, holding the two stages the sub-chain composes. Verified by mutation — calling the resolver directly again leaves zero records.
…he conversation Ruling 2: nothing in the pipeline is written into, and nothing in it needs a clone. The three affinity materializers were the last place that broke both halves at once — each deep-cloned the record's payload per candidate and then wrote into the copy, which is what `delete replacement.signature` and `content.parts = …` were for. Cloning was how they reached a value they were allowed to edit, and freezing the record is what makes that unnecessary rather than merely wasteful: the payload they are handed cannot be edited, so the only thing left to do is return a new one. Rebuilt identity-preservingly, what one candidate is owed differs from what the next is by the handful of objects a projection actually touched, and a message no projection touched is the same object — instead of a copy of the whole conversation per candidate, on a surface that runs once per failover candidate. The three helpers this needs are the ones the Copilot boundary rules already had, so they move to `@floway-dev/protocols/common` and the provider's private copy goes. `withIndexesChanged` is new: replace-or-drop by index, which is what both the block filter and the part filter were open-coding.
The migration is over, so the vocabulary that described it should not outlive it. Four comments and six test titles still called the chat rules interceptors, and `stage.ts` still justified the pass-through direction by counting calls in the surface it replaced — a census that reads as current fact and is not one. No behaviour here; the names are the deliverable.
…it names Ruling 4: the coercion the replaced surface applied is behavior to keep, not a rough edge the migration was free to drop. A Google-RPC envelope states its code numerically in `error.code` and by name in `error.status`, and this protocol has a name for nine codes — everything else is `INTERNAL`, which belongs to 500 alone. So a 402 refusal minted from an upstream that spoke another protocol was answered as `code: 402` beside `status: "INTERNAL"`, sent as a 402: three statements, two of them contradicting the third. `main` answered 500 (`gemini/respond.ts:132`); nothing tested it, which is how the migration lost it. What let it be lost is that the body and the status were separate decisions — `renderFailure` returned an envelope and every edge then wrote `answer.status` beside it, free to disagree. They are now one: `renderFailure` hands back both, and only the third tier — the envelope this gateway words itself — may answer with a status other than the failure's. A forwarded upstream body and a refusal's own envelope keep theirs, because neither is this gateway's to reword. That makes the class structural rather than remembered. `mintedAs` covers the protocols whose envelope says nothing about its status, Gemini generateContent mints both together, and the name-for-a-status function is private to the module that mints — so no caller holds the half that can disagree. The three open-coded copies of the envelope (two edges and the models endpoint) go through it too.
…arse back Ruling 5: the non-JSON error is a corner case this gateway no longer pays for. `renderFailure` already states the whole of it — a body this protocol can carry is forwarded, anything else is an envelope minted from the status and the message — and what was left was code still shaped for the surface where that was not true. Three refusals the gateway writes itself were serialized so they could be read back. The Anthropic Messages shim built an `ApiErrorResult` around encoded bytes and its one caller decoded and re-parsed them; the server-tool stage stringified its envelope into the message field and set the body to the same object it had just set the envelope to. Both now state the envelope as the envelope: it is what `Failure.envelope` is for, and a refusal this gateway authored has no upstream bytes behind it to round-trip. The test harness had absorbed the difference — it encoded the failure's *message* as the response body, which only read as JSON because the message was a JSON blob. It now renders through `renderFailure`, so what a test decodes is what an edge would have sent. Two comments claimed `Failure.body` survives for a dump reader. It is what the client is answered with, on the tier that forwards it.
The naming branch takes `0084` for the flag-id rewrite, so this one moves to `0085`. Nothing references it by name and neither has been applied anywhere, so the number is the whole of the change; the two touch different tables and would have run correctly in either order regardless.
Brings in the flag-id rename from the naming branch. Three conflicts, all the
same shape — the naming branch edited a surface this branch had already moved or
deleted:
- `web-search-shim.ts`: keep the pipeline's exported entry point, take the
renamed flag id.
- `compact-shim.ts`: keep this branch's file. The interceptor the other side
renamed a flag inside is gone; `simulatesCompaction` reads the flag now.
- the Copilot boundary comment: keep this branch's wording, take the new id.
The stage-era files the rename could not see — this branch's own pipelines and
their tests — carry the four ids too, and are renamed here in the same commit so
no state of this branch reads a flag by a name nothing sets.
`0084` is the flag-id rewrite from the naming branch; this branch's dump-record
migration moved to `0085` in the commit before this one.
Ten comments justified a decision by naming what the surface this replaced did. That reads as a citation and resolves to nothing once this ships — the reader has no replaced surface to look at. Each now states the property itself, and where the point really is a contrast, contrasts with the alternative a reader can still evaluate: what a target-API guard inside each rule would have said, what forwarding an unreadable body under a 200 would have claimed. No behavior and no assertions change; every one of these is a comment.
`source-mapped-stack-status_test.tsx` failed twice in a row under `pnpm run verify` and passed every time the suite ran alone — the signature of a test that races rather than one that is wrong. `settle()` is one macrotask hop, and its contract says that is enough because everything queued before it is microtasks. Restoring a frame breaks that premise: it fetches the script, reads the body, follows `sourceMappingURL`, fetches the map and reads that body too. Two `Response` body reads are real stream reads, so how many turns the chain takes is a property of the machine, and under a loaded `verify` it takes more than one. The three assertions that wait for a restoration to finish now use `waitFor`, so they wait for the outcome instead of assuming a turn count. The fourth keeps `settle()` deliberately: it asserts that a superseded restoration writes *nothing*, and draining exactly what the release queued is the window in which it could have written. Nothing here is new to this branch — the test and the hook are unchanged from `main`. What is new is 170-odd more test files running beside it.
…t writing into a frame
The last clone in the pipeline, and the reason it was there. The signature
transducer deep-cloned every event on the way in so the rules could write into
the copy: `delete part.thoughtSignature`, `parts[i].thoughtSignature = …`,
`candidate.content.parts = …`, `next.candidates = …`. Ruling 2 admits no
carve-out for a frame — nothing in the pipeline is written into, and nothing in it
needs a clone — so the pass is a rebuild and the clone is gone.
Two things kept the old shape honest and both were keyed on object identity,
which a rebuild destroys. A `WeakSet` marked events the pass had suppressed, and
another marked candidates a relocation had emptied; both are now carried. The
held event travels as `{ event, suppressed }`, and each relocation hands on a
`CandidatePair` stating what the pair now is and which side it emptied. The five
`void` mutators — `normalizeElementSignatures`, the three `relocate*` and
`removeRelocatedSignatureParts` — return values instead.
Nothing about the algorithm changes: same sliding window, same relocation rules,
same anchoring, same vendor references. The 28 existing tests pass unchanged,
which is the point — this is a rewrite of how the pass expresses itself, not of
what it decides.
The new test is the property the rewrite exists for: a deeply frozen source
survives the pass. Verified by mutation — making `withParts` assign into the
candidate it was handed fails it with "Cannot assign to read only property
'parts'", where the old code would have silently written into the caller's event.
The other clones the sweep turned up went the same way. The stored-items
scratchpad copied private payloads in and out, though the payloads it is handed
come out of the record already frozen and its one reader splices a sub-object of
them into an array. `cloneAnthropicMessagesUsageIterations` deep-copied an opaque
vendor array at three hops; nothing reads inside an entry and nothing writes one,
and the two tests that pinned the copy asserted it by mutating the source
themselves — they now assert what the snapshot actually owes the array.
What stays is stated where it stays. `getItemById` copies because what hydration
builds from that row is handed to `move()`, and handing over a live cache entry
would deep-freeze the cache.
…siblings do
The fourth materializer. It never cloned, so the clone sweep passed over it — it
built a mutable draft with `{ ...item }` and wrote the slots into that, which is
correct but is the one of four that says it differently. An item no projection
touches now rides through by identity rather than becoming a fresh copy, and the
remove branch is `undefined` in a patch rather than a `delete` on a draft.
Same helpers as the other three, so the family reads as one thing.
…ee tiers The chat families answer a failure through `renderFailure`; the five non-chat families called `renderErrorEnvelope(message, body)` directly and wrote `answer.status` beside it. Two expressions of one concept, and the one they used has two tiers where the mechanism has three — a `Failure` carrying `envelope`, the refusal a stage wrote in its own words, was silently dropped for `/v1/completions`, `/v1/embeddings`, rerank, images and audio. Nothing sets `envelope` in those families today, so this fixes no live defect. It closes the way one would arrive: the tier is optional on the type, so the next stage to write a refusal with a `code` and a `param` would have found them gone with nothing failing. Now every family answers through the same function, and body and status come from it together. `renderErrorEnvelope` stays what it always was — the OpenAI-shaped envelope, in the protocols package — reached through `mintedErrorEnvelope` as the third tier rather than called as the whole policy.
…hat makes it `renderProtocolError` was the forward-or-mint rule, and `renderErrorEnvelope` wrapped it so a caller could pass an upstream body and get either. With every family now answering through `renderFailure`, the gateway holds that rule in one place and passes no body here — so the wrapper's second parameter had no caller and the rule underneath it was a second copy of a decision already made. What is left is what this package actually owns: the shape an OpenAI-shaped protocol states a gateway-authored refusal in. The test follows — it was asserting the tiering through this seam, and the tiering is tested where it now lives.
…ir own modules Thirteen of the fifteen composed pipelines are exported because an http module mounts them. These two are not: each is reached only by the `run*SubRequest` beside it, so the export said "public" about a seam nothing outside can use. Found by asking which composed pipelines are reachable from outside the file that composes them, rather than by counting how many are exported.
`Interceptor<Ctx, Env, Result>` had a second slot for what surrounds a call, and
the module said what it was for: "the gateway's chat chains pass the
request-scoped gateway context there, while the provider-boundary chains have
nothing ambient to hand down and pass `{}`". Those chat chains are stages now,
so all eight call sites pass `object` and every interceptor names the parameter
`_env`. The slot outlived its only user.
It is gone, along with the twenty-three `_env: object` parameters and the
`stubRequest = {}` fixtures that existed to fill it. The header says what the
framework is now: one slot, provider boundaries only.
The mutation convention stays, and now says which side of the house it belongs
to. A boundary shapes one wire call it owns outright and writes into `ctx`; a
turn travelling through the gateway is a fact record, frozen at every handover.
Both are true, and the comment used to describe only the first while the second
was being built.
The codemod over-applied twice — `, {}, ` is an ordinary argument in
`packages/translate` and in five gateway tests (import data, a search body, a
Hono env, a usage map). Both reverted after reading the diff against the files;
what is left is the interceptor pattern only.
The flag-id rewrite merged into `0083` — one migration for one table — which frees the number this one was moved off. Numbering is contiguous again: `0083` rewrites the stored protocol names, `0084` drops the edge-shaped dump records.
Brings in the merged protocol-name migration and the `Stateful*` reordering.
Three conflicts, all where the rename touched a file this branch had rewritten
or moved:
- `items/store.ts`: take the incoming wording. This branch's comment said the
scratchpad "is what will write there once that shim is a stage" — it is a
stage here, so the future tense was already wrong on this branch.
- `serve-prep.ts`: keep this branch's two imports. The other side still needs
the pre-pipeline serve path's dozen.
- `server-tools/shim.ts`: keep this branch's imports and its deletion of the
210-line interceptor-era usage accumulation.
`prologue.ts` and `openai-responses/websocket.ts` exist only here, so the rename
could not see them; they are renamed in this commit.
`f70d833` says "the twenty-three `_env: object` parameters". Twenty-three is the count of the inline `(ctx, _env, run)` form; the `_env: object,` form it names is twenty. The commit removed both: 43 parameters in total. Recorded here rather than by rewriting the message, because that commit sits under two merges — rewriting it renumbers 119 commits and drops the review anchors on #476 for one figure. Whoever writes the squash message should use 43.
Menci
added a commit
that referenced
this pull request
Aug 19, 2026
#493 landed on main: an upstream usage counter reported as `null` now reads as an absent counter rather than erasing what an earlier event stated, and the usage types split into `UsageDelta` (nullable, what the wire carries) and `Usage` (the totals). Eight conflicts, all the rename against that reshaping. `protocols/anthropic-messages/usage.ts` is taken from main and renamed rather than patched — main rewrote most of it, and reconciling hunk by hunk would have produced something neither side wrote. The clone of `iterations` stays here: it is #476 that deletes it, not this PR, and taking it out during a merge would pull that PR's concern down into this one. Two test files gained a test from main that arrived without a conflict, because main appended where this branch had not edited. Neither the merge nor the rename can see those: a file that already existed gets no rename pass, and git has no reason to flag text it merged cleanly. Both are renamed by hand here, which is the second time this class has come up in two merges.
Brings main's #493 down. Two conflicts, both on the file this branch deletes a clone from while main rewrote it: - `usage.ts`: main's structure — the nullable `UsageDelta` split from the totals, and the `present()` collapse — with this branch's removal of `cloneAnthropicMessagesUsageIterations` re-applied on top. The vendor reference moves back onto the type it describes. - `usage_test.ts`: main's four new null-counter tests, and this branch's two rewritten iterations tests. The pair it replaced asserted the deep copy by mutating the source themselves; what stands now is what a snapshot owes the array.
Menci
added a commit
that referenced
this pull request
Aug 19, 2026
Six comments in this PR's own files justified a decision by naming what the surface this replaced did. That reads as a citation and resolves to nothing once this ships. Each now states the property itself, and where the point really is a contrast, contrasts with the alternative a reader can still evaluate. These were swept on #476 by mistake — the files are this PR's, and the sweep belongs where they live. The final state was already right either way, which is exactly why a per-PR review catches this and an end-state sweep does not.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #475. Draft until its base lands.
Every chat entry serves through a pipeline: four families'
generate, both count-tokenoperations, the compaction, and the WebSocket transport. The onion below them is deleted.
The fact space
Four source protocols, ten translation directions between them, and one shared array of
stages that every chain runs. Two things the key layout says that the old context object
could not:
consumes the source and provides the target, so which protocol a stage is looking at is
a declared need rather than an ambient field.
ingress.*is what the client asked for. It survives the switch to whatever protocolthe upstream turned out to speak, which is why
wantsStreamis four disjoint keys.Gemini is source-only — nothing translates into it — so by the ruling it contributes one
pipeline rather than two.
Where a wire is chosen
A candidate that can serve a request may speak a different protocol than the client did, so
the wire is picked per candidate by the chain's last stage, which hands into the chain for
it. Failover re-runs the whole suffix including that stage, so the next candidate re-picks
with no mechanism of its own — which is what "only the last stage may name a target" buys.
All nine wires are built. A translated one is
handOff(pair)followed by the targetprotocol's own ending, so one chain serves a native request and a translated one and nothing
below the handoff knows which it is. A rule that speaks about a wire lives in that wire's
chain rather than in a source chain, which is how a translated turn gets shaped for the
protocol it is going to instead of the one it came from.
What only wiring exposed
The chains were tested against their declarations long before any route ran one, and every
defect below survived that. They are one lesson rather than a list of accidents: a stage
tested through its declaration is tested against what it says, and these are all things a
declaration cannot say. Four of them are about a seam — between a source chain and a wire,
between the record and a provider, between a run and its recording, between a translator and
the run around it.
interceptors shape their payload in place, down to nested nodes. A shallow spread hands them
a fresh top level over frozen children, so
Cannot add property copilot_cache_control, object is not extensiblecame back from the point in the stack least able to explain it.Every Chat Completions turn, every Messages turn carrying
cache_control— essentially allClaude Code traffic — and everything dialled over those wires. Fixed at one boundary,
bodyForAttempt, so a wire cannot express the dial any other way.the belief that every rule had been ported. Every Chat Completions turn billed zero, and so
did every other family's turn over that wire.
been left in the source chains.
stream.framehad no producer on any chatchain, so a key with retention configured paid for records holding stage boundaries and
nothing else, and the dashboard's collected view had nothing to replay.
the context from an options object and then opened without the recording those options had
started. The WebSocket entry passed its own sink, which is why the gap survived.
the caller's fault; the replaced surface said so in the caller's own protocol. The handoff
now answers it as a 400 there — and as a value, so failover can try a candidate whose wire
is a protocol that can carry the body, which the replaced surface had no way to express.
narrowing.refusenever provided thestreamed-usage key, so settlement's still-reading test read
undefinedand deferred a turnthat had no stream to wait for.
flag after being resumed past the terminal frame, and the client-output wrapper returns at
that event, so the resumption never came.
TokenUsagepass forUsageQuantities.suppress-thought-partsamong them,so thought parts reached clients that never asked for them.
down and none wrote it back, so a follow-up turn could not be pinned.
copy rather than the record.
count_tokensskipped settlement, against a ruling that had been explicitly ratified.And what only re-reading exposed
A three-way review against the design documents and the transcript found four more, every one of
them the same defect as an entry above, in a file that entry's fix did not touch:
over a translation ran neither — the fix the other two families got, missing from the third.
Collapsing that also removed a second tee in the WebSocket transport, which would otherwise have
recorded every frame twice.
of its own, so every Gemini refusal was reaching its client in a protocol that client cannot
read: a Google SDK reads
error.status, which an OpenAI envelope has not got, anderror.codeas a number where it would find a string.
/v1beta/…:countTokenslet a translator's refusal escape as a 500, where generation answersit as a 400 — the same fix, on the operation beside the one that got it.
The more useful lesson than the list above it: a fix applied to the family it was found in is half
a fix. Each of these was already understood, written down, and left undone somewhere else.
What closed the gap
Route-level coverage. Before it, of the 81 test files under the chat directory two reached the
real app at all — the Responses WebSocket transport's, and one that asks the target picker a
question through it; everything else called pipeline helpers directly, and no HTTP chat route was
driven end to end by anything. Chat Completions, Messages and Gemini now have six rows each — a streamed turn,
a collected one, an upstream refusal, a refusal on a request that asked to stream, a usage row
with real quantities, and a turn dialled over a translated wire — run against a Copilot upstream
so the provider's own interceptor chain executes, and each verified by mutating the mechanism it
covers and watching exactly that row go red. Responses has one such row rather than six, which is
the thinnest of the four and is said here rather than left to be found.
The freeze defect is the measure of what that is worth: reverting the copy to a shallow spread
turns every Chat Completions row red, the four that expect a 200 reporting
expected 502 to be 200.What is deleted
78 files: all four
attempt.ts, the serve and respond layers, the three interceptorregistries, and every rule that became a stage.
iterate-candidates.tsgoes with them — thefork is a stage, and that loop was the last thing holding the old shape. With the chains
serving, the replaced surface's refusal renderers, its translator-error builders and the
result builders under them went too, and
ChatServeFailureshrank to the one shape that stilltravels as a throw.
What stays is what a chain still reaches: the Claude Code probe's recognition and frames, the
web-search request shaping the count-token chain runs, the compaction shim's summarization,
and
SourceStreamStatefor the socket.Known gaps, stated rather than implied
The code is kept deliberately so that porting it does not start from nothing — deleting it
would turn an unreachable feature into a lost one.
ctx.targetApiguards are enforced by position, not structurally:composewalks one array, and a source chain and the wire chain it hands into are separate
compositions joined by
into.24-rulings.mdrecords what the check actually covers.stream.endhas no producer,await drain()has no coverage, anddump.failed()isnever called on a failed run.
than a 502 or a 500, on every chat entry.
is answered as this protocol's own envelope carrying that text as the message, where the
replaced surface forwarded the bytes with the upstream's own content type. The status is
still the upstream's. This follows the ruling the branch already made for a body a family
cannot parse: a gateway that cannot read the answer says so in words its client can read.
Deferred<T>— implemented in feat(pipeline): add the fact-record pipeline core #474 and used by the runner's teardown, but settlementstill reaches the background scheduler rather than providing a deferred fact.
Two decisions left open
Both are recorded in
24-rulings.md§13a rather than settled here.is broken there, and the reason is structural rather than lucky: interceptor chains exist
only for the three chat protocols. The general fix is to copy at the provider contract
boundary rather than at each family's wire, which reaches across feat(gateway): serve the non-chat families through the pipeline #475 and the provider
packages.
cache-bucket fold and the vendor field renames change what the client is shown but not what
is billed. Faithful to the surface this replaced and consistent across the families, but
nobody has ruled on it.