Event feed connector: the run loop and tier-2 driver (2/3) - #705
Conversation
There was a problem hiding this comment.
Pull request overview
Adds the experimental Go Event Feed reference connector, deterministic conformance infrastructure, WebSocket transport, and checkpoint persistence.
Changes:
- Implements the push/poll state machine, recovery, deduplication, and checkpointing.
- Adds real and fake transports plus tier-2/tier-3 conformance tests.
- Documents the experimental API and architecture exception.
Tip
If you aren't ready for review, convert to a draft PR.
Click "Convert to draft" or run gh pr ready --undo.
Click "Ready for review" or run gh pr ready to reengage.
Reviewed changes
Copilot reviewed 54 out of 55 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
AGENTS.md |
Registers Event Feed infrastructure. |
CONTRIBUTING.md |
Documents conformance verification. |
go/README.md |
Adds Event Feed usage guidance. |
go/go.mod |
Adds WebSocket dependency. |
go/go.sum |
Locks WebSocket dependency. |
eventfeed/backoff.go |
Implements retry timing. |
eventfeed/backoff_test.go |
Tests retry timing. |
eventfeed/cable.go |
Implements cable framing. |
eventfeed/cable_test.go |
Tests cable framing. |
eventfeed/catchup.go |
Implements catch-up and streaming. |
eventfeed/catchup_test.go |
Tests catch-up behavior. |
eventfeed/checkpoint.go |
Defines checkpoint contracts. |
eventfeed/clock.go |
Implements production timers. |
eventfeed/clock_test.go |
Tests production timers. |
eventfeed/connector.go |
Defines the public connector. |
eventfeed/connector_test.go |
Tests connector construction. |
eventfeed/continuation.go |
Validates continuation URLs. |
eventfeed/dedupe.go |
Implements delivered-ID deduplication. |
eventfeed/dedupe_test.go |
Tests deduplication. |
eventfeed/digest.go |
Implements filter digests. |
eventfeed/digest_test.go |
Tests shared digest vectors. |
eventfeed/doc.go |
Documents the package contract. |
eventfeed/errors.go |
Defines terminal errors. |
eventfeed/errors_test.go |
Tests error taxonomy. |
eventfeed/event.go |
Defines feed events. |
eventfeed/event_test.go |
Tests event decoding. |
eventfeed/export_test.go |
Exposes test-only hooks. |
eventfeed/filestore.go |
Implements file checkpoints. |
eventfeed/filestore_test.go |
Tests file checkpoints. |
eventfeed/filters.go |
Defines filter validation. |
eventfeed/filters_test.go |
Tests filter validation. |
eventfeed/loop.go |
Implements connector lifecycle. |
eventfeed/loop_test.go |
Tests lifecycle behavior. |
eventfeed/reconnect_test.go |
Tests reconnect and staleness. |
eventfeed/recovery.go |
Implements poll recovery. |
eventfeed/recovery_test.go |
Tests recovery paths. |
eventfeed/scenario_conformance_test.go |
Replays conformance fixtures. |
eventfeed/scenario_fixture_test.go |
Decodes fixture contracts. |
eventfeed/scenario_harness_test.go |
Provides scenario harnessing. |
eventfeed/scenario_selftest_test.go |
Tests driver strictness. |
eventfeed/seams.go |
Defines connector seams. |
eventfeed/transport.go |
Enforces cable URL policy. |
eventfeed/transport_contract_test.go |
Defines transport contract tests. |
eventfeed/transport_test.go |
Tests cable URL policy. |
eventfeed/websocket_transport.go |
Implements WebSocket transport. |
eventfeed/websocket_transport_test.go |
Tests real WebSocket behavior. |
eventfeed/feedtest/clock.go |
Adds deterministic virtual time. |
eventfeed/feedtest/clock_test.go |
Tests virtual time. |
eventfeed/feedtest/minter.go |
Adds scripted ticket minting. |
eventfeed/feedtest/minter_test.go |
Tests scripted minting. |
eventfeed/feedtest/polls.go |
Adds scripted polling. |
eventfeed/feedtest/polls_test.go |
Tests scripted polling. |
eventfeed/feedtest/store.go |
Adds scripted checkpoints. |
eventfeed/feedtest/transport.go |
Adds scripted cable transport. |
eventfeed/feedtest/transport_test.go |
Tests scripted transport. |
Suppressed comments (1)
go/pkg/basecamp/eventfeed/filestore.go:230
- The rename is atomic but not durable without syncing the staged file and parent directory. After a system crash, the first checkpoint file can disappear; the next
Loadthen reports Missing and starts at the present, which can skip history rather than merely replay from an older position. SinceSaveand the package advertise durable checkpointing, sync the file before rename and the directory after rename, or stop claiming crash durability and avoid treating disappearance as a safe present entry.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ceb8398f4c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 54 out of 55 changed files in this pull request and generated no new comments.
Suppressed comments (4)
go/pkg/basecamp/eventfeed/websocket_transport.go:73
- The offered subprotocol is not verified after the handshake. coder/websocket v1.8.15 accepts a 101 response with an empty
Sec-WebSocket-Protocol, so this can return a connection even thoughactioncable-v1-jsonwas never negotiated, contrary to theCableTransportcontract. Checkconn.Subprotocol()and reject/close a missing selection; add a server case that intentionally selects none.
go/pkg/basecamp/eventfeed/catchup.go:302 - This parks an already-observed overflow until the poll returns. SPEC §23 requires semantic signals at the first consumer-context opportunity after the condition arises (with “before the next save” only as the outer bound), so a stalled poll can postpone the handler forever even though this goroutine has received the dropping frame. Dispatch the overflow immediately here; an Accept disposition can continue awaiting the poll, while Terminate should cancel the attempt/poll.
} else if over {
// The buffer, not the socket: the call is unaffected and is
// awaited to completion, and the drop's disposition runs
// before this page's position moves anything durable.
l.deferred = &deferredFrame{item: item, overflow: true}
if l.hooks.frameDeferred != nil {
l.hooks.frameDeferred(true)
}
r := <-done
return r.page, false, r.err
go/pkg/basecamp/eventfeed/websocket_transport.go:73
checkCableURLexplicitly accepts case-insensitive schemes (the new unit test includesWSS://), but this passes that original spelling to coder/websocket. In v1.8.15 its handshake switch recognizes only lowercasews/wss, so a URL accepted by policy fails as a transient dial and is retried indefinitely. Normalize only the scheme before dialing, while leaving the ticket-bearing remainder unchanged.
This issue also appears on line 70 of the same file.
go/pkg/basecamp/eventfeed/websocket_transport.go:183
- This synchronous graceful close can block teardown for several seconds: coder/websocket v1.8.15 waits up to 5 seconds to write the close frame and another 5 seconds for the peer response. Because
disposecalls this before cancelling the attempt, caller cancellation,Connector.Close, terminal outcomes, and reconnects can all stall on a live peer that ignores the handshake. Use a bounded teardown strategy that preserves the required close frame without letting the library's full handshake timeout delay the universal Closed edge.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 54 out of 55 changed files in this pull request and generated 3 comments.
Suppressed comments (1)
go/pkg/basecamp/eventfeed/catchup.go:504
- A live event admitted by
drainScanis stranded when the buffer was empty at the start of this iteration:batchstays empty, so this returns even though the scan just repopulatedl.buffer. Streaming never drains that buffer, delaying the event until a later repair walk and allowingcaught_up(and a held save) to happen first. Continue the loop when the scan added events.
batch := l.buffer.take()
if out, done := l.drainScan(at, &budget); done {
return out, true
}
if len(batch) == 0 {
return cycleOutcome{}, false
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6eb58e6010
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Round-2 note on the suppressed comments, since three of them are load-bearing and one changed the design. catchup.go:302 (overflow parked until the poll returns) — taken, and it reframed the visible finding above it. Rather than add a third dispatch site I deleted the overflow deferral outright; catchup.go:504 (live event stranded by drainScan) — real, and a defect introduced by my own round-1 change: websocket_transport.go:73 ×2 and :183 — all three taken and in progress: the negotiated subprotocol is never verified after the handshake, a Thanks for putting the sharp ones in the suppressed block; they've been the most useful part of both rounds. |
|
Closing out the remaining suppressed comments from round 2 (02fec5f). Two fixed, one declined with evidence. Unverified subprotocol — real, fixed. Confirmed in the library rather than assumed: Unbounded graceful close — real, fixed. Case-insensitive scheme — not a defect, declined. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 54 out of 55 changed files in this pull request and generated no new comments.
Suppressed comments (3)
go/pkg/basecamp/eventfeed/catchup.go:158
- A socket outcome deferred while this poll was in flight is skipped when the poll itself fails. For terminal poll branches,
recoverPollcallsdisposeAttempt, which clearsl.deferred; for retryable failures, the deferred frame can remain undispatched through arbitrarily many retries. In particular, aninvalid_event_stream_commandobserved during CatchingUp can be replaced bypoll_failed/authorization_failedor delayed indefinitely, despite SPEC §23 requiring protocol-fatal to terminate from every socket-open state. Dispatch the already-observed socket outcome here before applying poll recovery; a failed poll has no successful page boundary left to finish.
if p.err != nil {
step, out, done := l.recoverPoll(at, cursor, p.err)
go/pkg/basecamp/eventfeed/transport.go:40
- Checking
u.Hostdoes not ensure that the URL has a hostname. For example,wss://:443/cablehasHost == ":443"but an emptyHostname(), so it passes the policy check and is classified as a transient dial failure instead of terminalinvalid_cable_url, causing repeated re-mints/dials for a structurally unusable URL.
go/pkg/basecamp/eventfeed/cable.go:250 - The tier-2 push-event schema requires all nine keys, including presence-bearing
visible_to_clients, and SPEC §23 treats a correlated message missing a required event key as an invalid frame. Omitting this field from the presence checks accepts both an absent value and JSONnull, exposing a nil value on a push event instead of taking the socket-failure recovery edge.
{"creator_id", p.CreatorID != nil},
{"recording_id", p.RecordingID != nil},
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 02fec5ff1f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 54 out of 55 changed files in this pull request and generated no new comments.
Suppressed comments (1)
go/pkg/basecamp/eventfeed/catchup.go:160
- A deferred socket outcome is skipped when the in-flight poll returns an error.
recoverPollmay retry, increment authorization failures, or terminate, and disposal then clearsl.deferred; this can even swallow an already-observedinvalid_event_stream_commandinstead of producingprotocol_fatal. Since no page succeeded, dispatch the deferred outcome before classifying the poll error (the finish-page ordering only applies to successful pages).
if p.err != nil {
step, out, done := l.recoverPoll(at, cursor, p.err)
if done {
return out, "", true
Holding two findings open deliberately: the in-flight-poll mechanism has drawn four roundsTwo round-4 findings — the suspendable bound in The ledger on one mechanism, in order:
Each fix was principled and each was verified, but four rounds of edge-findings on one structure is evidence about the structure. What they all orbit: the state machine both awaits a poll seam call and services the frame queue during it, parking one out-of-band frame in a single The concurrency is not optional — fixtures 01 and 19 require a live frame admitted after
I lean (a) plus a note, because the deferral that remains is the one SPEC explicitly sanctions (transition 21's deferred consumption) and (b) risks the delivery-ordering guarantees that nine fixtures and the save-ordering invariant pin. But it is a judgment call about this PR's core, so it goes to a human rather than to me. The other three round-4 findings (synchronous cancellation on |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a9103dd83a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 54 out of 55 changed files in this pull request and generated 1 comment.
Suppressed comments (3)
go/pkg/basecamp/eventfeed/loop.go:152
ctxis the attempt context, which is a child ofrunCtx.Connector.Close(or caller cancellation) therefore cancels the pump immediately, before the state machine can reachdisposeand callconn.Close. The default WebSocket read may abort the socket on that cancellation, so normal shutdown can still produce the abrupt disconnect that the close-before-cancel ordering is meant to prevent. Give the pump a cancellation scope that is canceled only by disposal after the socket close; HTTP seam calls can continue using the attempt context.
go/README.md:615- This says the connector performs no wire I/O, but the package directly performs the sanctioned WebSocket cable dial. That contradicts both the implementation and the architecture rule. Limit the claim to HTTP requests and explicitly name the cable dial exception.
**Experimental: the Layer-1 seam adapters have not landed yet.** The connector performs
no wire I/O of its own — every HTTP exchange reaches the wire through a seam backed by
a generated operation — and the adapters that build those seams over the generated
`CreateStreamTicket` and `PollEvents` operations are still to come. Until they do, a
consumer must supply the `TicketMinter` and `PollSource` implementations itself, and the
exported surface may still change as they land.
go/pkg/basecamp/eventfeed/loop.go:638
- Cancellation is not re-checked after
Load. A custom store that honors the supplied context can returncontext.Canceledwhen the caller cancels or callsClose, and this path then yieldscheckpoint_loadeven though cancellation is documented to end iteration cleanly. Match the mint/poll paths by givingrunCtxcancellation precedence over the load error.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 93a3d5a457
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
93a3d5a to
308df85
Compare
Completes 6ea9d12 (event-feed-foundations), which typed the transport's read-limit abort: coder/websocket rejects an over-limit message during the read, and ReadFrame now returns the exported ErrFrameOversize sentinel flat. This branch's run loop never learned that vocabulary. observableSocketError preserves package-owned sentinels, *CloseError, and *invalidFrameError, and degrades every stranger to errSocketFailed — so the one invalid-frame shape the SPEC assigns to the transport arrived typed and left generic, and Observer.Disconnected lost the indication §23 requires it to carry. The contract is the invalid-frame class (SPEC.md §23, the three-shapes bullet — SPEC.md:2810-2821 at this revision): "a frame exceeding EVENT_FEED_MAX_FRAME_BYTES" is one of "three shapes, one disposition", each dispatched as a socket failure "with Observer.disconnected carrying an invalid-frame indication; never an untyped decoder error escaping", and "the size check binds inside the transport". The disposition was already right — a pump read error takes the current state's socket-failure edge to Backoff regardless of type — so this is observability, not a verdict change. Classification design: the sentinel passes through, rather than widening invalidFrameError with a third shape. The sanitizer's own taxonomy decides it. invalidFrameError names violations the CONNECTOR judged in frames it read; the oversize frame is never materialized, so there is nothing for the connector to classify — the seam's sentinel IS the classification. A third shape would also widen the security invariant's pinned rendering vocabulary ("names its shape (frame_parse / event_decode) and nothing else") for no gain. The new arm follows the sentinel rule already stated there: reduced to the bare package value, never the argument, because a seam's wrapper text is where a cable URL rides. ErrFrameOversize is exported so seams can return it, but a var can only be referenced — unlike the exported struct types the sanitizer evicted, it cannot be rebuilt around peer text. feedtest.Conn now wraps the sentinel in its read-limit violation, aligning the fake with the seam contract its interface documents — an untyped fake would keep every test green against a classification the real transport defeats. The red proof stages an actually oversized frame through the pump (one byte past the dial's own recorded cap) and failed against the pre-fix sanitizer with "Disconnected err = event feed socket failed, want the ErrFrameOversize indication"; green after, asserting both the errors.Is match and the bare-sentinel rendering, with the Backoff timer arming to pin the disposition unchanged.
…as the bug
Five of this round's eight findings taken; the notable decline is recorded
at the end.
The continuation validator had the defect validateConfig was cured of two
commits ago, on its other consumer: checkContinuation canonicalized the
server-supplied URL first, and CanonicalOrigin's ToLower rewrites every
invalid byte to U+FFFD — so a continuation host carrying a raw 0xff
collapsed to the same canonical form as a configured origin that
legitimately contains the replacement character (valid UTF-8, so
construction accepts it), and §8 same-origin validation equated two
distinct byte strings on the authenticated path. The raw bytes are refused
before the lossy step; red proof: the collapsed URL was accepted against a
U+FFFD base.
Two waits treated a staleness firing as a verdict where §23 says it is a
wake. In awaitSupersededPoll, the stale=false arm returned superseded on an
authoritative expiry — but a read error does not reset staleness, so the
PRE-EXISTING window, mostly spent when the failure was deferred, abandoned
the in-flight poll almost immediately: a failure landing 7s into a 7.5s
window granted the poll 0.5s of the grace phase §23 measures from the
deferral ("wakes may be early or late, and the deadline is what decides").
The arm now latches through graceWake exactly as the stale caller does, and
the deadline alone ends the wait — red proof: teardown on the old firing,
green: a wake, then teardown only when virtual time reaches
deferral + EVENT_FEED_STALE_AFTER. In waitPollRetry, a select with both
poll-retry and staleness ready could take the retry and start another Poll
on a socket whose expired window was already evidence — and a blocking call
then defers the stale verdict and buys a fresh grace phase, overrunning
§23's published bound ("detection window + grace phase... nothing waits
longer than their sum"). The retry arm now re-checks staleness before
polling; the red proof parks the run goroutine in an overflow handler while
both timers fire, and 17/40 rounds issued the second poll.
Transition 6 was announced late: newLiveConn opens the socket, arms
staleness and starts the pump — the transition's own definition — but the
state stayed Connecting until awaitConfirmation, so Observer.Connected ran
with AwaitingWelcome's timer set {handshake-deadline, staleness} against an
announced state whose set is {handshake-deadline} alone. The announcement
now precedes the callback; red proof: Connected observed "connecting".
Wait could report quiescence for a run that had started: the single-shot
claim was an atomic CompareAndSwap taken BEFORE the mutex section that
publishes runDone, and the claim is observable in that window — a second
consumption yields the usage terminal — while Wait still read nil and
returned. The claim now lives in the same critical section as the
publication, so Wait runs wholly before it (no run to await) or wholly
after runDone exists. No red proof discriminates this one honestly: the
window was two adjacent statements with no seam between them, and a hook
would exist for nothing else — the same account the terminal-claim fix gave
in an earlier round. The existing single-shot and Wait suites hold the
contract on both sides of the change.
AGENTS.md's architecture row still called this package foundations-only
with the run loop pending; this branch is where that stops being true, so
the row now names foundations, the run loop, and the tier-2 driver, with
the Layer-1 adapters and the other SDKs still pending.
Declined: terminating on a post-confirmation reject_subscription. The state
machine draws transition 12 solely from AwaitingConfirmation; §23's "always
terminal — first attempt or reconnect" pins which ATTEMPT a rejection lands
on, not a state-independent verdict. Action Cable rejects only from the
subscribe callback and the connector subscribes exactly once per socket, so
a post-confirmation rejection is an unsolicited frame — and honoring it
would hand one such frame the connector's most severe verdict, ZERO
reconnects, which is precisely what the pre-subscribe gate was added to
prevent in an earlier round. The default arm's liveness-only comment says
this on purpose.
All five of this round's findings taken — four expired-verdict races in the
run loop's family, and a conformance-driver fidelity hole.
The dial result could outrun the handshake deadline: both ready, the select
random, and accepting the dial installed the pump, armed staleness, and
announced Connected for an attempt whose window had already closed —
transition 7 bypassed by a coin flip. The success path now drains the
deadline before transition 6, as a drain rather than a Stop-probe because
an unfired deadline must keep running to welcome. The red proof fires the
deadline from inside Dial, sequencing it strictly before every dial result,
so any Connected at all is a violation: 1/100 rounds pre-fix, 0 after.
The subscribe write's bounded wait selected only {written, close, phase
deadline}, on the recorded premise that the phase deadline is always the
tighter bound. It is not, twice: staleness (7.5s) undercuts even the
default 10s handshake deadline on an immediate first welcome, and a
duplicate welcome resends under the confirmation deadline, which is
configurable to anything — a dead socket sat blocked in the write
arbitrarily past the expiry rows 9/15 say tears it down. The wait now
carries the staleness case like every other socket-open wait; red proof:
a stalled write plus a fired window produced no StaleConnection within the
watchdog.
Streaming could deliver past a latched expiry: a frame received AFTER the
window fired does not reset it — staleHolder.arm latches, which is §23's
authoritative-firing rule working — but the fired timer and that frame were
then both ready, and the frame arm delivered first. The arm now re-checks
the verdict before delivering, with evaluate as the arbiter: a frame the
pump received first moved the generation and delivers as ever, a latched
expiry takes transition 25 with the frame discarded to the reconnect walk.
Red proof, rounds-driven off a collector parked mid-delivery: 17/40 rounds
delivered event 102 past the latched expiry.
A throttled Retry-After of zero was read as absence. SPEC §6 fixes the
adapter mapping — a retryable outcome whose last response carried a parsed
Retry-After maps to throttled(retry_after) whatever its status, one without
maps to transient — so PollThrottled always carries a server-directed wait,
zero included, waited exactly and cap-exempt. pollRetryDelay is now
kind-keyed; a parsed zero stays zero instead of drawing up to 60s of local
jitter (the red run drew 315ms), and a negative value clamps rather than
arming a negative timer.
The tier-2 driver's arrival-strict lookahead judged expectBuffered against
GLOBAL occupancy history, so occupancy reached and left in a long-departed
era satisfied a pending expectBuffered and let the scan read through to a
following expectCheckpoint — an out-of-order save accepted by the very
instrument that exists to reject it. History is now scoped to an era
boundary captured whenever the step pointer moves (enterStep and the
await advance), and expectBuffered's own rendezvous scans from the same
boundary, which also closes the reached-and-left-before-runStep flake its
run-time capture had. Red proof: the scan judged an arriving save against
step 3 (expectCheckpoint) on stale history. Every committed fixture still
passes under the scoped scan.
…o ask it
Four follow-ups taken, one of them a correction to this branch's own
previous round — stated plainly: the round-4 throttled-zero change overshot,
and Copilot caught it.
The two §6 texts compose; they do not conflict. The seam mapping is
presence-keyed — throttled iff the last response carried a PARSED
Retry-After — but "parsed" is defined by §6's parsing algorithm, and that
algorithm cannot yield zero: step 1 requires an integer > 0, step 2 returns
max(0, date − now) only when > 0, and the rounding rationale says why —
"zero is read as 'no usable value' and drops the request onto the local
backoff curve". So a conformant adapter's throttled ALWAYS carries a
positive value, the original value gate honored every one of them exactly,
and keying the delay on the kind bought nothing while turning a
nonconforming adapter's throttled-with-zero into a zero-delay poll-retry —
a tight loop against a server that is throttling the caller. The red run
shows it literally: "poll-retry armed for 0s". pollRetryDelay is reverted
to the value gate with the reconciliation written above it; the round-4
test asserting the wrong contract is replaced by one asserting the jitter
fallthrough. What WAS wrong is the seam doc, which still described both
throttled kinds by status ("429/503") — the exact status-keyed mapping the
§6 sentence exists to forbid — and both docs now state the presence-keyed
definition with the always-positive consequence.
The tier-2 driver had the same status-keying in the flesh, on both lanes: a
schema-valid 500/502/504 carrying Retry-After became transient and its
delay never reached classifyMintFailure's floor, while a bare 429 became a
throttled zero. Both outcome mappers now classify retryable statuses by
presence of the parsed value. And the driver's Retry-After parse was
integer-only — a valid RFC 7231 HTTP-date failed the scenario outright
(red: strconv.Atoi rejecting the date), zero and negatives became
durations. retryAfterFrom now implements §6 against the harness's virtual
clock — integer > 0; else HTTP-date with the sub-second remainder rounded
UP; else undefined — returning presence separately from the value.
The round-3 grace-phase fix also overshot, in the other direction: an
authoritative firing late in the phase re-armed graceWake for a FULL
window, so a firing 1ms short of the deadline pushed the only remaining
wake almost a whole window past it — the wait ended nearly two windows
after the deferral instead of one. graceWake now takes its wake distance
and every caller passes the remainder to its deadline (the full window at
the deferral itself, where they coincide). The new red proof pins the upper
bound the round-3 test could not see: advance to 1ms short, fire, advance
the last 1ms, and demand the teardown there — red sat parked until the
watchdog; green ends at the deadline exactly.
… lost to an announcement All seven of this round's findings taken — four in the loop's established families, three in the README, each verified against the code first. A superseded poll is now cancelled AT the grace deadline, in time and not merely in control flow. Disposal closes the socket gracefully before it cancels — deliberately, so the peer sees a close frame — but a peer that never acknowledges holds that close for the transport's full grace budget, and the abandoned seam call (carrying the caller's bearer) stayed live through it, stalling reconnection past §23's detection-plus-grace bound. The deadline branch cancels the attempt context itself; the disposal that follows finds its own cancel a no-op. The red proof captures the pump's read context and parks the teardown's close: pre-fix the context was still live at the park. AwaitingConfirmation's frame arm gains the latched-expiry drain the Streaming arm got last round — the same shape for a worse consequence: a frame received after the window fired could win the select and either admit a correlated message to the loop-wide buffer, which survives the teardown and resurfaces after reconnect, or hand a confirmation a live catch-up on a socket whose verdict was already in. Rounds red: 11/40 announced Confirmed past a latched expiry. Two Close-outranks-the-announcement gaps, both on the far side of observer callbacks that are documented Close sites. outcomeFailed announced Backoff and asked the product clock for a reconnect timer with runCtx already cancelled by a Close inside Disconnected or StaleConnection; finishDrain continued past CaughtUp into dispatchDeferred and stream, which re-arms the repair cadence through the same product seam and announces Streaming. Both now take the Closed edge first — a host clock that blocks in NewTimer would otherwise keep the iteration and Wait from ever reaching Closed. Red proofs: the state ledger carried "backoff" (and "streaming") after Close returned; green goes straight to closed. Three README corrections, each checked against the code. The terminal-vs- continuable section implied every continuable outcome waits a timer; rejected positions do not — reenterWalk re-polls a replacement cursor immediately on the same socket, and the text now says which failure continues how. The unwrap claim promised a generated error behind every poll_failed; the connector's own verdicts carry none (a 200 whose page has no position), so the claim is now conditional. And the consumer-surface section stated "Close stops the feed" without the lifecycle this branch actually ships: Close only cancels and can return with a save still in flight, Wait is the quiescence point, and a replacement connector opened straight after Close races the prior run's last save — the README now says so where the store guidance lives.
…dy race All six of this round's findings taken. The three dispatch frame arms — AwaitingConfirmation, Streaming, and the poll-retry wait — now let the Closed edge win the both-ready race Close itself creates: cancellation makes the conforming ReadFrame return its cancellation error, the pump hands it off, and the frame case dispatching it reported the consumer's own Close to Observer.Disconnected as a socket failure, after Close had returned. Rounds red: 17/40 fired Disconnected(context canceled) post-Close. A panic in host code — an observer, the signal handler, a checkpoint store, the consumer's range body, all of which run on the run goroutine — unwound with only the run context cancelled: no deferred path closed the CableConn, and a compliant conn is only required to unblock reads on CLOSE, so under an outer recovery the socket stayed open with its pump parked and a later Connector.Close could not reach it (cancelRun cleared on unwind). runCycle now re-disposes the live attempt inside a recover and re-panics; disposal is idempotent, so paths that already disposed are unharmed, and phase timers held as locals are deliberately not chased — unfired, they fire into nothing. Red: the socket stayed open and the staleness timer stayed armed through a recovered panic. The repair-poll arm gains the family's expired-verdict drain: entering the walk made an already-rendered stale verdict look in-flight — pollPage deferred it into a grace phase, and the poll's page could be accepted first — where transition 25 fires directly from every socket-open state. Rounds red: repair polls issued on an expired socket. Both unauthorized paths forwarded a seam-supplied RetryAfter into the reconnect floor. The SPEC's loop table names `unauthorized` "a kind carrying no retry_after" and row 4 says the rest: "the backoff draw alone governs". Both arms drop the field; red on each showed the 5-minute floor verbatim. The tier-2 driver's redirect leg now runs the SHIPPED per-hop predicate. Fixture 30's description promised a sentinel listener the harness never binds, and the driver classified the scripted 302 by an ad-hoc origin comparison — synthesizing the refusal rather than exercising the decision, and turning an unreducible hostile Location into a SCENARIO error instead of a refusal. redirectRefusalFrom now classifies through checkContinuation (exported to the driver as a test shim), so the decision tier 2 pins is the one that ships. What tier 2 cannot see — an adapter's HTTP client auto-following inside one seam call — is below the poll seam by construction, and pretending otherwise was the finding: the fixture's description now claims exactly what the tier verifies (the refusal decision, the continuation terminal, the closed socket, no further seam calls) and names Layer-1 adapter conformance as the owner of the zero- egress property, where the pending adapter PR's sentinel can actually observe a request. The description edit is the only fixture change, one line, and the fixture gate stays green.
Copilot's exact-head review carried one suppressed finding, and it is a real residual of the round-5 grace fix: the latched, deadline-remainder wake was armed only when the DEFERRED OUTCOME was a staleness expiry. A deferred disconnect or invalid frame left the phase's wakes riding the ordinary staleness window — which the pump still re-arms on every inbound frame — so a peer whose last frame landed just short of the deadline pushed the only remaining wake to frameTime + staleAfter: the rearm's own wake ran the deadline check a hair early, then nothing fired until almost a full window past the deadline, and the abandoned poll outlived the one-window phase §23 measures from the deferral. The SPEC's immunity is unconditional — "a frame arriving inside the phase re-arms staleness in the ordinary way and MUST NOT move the deadline" — and a deadline nothing wakes is moved in effect. The existing TestGracePhaseIsImmuneToFrameResets could not see it: its staleDeferralHarness drives a silent socket, so the deferral is always the stale kind, the one path that already had the latch. pollPage now arms graceWake at the deferral for EVERY deferred kind. The latch is what buys both published immunities — pump resets are refused, a latched window cannot be suspended — and the deferred outcome is untouched: the walk still dispatches the PARKED item, never a substituted staleness verdict (the new test pins that too — the teardown reports the disconnect's redacted "other", not staleness's empty reason). With the wake armed at the deferral, awaitSupersededPoll's firing arm collapses to one case: every firing is a wake re-armed for the remainder, the two-regime split and its evaluate call are gone, and the `stale` parameter with them. Red proof, the frame-deferral sibling of the existing immunity test: a deferred remote disconnect, the peer's last ping 1ms short of the deadline, then silence — pre-fix the wait sat parked past the deadline until the watchdog; green tears down at the deadline exactly, dispatching the parked disconnect, with Backoff arming after.
… and two honest declines Five findings; three taken, two declined with the mechanism named — and one of the takes ships with a no-red account rather than a proof, stated plainly. The subscribe write's result arm gains the family's Closed-edge check: cancellation surfaces through a conforming WriteFrame as its own error, so `written` and runCtx.Done() can be ready together and the result arm dispatched the consumer's own Close to Observer.Disconnected. No red proof discriminates this one, and two attempts taught why. The written channel's readiness is causally downstream of runCtx's, so a machine PARKED in the select always takes the Closed arm first — the race lives only in the spawn-to-select entry window, which no hook can hold open. A staged version that raced a genuine socket failure against Close fired on nearly every round on BOTH sides of the fix, because a real failure dispatched before the cancel lands is legitimately Disconnected and no assertion can tell the orderings apart. The check is kept because the family closed every sibling arm and the entry window is real; the terminal-claim fix set the precedent for saying so instead of shipping a test that proves nothing. Declined: the pump-exit (!ok) racing runCtx.Done. The interleaving is structurally unreachable: handOff's FIRST send is non-blocking, so with queue room the error item always lands and close-without-item requires a full queue — whose drained items hit the ready-item Closed check first, one round's fix ago. Forty staged rounds produced zero occurrences; the test was dropped rather than kept as decoration. Declined: redacting the generated error out of Terminal(poll_failed). The SPEC pins the opposite twice — the out-of-inventory edge list and the terminal-reason table both say poll_failed is "passed through with the generated error attached" — and the URL it may render is the VALIDATED same-origin continuation: the consumer's own configured origin and a feed cursor, carrying no credential (the bearer rides the header; tickets never touch the poll lane). The redirect arm redacts a different trust class — its URL is the hostile Location that FAILED validation. The observer surface is already closed-vocabulary either way. The live buffer is now a head-indexed ring. The finding's magnitude was wrong — reslice-and-append amortizes to O(1), not an O(capacity) copy per event — but its direction was right: each growth step copied the full backing and transiently retained a SECOND buffer's worth of payload, the exact class add's zeroing exists to prevent. The store still grows lazily while filling, wraps only at capacity, zeroes vacated slots, and exposes logical order through a snapshot the hand-off boundary now uses. Red: a full window of sustained overflow made 10004 allocations against the 10000 the dropped-ids slices account for; green makes exactly 10000, and the eviction/zeroing/order pins all hold. The driver's Retry-After parse now matches the SDK's parseRetryAfter exactly: RFC 9110's 1*DIGIT checked before ParseInt (which honours a sign Atoi also accepted — a schema-valid "+5" classified as throttled), int64 so the verdict cannot vary with the platform's int width, digits beyond int64 malformed, and a representable value clamped to the portable MaxInt32-seconds ceiling instead of overflowing the duration multiply into garbage. Red: "+5" → throttled(5s); green: the sign, width, overflow and saturation rows all land where client.go's parser puts them.
The Race Detection job on the conformance-driver branch failed TestProtocolFatalBehindADeferredRecoverableOutranksIt/page_boundary with mint_failed where protocol_fatal must win — while the same loop code passed this branch's own CI, and the driver branch's eventfeed diff is comment-only. The defect is the TEST's rendezvous, latent since the wedge was added, and a loaded two-core runner is what finally scheduled it. The choreography serves the wedge ping without awaiting it, drains the hand-off signal channel, serves the fatal, and waits for ONE signal. When the ping's hand-off lands after the drain, that one signal is the PING's: the fatal is still between the pump's read and its hand-off — where §23's normative boundary says no scan need find it — when the poll returns. The entry cut spends its capacity-1 budget on the ping, the probe scans a queue the fatal has not reached, the deferred recoverable's teardown wins, and the reconnect lands on the terminally scripted second mint: mint_failed, exactly the CI signature. The loop is correct; the test counted one signal for two frames. The rendezvous now owes one awaited signal per frame served after the drained baseline (the deferral rendezvous proves every earlier signal had landed), so the fatal is handed off — the state the guarantee is stated over — before the poll is allowed to return. The CI failure is the red capture; the interleaving cannot be forced locally without a hook into the pump's hand-off, and 20 through 100 stress runs under GOMAXPROCS=2 never hit it here. Post-fix: 100 race runs green under the same constraints.
…cts that lost their turn Five findings from the exact-head review; four taken, one declined. The panic disposal now stops the attempt's active phase timer. The round's own fix handed disposal a nil deadline — its commit even said phase timers were "deliberately not chased" because an unfired timer fires into nothing — and that reasoning holds for the system clock only: the clock is a product seam, and a handshake deadline left registered in a host's custom clock outlives the run indefinitely, contaminating every later user. The attempt retains its currently armed phase timer (handshake and confirmation deadlines, repair cadence, poll-retry), the recover stops it, and ordinary teardowns double-stop harmlessly. Red: a panic from Observer.Connected left map[handshake-deadline:1] outstanding. The in-flight poll's done arm now parks an authoritative expiry before returning the result. Both can be ready, and accepting a FAILED verdict let recoverPoll's disposal discard the socket's fired window — an unauthorized verdict incrementing the shared counter, a 410 raising FeedGap, where transition 21 should have torn the dead socket down. The walk's existing precedence (probe, then deferral, then recoverPoll) dispatches the parked expiry ahead of any failed poll, and a succeeded page keeps finish-the-page ordering exactly as if the expiry had been observed mid-flight. Rounds red: 20/40 dispatched the unauthorized verdict over the fired window, with StaleConnection never reported. Declined: arbitrating the superseded wait's done arm against the grace deadline. The deadline bounds WAITING — "nothing waits longer than their sum" — and a result that is already ready costs no wait to accept: the page's deliveries and save are §23's own finish-the-page preference, and the deferred outcome still dispatches at the very next boundary. Discarding a served page because its acceptance raced the final wake would trade data for nothing the bound asks for. Both semantic dispatch sites now let Close outrank the handler. Observer.BufferOverflow and Observer.Gap are supported Close sites, and the handler that followed them is host code that may block — run after Close returned, it kept the iteration and Wait from terminating, and its disposition would govern a feed the consumer already ended. Red, both sites: the handler ran after Close returned. The unauthorized-threshold and filter-invalid terminals rebuild their causes without the generated error. §23's terminal table mandates the attachment for poll_failed and mint_failed ONLY — the previously declined poll_failed finding stands on exactly that mandate — while these two reasons' contracts are the counter message and the server's verbatim filter message, and a generated error routinely renders the full request URL: on this lane the server-controlled continuation, path and query included. Red, verbatim: both terminals rendered "?page_token=SECRET-CURSOR" through Error and the unwrap chain; green keeps the reason, the counter text, and the verbatim filter message, on a connector-authored cause.
Foundations' round-12 P1 made every rejected-continuation rendering zero-interpolation closed vocabulary; these two end-to-end pins still required the rejected origin in the terminal message and went red on the merge. They now assert the absence of every server-authored component, matching the checkContinuation pin that owns the exact phrase.
…spelling
The routed half of foundations' round: PollError.LocationOrigin is now
data-not-rendering by contract — a hostile redirect can reflect the
caller's bearer into a host label, the CloseError.Reason precedent — and
PollError.Error already omits it, but this branch's refused-redirect
terminal still concatenated the origin into TerminalError.Msg, undoing
the field's demotion at the one surface consumers read first. The
terminal now renders the fixed violation-class phrase the continuation
rejections converged on ("the poll refused a cross-origin redirect");
the origin survives as a field on the retained sanitized cause for a
caller that reads it, which no rendering includes.
Red proof, bearer-canary form: a LocationOrigin of
https://bearer-canary-abc123.invalid appeared verbatim in the terminal's
Error() pre-fix and appears nowhere in the rendering or the unwrap chain
after. Two tests pinned the old origin-bearing message —
TestRedirectRefusalExposesOnlyTheLocationOrigin (now
TestRedirectRefusalRendersNoServerValue, inverted into the canary proof)
and TestPollFailureClassification's redirect row, which now expects the
fixed phrase and points at the negative pin.
The live buffer's ring wrapped an insert whenever the tail's index fell below the head, and grew the backing whenever it did not. Those are the right two moves in the wrong order. A drain's shifts move the head off zero long before the store reaches capacity, so a pre-capacity store whose tail has reached len(events) satisfied the wrap test first: the insert circled to index 0, and the next insert — now past the head — appended, changing the modulus every existing index is taken against. With capacity 3, add 1 and 2, shift 1, add 3, add 4 read back as 2, 4, 3. fatalScan interleaves admissions with the drain's shifts, so this was the drain's own delivery order. The insert now asks whether the store has reached capacity, which is the invariant the layout actually rests on: below it the occupied range is contiguous and ends at or before len(events), so growth is the only move that keeps every index valid, and a wrap can exist only at capacity, where the modulus never changes again. The regression test drives the three partial-wrap shapes and reads the buffer back in logical order.
…ived Three receiving arms were missed when the others learned that a ready item does not outrank a Close: the in-flight-poll servicing, the bounded admission pass, and the drain's protocol-fatal scan. Each dequeued past a cancelled run context, and each did post-Close work the Closed edge exists to rule out. The admission pass dispatched the pump's own cancellation error through the live-frame path, so Observer.Disconnected reported the consumer's Close as a socket failure after Close returned. The other two admitted frames into a buffer the next run never reads, and when that admission dropped, called Observer.BufferOverflow after Close returned — a registered handler that closed the connector and returned Accept saw the pass it returned into keep admitting. The check is added at the three receives, and once more at the one place every admission passes: admitLive refuses an admission under a cancelled run context before it touches the buffer. That is what covers the accepting-handler case wherever the pass that dispatched it receives next, rather than only at that pass's own dispatch point. Three tests pin the three arms, each red against the previous revision: the handler queues a third frame before closing and asserts no overflow observer call follows; the pass is parked by a frame-handled callback while Close lands and the cancellation queues behind the parked frame; and the drain's loop body closes with a protocol-fatal disconnect and two events already queued behind the event in hand — the fatal queued first, because it bypasses admitLive's guard and so pins the scan's own check.
Four places acted on a socket whose verdict was already rendered, and each made that verdict look like something else. The walk's page boundary probed the socket before dispatching the outcome parked during the page's own poll. A deferral latches a grace wake on the staleness timer, so a page whose delivery ran past the grace deadline reached the boundary with that wake fired; the probe read it as an expiry, disposed the attempt — which clears the slot — and reported staleness where the parked verdict was the server's own invalid_event_stream_command. The boundary now dispatches a parked verdict first, and only when one is parked: the fatal probe admits what it dequeues, so with nothing parked the ordinary order — the expired window before any admission — stands, or an overflow the probe manufactured would end the cycle ahead of a staleness verdict already in. The walk issued its first poll without probing staleness, so a window that expired during Observer.CatchUpStarted, or during a re-entry's callbacks, was deferred as if it had fired mid-call and granted a grace phase, with the page possibly accepted first. The repair cadence and the poll-retry arms already probe before entering the walk; the walk's own entries do now. The subscribe write's wait has no frame arm, because a write in flight cannot dispatch frames without re-entering the handshake it belongs to. A conforming transport is only required to unblock a write on Close or cancellation, so a verdict the pump queued behind a blocked write died with the socket when the wait ended for its own reason, and the connector reconnected into a rejection §23 says is terminal with zero reconnects: a raw invalid_event_stream_command, or a correlated reject_subscription queued behind a duplicate welcome's blocked retransmit. The same loss turned out to exist wherever the connector tears the attempt down for a LOCAL reason with the server's last word already queued behind it — a confirmation deadline against a queued rejection, an expired window against a queued fatal, a garbled frame ahead of a queued fatal, a lapse ahead of a queued pre-welcome `unauthorized` that the shared counter needed — and which arm won the select, or which frame arrived first, decided it. So the rule is stated once: every recoverable teardown (failSocket, lapse, staleTeardown) scans the queue for the server's own verdict before disposing — any disconnect frame, dispatched by reason, or a correlated rejection — and a frame arm holding one dispatches it ahead of its expiry probe. The non-blocking probe those arms each spelled out is one helper now, expiredStaleness, and the StaleConnection-then-Disconnected pair every staleness edge reports is observeStale; the five inline copies and seven pairs are gone. Eight tests pin the defects, each red against the previous revision. The written-arm probe carries no test of its own: the fake transport's write and the fired window cannot be made ready together from outside the select, and a test that passes for either behavior is worse than none.
…aking Two shapes of host-seam misbehavior reached the consumer's goroutine as a crash or a leak where the documented outcome exists. A seam returning a nil *MintError, *PollError, *DialError or *CloseError through a non-nil error interface — the ordinary Go footgun — satisfied errors.As with a nil target, and the classification that followed dereferenced it. The four sites now treat a nil target as the unclassified case: the mint and poll lanes take their unclassified terminal with a named cause rather than the nil value (rendering it is the same dereference), a dial is a transient failure into backoff rather than a policy verdict read off a nil Kind, and the observable-error reduction degrades to the generic socket failure. A product Clock that panics arming the staleness timer does so inside newLiveConn, after the dial succeeded and before the attempt owns the connection. The recovery disposed a live attempt through at.lc and stopped the phase timer otherwise, so the dialed socket stayed open with its pump never started — a permanent leak in a process an outer recovery keeps alive. The connection is declared ahead of the recovery and closed on unwind when no liveConn owns it yet. Tests: each typed-nil site, and a clock that panics on its first staleness timer, all red against the previous revision.
The third unauthorized mint's authorization_failed terminal retained the generated error as its cause, where the poll lane's threshold terminal rebuilds a bare classified cause: §23's terminal table attaches the generated error to mint_failed and poll_failed only, and this reason's contract is its counter message. The mint lane now rebuilds the same way. The redirect-refused terminal rendered "a cross-origin redirect", but the kind covers every Location that fails the per-hop rule — downgraded, non-HTTP, unparseable — so the phrase misdiagnosed the other three. It names the rule the Location failed now, which is the violation class the continuation terminals already render. The CableTransport.Dial contract described a policy error's Reason and cause as reaching the consumer through the iteration's terminal, and hung the never-render-the-URL obligation on that. The loop reads the Kind alone and forwards nothing a transport wrote, for every kind; the doc says so, and states what the obligation still protects: the seam's return is opaque text to everything outside this package, and a host that wraps or logs it directly has no reduction in front of it.
Wait's exclusion named consumer callbacks alone, and §23 said the same; the run also awaits every seam it calls synchronously — minter, poll source, transport and connection, store, clock — so a wait() from inside one of those blocks the very call the run is blocked on. Both now say "anything the run is waiting on" and list the seams. doc.go called the package unusable against the live API while the next sentence, and the README, describe hosts supplying TicketMinter and PollSource over the generated operations. Nothing ships those two seams yet; a host-supplied adapter is the supported path, and the paragraph says so instead of telling that host its path is impossible. Fixture 30's kill claim and the kill matrix's row 15 are deliberately not touched here: #778, stacked on this branch, already rewrites both (6135ca7) and scopes the zero-egress obligation to the Layer-1 adapter (a3189b5), and repeating that work here would only conflict with it.
…ader sees twice The driver's mint mapping populated RetryAfter before classifying, so an unauthorized or unrecoverable outcome carried a value §23 defines for the throttled kind alone — and its comment said row 4 floors the reconnect on it, which is the opposite of what row 4 and classifyMint say. The value rides on the throttled branch now, and the comment names the tier-3 pin that proves the connector ignores a nonconforming one. The driver's HTTP-date branch could report a negative Retry-After: Sub saturates at time.Duration's ~292-year maximum, and rounding that up to whole seconds and multiplying back wrapped. It clamps to the digit branch's MaxInt32-seconds ceiling, and a year-9999 date is pinned. The loader promised that nothing in a fixture is silently ignored, and encoding/json broke that promise for one shape it cannot see: a member named twice keeps the last value and DisallowUnknownFields never meets the first, so a duplicated expectation was discarded while the fixture stayed green — and the schema gate's own parser collapses duplicates the same way. Fixtures are now walked as a token stream before decoding and any object naming a member twice, at any depth and under escape-equivalent spellings, is refused; the four levels are probed.
Streaming arms its repair cadence before the state is announced, and says
why: the announcement is made with the exact set §23 publishes for the
state already outstanding, so an observer reacting to it sees that set and
not the empty one a statement earlier. Connecting and Backoff were the two
states that announced first and armed after — Connecting's handshake
deadline sat behind the cable-URL check, Backoff's timer behind the
announcement — and a tier-2 scenario that awaits a state and then asserts
its timers could read {} where the table says {handshake-deadline} or
{backoff}. Both arm first now; a cable-URL refusal stops the deadline on
its way to the terminal, whose set is {}.
…not the process Dial, WriteFrame and Poll run on worker goroutines so the state machine can keep selecting while they are in flight, and Go's recovery is goroutine-local: a host seam panicking on one of them ended the process, bypassing the recovery that — for the same panic on the consumer's goroutine — disposes the attempt and lets the consumer's own recover see it. Each worker now captures the panic and carries it back over its result channel, and the receiving arm re-raises it on the state machine's goroutine, where runCycle's deferred recovery runs as for any host-code panic: the socket is disposed where one exists, the phase timer is stopped where none does, and the consumer recovers the seam's own value. A poll abandoned at the grace deadline has no receiver left for its panic — its attempt is already torn down, and there is no consumer context to propagate into — which the result type says. One test per seam, each ending the test binary against the previous revision.
…ts join The two arms that cancel a pending dial — the handshake deadline and Close — join the worker and inspect only the connection it may have returned. A dial that answers cancellation by panicking had that panic captured by the worker (the previous commit) and then dropped at the join, so the connector reconnected, or closed, over a host panic the consumer never saw. Both arms re-raise it once any returned connection is closed, with the handshake deadline stopped first on the Close arm as on its ordinary path. One test per arm, each red against the previous revision.
The holder latches an expiry on arm's Stop-false verdict — a frame that arrived after the window closed — and the shipped SystemClock fires through time.AfterFunc, whose callback deregisters and then sends, so Stop reports false a moment before the timer's channel is ready. The non-blocking probe every arm runs before acting on the socket consulted the channel alone, so inside that gap a frame was delivered, or a poll issued, from a socket whose verdict the holder had already decided. The probe reads the latch first. The test wraps the clock so a staleness firing is held back from its channel after Stop has reported it, serves a frame, and asserts nothing is delivered; red against the previous revision.
The shipped feed's 410 `resume` URL re-enters at the epoch with the canonical filter set preserved — positioned in served history, so the servable history above the fence is not skipped — not at the present as the pre-merge branch documented. The recovery matrix accordingly treats an accepted gap's resume as a position-resume entry: pages save on acceptance and the buffered live events drain after them, since everything between the fence and the present is poll-served and nothing behind such an entry depends on the live buffer. The inbox lane's resume (since=0, the earliest retained item) is the same class for the same reason. A 409's two digests reach the host: Observer.FilterConflict fires with the body's position_digest and filters_digest before PositionRejected(filter_changed). The re-entry is unchanged — the digests are diagnostics, and the one that matters is a filters_digest differing from the SDK's own Filters.Digest() for the same set. The tier-2 driver follows the foundations: config and identifier params carry performers, exclude_performers and actor_types; poll rows forward performed_by_id and details; push events are pinned at the shipped eleven-key shape. Fixtures spell the resume URL as the epoch re-entry (since=<epoch_after_id>), tests spell continuations as the position- carrying URLs the server actually returns, and SPEC §23 names the class change, the observer callback, the header echoes and since's signed 64-bit range.
…and settle two §23 sentences
The tier-2 driver now proves more of what it claims. A feedGap signal
expectation can pin the resume URL the handler received — whole, as the
record contract says — and fixtures 16, 23, 25 and 27 do; a scripted
409's digests reach the connector on the poll error; every expected
client close must carry status 1000; a terminal element must be exactly
(Event{}, error); an observation record no expect step consumed —
a second Gap, a stray PositionRejected, an unmatched signal — fails the
scenario at `finally` as an unmatched outbound action does; and the
query allowlist admits the three feed dimensions the schema already
permits.
Two §23 sentences are settled against the code rather than left in
disagreement with it. Subscribe is sent on the welcome that opens each
connection, which is the only welcome Action Cable sends; a later frame
of that type is liveness only. And the two URL-bearing observer
callbacks carry poll and resume URLs reduced to their origin — they are
server-chosen text on a logging surface — while the FeedGap signal hands
the handler the resume URL whole, since its disposition is a decision
about which URL to follow.
…blishes quiescence last, and tier 2 pins the observer's reduced gap URL A consumer that closes the connector from Observer.FilterConflict no longer receives PositionRejected after Close returned: the re-entry checks the run context first, as every callback that precedes a further act does. Events' deferred cleanup unwinds in the order the guarantee needs — cancel the run context, clear the registered cancel, then close the done channel — so Wait never returns while the run context is live. The tier-2 driver records Observer.gap's URL argument, and an expectGap step can pin it: the resume URL reduced to its origin, or the fixed cross-origin placeholder — never the whole URL, which only the FeedGap signal carries. The four 410 fixtures pin both.
…ns the 409 re-entry The rejection ledger is one ordered record for both Observer.positionRejected and Observer.filterConflict, so a script that places expectFilterConflict — the 409 body's two digests — before expectPositionRejected(filter_changed) pins the payload and the ordering. Fixture 34 serves a 409 with both digests, watches the conflict callback precede the rejection, and follows the walk into its present-class re-entry at since=now: the 409 pin the family README had deferred.
The Go reference implementation of the SPEC.md §23 Event Feed connector — BC3's
account-wide event feed over Action Cable push plus polling catch-up — and the
tier-2 conformance driver. All 22 fixtures pass (23 with #778's addition).
Layer 2 only: the connector reaches the wire through
TicketMinter/PollSourceseams and one sanctioned cable dial (AGENTS.md Hard Rule 2). TheLayer-1 adapters are deferred to G1b, so the package is experimental and a
consumer supplies the seams today.
What is here
connector.go(New, the options,Events,Close),loop.go(states,transitions, timers, live buffer),
catchup.go(the poll walk, its pageboundary, the drain),
recovery.go(the 400/409/410 matrix), and the tier-2driver with its fixture model, harness and self-tests.
The foundations — seams, wire types, transports, filters, checkpoint identity,
the file store, dedupe, backoff, clock, cable codec,
feedtest/— are #777.Rebase note
The 29 commits were re-cut, not rebased. Rebasing onto #777 was attempted
and abandoned: 8 of them touch only foundation files and would replay empty, and
21 of the remaining 22 mix both halves, so every one conflicts against the four
fixes #777 applies. Resolving 22 interleaved conflicts by hand is a worse
guarantee than re-cutting, which is byte-exact by construction — the foundation
paths come from #777's tip, these 17 from the pre-split head, verified with an
empty
git diffagainst both.The five fixes on top
Blocker 2 — a poll page carrying no position is malformed
The walk took
page.Positionon trust, and an empty one silently skippedhistory in two different ways.
Position-resume:
acceptPosition("")setsl.position = "", andentryCursorselects onl.position != ""— so it does not preserve the oldcursor, it falls through to a bare present entry. The feed resumes at the
server's head with everything between skipped, reporting nothing.
Present-class is worse.
helduses""as its sentinel for "the final entrywas not present-class", so an empty position is not saved as empty — it
collapses into the sentinel, the
held != ""guard skipsacceptPositionandsaveCheckpointoutright, andcaught_upannounces anyway. The position isdiscarded, with the drain's deliveries already handed to the consumer.
Refused before delivery, counter resets, and every mutation. That placement is
what the mutation check exercises: moving the guard after the delivery loop
fails three of four subtests on
delivered ids = [101], want [].Blocker 6 — narrow the promise, and make Wait the quiescence point
Close's doc claimed "no seam call and no delivery can begin after Close hasreturned". That is not true and cannot be made true: the run goroutine checks
its context at each dispatch point and then acts, so a Close landing in between
cannot stop the call from starting — only from starting on a live context.
Closing that window means holding a lock across arbitrary host code, trading a
benign race for a deadlock reachable from any callback.
The one effect that outlives the process is the checkpoint save, and this PR
deliberately does not gate it. An accepted page's position saves even when
Close lands first — its events were already delivered, and dropping the write
would silently re-deliver them — and the save runs under a context detached
from the run's cancellation (
context.WithoutCancel: values kept, cancellationdropped), because a store that honors its
ctxis compliant and wouldotherwise lose the position Close raced. A save decided just before Close can
land just after it; that is intended, not residual.
Ordering a second connector over the same store is therefore the consumer's
to do, and
Waitis the tool: it blocks until the run goroutine has exited, sono save can be in flight — by construction, not by a narrow window. Await the
iterator's termination, or
Wait, before opening a second connector over thesame checkpoint store. (An earlier revision ordered saves through a
durableGateclaimed inside Close and tracked the unclosable [claim, write]residual in #784; the gate is gone,
Waitreplaces it, and #784 is closed withit.)
Also: a cancelled checkpoint load is no longer
Terminal(checkpoint_load).Blocker 7 — observers see origins only
#777's redactor applied to
Observer.Gapand bothCatchUpStartedsites. Anaccepted 410 latches the server's resume URL as reconnect state, so the
reconnect announces its walk carrying it — redacting
Gapalone would have leftthe identical URL leaving through a different callback one reconnect later.
Observer.Disconnectedis redacted, and this paragraph used to say theopposite. The original reasoning — an error is opaque text, and stripping a
credential out of arbitrary text means modelling the credential, which §23's
"opaque bearer" contract forbids — argued for leaving it alone. A later round
found a ticket reaching the callback through the raw seam read error, which no
seam obligation can repair from the connector's side, so both arguments now go
through closed vocabularies:
observableDisconnectReasonmaps everyunrecognized peer reason to
"other", andobservableSocketErrorreduces everycause to one the connector owns, degrading anything unrecognized to
errSocketFailed."Reduces" is load-bearing and was the last hole: matching a sentinel is not the
same as being one, so a seam returning
fmt.Errorf("read %s: %w", cableURL, context.Canceled)matched a recognized arm while its text carried the ticket.Every arm now returns the connector's own value rather than its argument;
errors.Isstill matches, and the wrapper does not survive.The seam obligation remains on
Dial,ReadFrameandWriteFrame— it is whatkeeps a preserved typed error safe — but the connector no longer depends on it
being honored.
#763 — staleness arms at socket open
Observer.Connectedfired between the socket opening and the window thatmeasures silence on it. Whatever a host's callback spent was time the window
never counted. Observed from inside the callback, which is the only place the
ordering is visible.
#760 — an occupied deferral slot no longer blinds the drain's fatal scan
drainScanreturned the moment it found the slot occupied, on the reasoningthat everything queued "arrived behind this one". That is a claim about arrival
order, where §23's carve-out is a claim about which verdict governs. With
any non-fatal outcome parked ahead of it the scan looked at nothing — and the
budget it runs under is
pumpDepth+1, sized indrain's own comment to reachevery frame the pump had already read. An occupied slot spent none of it.
The planned fix was a bounded deferral queue carved out of
pumpDepth; thatturned out to be unnecessary. Only one deferral is ever dispatched —
pumpExited,dispatchDisconnectand the invalid-frame teardown all end thecycle — so a queue would be a buffer sized for a delivery that cannot happen.
The scan keeps the first outcome and discards the rest as it passes them, which
removes the capacity question entirely: no share of
pumpDepth, no channelresize, no change to the published memory bound, no fixture sweep.
The new subtest needs both halves of the two around it: deferring during the
entry poll occupies the slot, and queuing the fatal frame mid-drain puts it
where only the scan can find it. Queued earlier, the ownership cut consumes it
first — which is how the first draft passed against un-fixed code.
On #758/#759: I could not reproduce the missing-wake-source hang. Every path
leaves a wake. The overshoot is real (close to two staleness windows) and
provably cannot exceed two, so the bound is documented in place rather than
patched, and carried to bc3 as an open §23 contract question.
Verification
Pristine worktree, one pass, clean tree before and after:
go build/go vet/-race -count=1/-count=5— passmake go-lint0 issues;gosecon the CI-pinned v2.23.0 (hash-verified) 0 issuesmake check— exit 0TestWalkFailureBetweenPagesboth subtests — the invariant that killed thereviewers' one-liner survives
-race: 0 failures, 0 data races, re-earnedbecause this PR rewrites the files that wait
Kill-matrix correction
Row 15's claim was inherited from the family README and is wrong; #778
corrects it. Tier 2 cannot prove zero egress to a foreign redirect target,
because the driver is the seam and manufactures the verdict. That is a
Layer-1 property, tracked for G1b.
Reconciled with the shipped feed contract (e28b337)
Stacked on the reconciled #777. What changed in the run loop against the shipped contract:
resumeURL is a position-resume entry. The server documents it as re-entering at the epoch with the canonical filter set preserved (the inbox's atsince=0) — positioned in served history, not at the present. Pages save on acceptance and the buffered live events drain after them; SPEC §23's present-class definition, row 17 and "Accept on FeedGap" say so, and fixtures 16/23/25/27 spell the resume assince=<epoch_after_id>.Observer.FilterConflict(positionDigest, filtersDigest)fires on a 409 beforePositionRejected(filter_changed). Diagnostics only — the re-entry is unchanged; afilters_digestdiffering from the SDK's ownFilters.Digest()for the same set is the drift signal.performed_by_id/details; push events pinned at eleven keys). Tests spell continuations as the position-carrying URLs the server returns; SPEC notes the header echoes andsince's signed 64-bit range.