Skip to content

test(comms): pin the ring-lag resync arms and replay-boundary exactness (RIG-3538) - #1123

Merged
trunk-io[bot] merged 1 commit into
compass-comms/rig3533-orchestration-depthfrom
compass-comms/rig3538-ring-lag-resync
Sep 12, 2026
Merged

test(comms): pin the ring-lag resync arms and replay-boundary exactness (RIG-3538)#1123
trunk-io[bot] merged 1 commit into
compass-comms/rig3533-orchestration-depthfrom
compass-comms/rig3538-ring-lag-resync

Conversation

@rigel-mintaka

@rigel-mintaka rigel-mintaka commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

This PR is part of a stack containing 4 PRs:

  1. main
  2. test(comms): deepen supervisor orchestration coverage (RIG-3533) #1114
  3. "test(comms): pin the ring-lag resync arms and replay-boundary exactness (RIG-3538)" (this PR)
  4. test(comms): pin dmChannelName properties and DM two-party read scope (RIG-3535) #1136
  5. test(comms): pin roster presence states, nil-source default, visibility clip (RIG-3537) #1137

internal/comms had no test that drove either resync arm of the comms
stream handler: grep -rn Lagged --include=*_test.go returned nothing, and
of the three conditions that make bus.Subscribe return
ErrBufferUnderflow, only the stale-epoch one was covered. Add five cases.

  • live-tail overrun (forwardComms, the sub.Lagged() arm) emits a
    CommsResyncRequired as its FINAL frame, then ends cleanly. Every frame
    ahead of it must be buffered content, so a degradation that dropped the
    live tail but still sent the terminal frame cannot pass.
  • a cursor at or beyond bus head resyncs at the current instance epoch.
  • a cursor older than the oldest retained event resyncs. A distinct branch
    from the one above, confirmed by mutation: disabling the eviction guard
    fails this case while the at/beyond-head case stays green.
  • replay-boundary exactness: a cursor equal to head replays nothing. Proven
    with a sentinel published after subscribing, not a sleep.
  • concurrent publishes are observed in one total seq order by two
    subscribers, strictly increasing.

Export events.RingCapacity (was ringCapacity). The overrun counts have
to exceed it, and hardcoding 1025/1100 against an unexported constant put
that invariant beyond the compiler's reach: raising the capacity would not
have failed these tests, it would have made one report a misleading message
and the other hang. They now derive from the constant. Comments in
internal/delivery's busLagFloodCount had the same literal against the same
cap, and its comment said the caps were unexported — no longer true, so it
derives from the constant now too.

The underflow tests bound their stream context. A dropped guard registers a
live subscriber that tails forever, which surfaced as a whole-suite timeout;
it now fails the single test in 10s with deadline_exceeded.

The two underflow guards are also covered at the bus layer. Keeping the
handler-level pair is deliberate — the handler collapses every underflow
trigger into one wire frame, so these confirm each trigger reaches a client
as a resync — and the doc comments say so.

All five live in the untagged subscribe_failclosed_test.go rather than
subscribe_test.go, which is //go:build pgtest and needs a live Postgres
DSN. These are handler/bus semantics with no database truth in them. The two
underflow cases drive SubscribeComms with a nil store: that arm returns
before any store access.

Each case was proven to bite by mutating production and watching it fail,
then restoring byte-identical (5/5), re-confirmed after these changes.

The design record calls the subscribe-time arm a "pre-replay lag arm"; it is
an ErrBufferUnderflow path with no Lagged() involved, and it has three
triggers rather than one. Record correction noted on RIG-3538.

Co-authored-by: Matt Wilkinson matt@rigel.build

…ss (RIG-3538)

`internal/comms` had no test that drove either resync arm of the comms
stream handler: `grep -rn Lagged --include=*_test.go` returned nothing, and
of the three conditions that make `bus.Subscribe` return
`ErrBufferUnderflow`, only the stale-epoch one was covered. Add five cases.

- live-tail overrun (`forwardComms`, the `sub.Lagged()` arm) emits a
  `CommsResyncRequired` as its FINAL frame, then ends cleanly. Every frame
  ahead of it must be buffered content, so a degradation that dropped the
  live tail but still sent the terminal frame cannot pass.
- a cursor at or beyond bus head resyncs at the current instance epoch.
- a cursor older than the oldest retained event resyncs. A distinct branch
  from the one above, confirmed by mutation: disabling the eviction guard
  fails this case while the at/beyond-head case stays green.
- replay-boundary exactness: a cursor equal to head replays nothing. Proven
  with a sentinel published after subscribing, not a sleep.
- concurrent publishes are observed in one total seq order by two
  subscribers, strictly increasing.

Export `events.RingCapacity` (was `ringCapacity`). The overrun counts have
to exceed it, and hardcoding 1025/1100 against an unexported constant put
that invariant beyond the compiler's reach: raising the capacity would not
have failed these tests, it would have made one report a misleading message
and the other hang. They now derive from the constant. Comments in
`internal/delivery`'s busLagFloodCount had the same literal against the same
cap, and its comment said the caps were unexported — no longer true, so it
derives from the constant now too.

The underflow tests bound their stream context. A dropped guard registers a
live subscriber that tails forever, which surfaced as a whole-suite timeout;
it now fails the single test in 10s with `deadline_exceeded`.

The two underflow guards are also covered at the bus layer. Keeping the
handler-level pair is deliberate — the handler collapses every underflow
trigger into one wire frame, so these confirm each trigger reaches a client
as a resync — and the doc comments say so.

All five live in the untagged `subscribe_failclosed_test.go` rather than
`subscribe_test.go`, which is `//go:build pgtest` and needs a live Postgres
DSN. These are handler/bus semantics with no database truth in them. The two
underflow cases drive `SubscribeComms` with a nil store: that arm returns
before any store access.

Each case was proven to bite by mutating production and watching it fail,
then restoring byte-identical (5/5), re-confirmed after these changes.

The design record calls the subscribe-time arm a "pre-replay lag arm"; it is
an `ErrBufferUnderflow` path with no `Lagged()` involved, and it has three
triggers rather than one. Record correction noted on RIG-3538.

Co-authored-by: Matt Wilkinson <matt@rigel.build>
@linear-code

linear-code Bot commented Sep 11, 2026

Copy link
Copy Markdown

RIG-3538

@rigel-mintaka
rigel-mintaka added this pull request to stack #1124 September 11, 2026 23:43
@rigel-mintaka
rigel-mintaka marked this pull request as ready for review September 11, 2026 23:43
@github-actions

Copy link
Copy Markdown

Compass engineering docs preview: https://compass-comms-rig3538-ring-l.compass-eng-docs.pages.dev

Deployed from compass-comms/rig3538-ring-lag-resync at 0f82610.

@trunk-io
trunk-io Bot merged commit d1628a7 into main Sep 12, 2026
16 checks passed
@trunk-io
trunk-io Bot deleted the compass-comms/rig3538-ring-lag-resync branch September 12, 2026 13:39
@trunk-io

trunk-io Bot commented Sep 12, 2026

Copy link
Copy Markdown

This pull request was merged into main as part of stacked PR 1137.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants