Skip to content

feat(gateway): a waiting clock write names a computer that has stopped watching at once - #636

Merged
earakely-scale merged 1 commit into
mainfrom
edgararakelyan/clock-quiet-followers
Nov 12, 2026
Merged

earakely-scale merged 1 commit into
mainfrom
edgararakelyan/clock-quiet-followers

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Part of FD-5420: a computer's own clock follows the gateway's universe clock. #614 made clock writes able to wait for the computers following the clock (wait_followers_s). This makes that wait stop for a computer that has lost its connection.

The rest of what was #634 (clock writes behind the control token, watches bound to each computer's read token) is deferred with FD-5472, along with moving tokens out of environment variables.

What changes

  • A computer that has stopped watching isn't waited for. A follower holds a watch up to 25 s and reconnects at once. One whose last watch is older than that plus 5 s (ClockFollowers.QUIET_S) has lost its connection. A write given wait_followers_s now names such a computer at once instead of waiting out its full time.
  • A computer that goes quiet during the wait ends the wait at its quiet deadline: stale_after wakes at each acknowledgement and at the next quiet deadline of a computer it waits for.
  • A computer that hasn't watched yet is still waited for, since it may be starting.
  • The 504 carries followers: each computer's last shown version and when it last watched.

Without this, once core passes wait_followers_s (FD-5473), a computer whose follower died would hold every clock change on it for the whole wait, up to 30 s.

Testing

  • Unit: tst/unit/env/gateway/clock_watch_test.py covers:

    • a computer that stopped watching named at once, with when it last watched, while the other is waited for as before;
    • one that goes quiet during the wait named at its quiet deadline, not after the whole wait;
    • one that hasn't watched yet waited for its full time;
    • the 504's followers.

    All 988 gateway unit tests pass.

  • Integration, on an EC2 host (x86_64): gateway_test.py's clock tests pass. These are the clock extension, its rate accuracy, the time triggers, the autonomous trigger and sync_env_clock.

  • E2E, on the same host, against the branch's gateway image with two real follower processes long-polling under their read tokens:

    Case Result
    Arming the clock (no token: the writes stay open) 200
    An advance waiting for both computers 200, in 4 ms
    One computer paused (SIGSTOP) for 36 s, then an advance waiting up to 10 s 504 in 5 ms, naming it, last watched 36 s before; the other acknowledged
    The paused one resumed, then an advance waiting 200, in 4 ms
    The gateway restarted, then a re-arm waiting 200

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no blocking issues were found.

Summary

Clock writes now stop waiting for computers whose last watch is at least 30 seconds old.

  • Clock writes stop waiting for computers that have gone quiet.

No actionable issues were found. Tests were inspected, not run.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  W[Clock write] --> C[Wake held watches]
  C --> S{Any followers behind?}
  S -->|No| OK[Return 200]
  S -->|Yes| A{Any still within quiet limit or never watched?}
  A -->|No| E[Return 504 with follower details]
  A -->|Yes| D{Wait deadline reached?}
  D -->|Yes| E
  D -->|No| P[Wait for acknowledgement or next deadline]
  P --> S
Loading

Reviews (3) · Last reviewed commit: "feat(gateway): a waiting clock write nam..." · Reviewed by Greptile

…d watching at once

A follower holds a watch up to 25 s and reconnects at once, so one whose last watch is older than that and 5 s more
(ClockFollowers.QUIET_S) has lost its connection. A write given wait_followers_s waited its whole wait for such a
computer; it now names it at once, and one that goes quiet during the wait ends it at its quiet deadline. One that
hasn't watched yet is still waited for, as it may be starting. The 504 carries each computer's last shown version and
when it last watched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner November 12, 2026 16:01
@earakely-scale
earakely-scale merged commit c3d949f into main Nov 12, 2026
61 of 63 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/clock-quiet-followers branch November 12, 2026 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant