Skip to content

cockpit wait: green only once GitHub has every run and job for the head - #118

Merged
addisdev merged 1 commit into
mainfrom
fix/wait-green-needs-github-runs
Oct 24, 2026
Merged

addisdev merged 1 commit into
mainfrom
fix/wait-green-needs-github-runs

Conversation

@addisdev

Copy link
Copy Markdown
Owner

Problem

cockpit wait <repo> --pr <n> (and MCP wait_for_checks) could exit 0 before a PR's checks had all registered. On a private product repo's PR #5 (2026-10-24) the wait started at 17:34:20Z, 8 s after a push, and printed 1 of 1 green in 0 s. Run 40116718009 on the new head (9 jobs) had been created at 17:34:12Z and was still queued. Every agent merges on exit 0.

Cause

  • Fleet path (what fired). findCheckSet learns a PR's head from the prHead of runs the glance has seen. Right after a push the fast loop has not seen the new head's run yet, so the newest run carrying prHead is the previous commit's. Its green set (1 of 1, the previous head's single ci run) was returned at once, with no call to GitHub.
  • GitHub path (same gap). githubState is green when every check run GitHub lists has passed. A job behind needs: has no check run until its needs finish, so after guard and changes passed, every listed check was green while the run was still in progress with six jobs to come. Nothing looked at whether the run itself had completed. There was no settle window for runs that register later (push and pull_request events, workflow_run).

Fix

  • waitForChecks asks GitHub for the head first (pulls/<n>, and the open PR's branch ref, which moves before head.sha does) and decides the fleet set for that head only.
  • A fleet green ends the wait only when githubVerdict agrees. It needs every workflow run for the head (actions/runs?head_sha=, newest per workflow) completed, every job of each completed run (jobs?filter=latest), plus other apps' checks and statuses, passed (success/skipped/neutral), and GitHub quiet for 30 s (SETTLE_MS, measured at fetch time). Until then the answer is pending with GitHub's reason. GitHub not answering is not a yes.
  • Greens carry verified: { head, runs, jobs, checks }. The CLI prints (GitHub: 1 run, 9 jobs).
  • CLI: a green without verified (an older daemon) is confirmed with the user's gh before it is believed. The dashboard-unreachable fallback (GHChecksProbe) uses the same rule: branch ref, runs for the head, settle.
  • Unchanged: skipped/neutral pass, cancelled is red with the hint, superseded commits are ignored, the re-run probe before a red, orphans and stale-view pointless, exit codes.

Tests

  • dashboard/test/checks.test.js replays the incident with a fake clock. The fleet holds the old head's green, and GitHub has the new head's run queued, then in progress, then complete with 9 jobs. The green now comes at ≥190 s, after the run finishes and 30 s of quiet. On the old code the same test reads green after 0 ms. Also covered: a second (workflow_run) run registering 12 s after the first finished, the wait=0 right-after-push case, GitHub red for a run the fleet has not seen, settle, unread jobs, replaced runs, and other apps' checks.
  • APITests.swift: GitHubChecks with a green rollup while the run is in progress is pending, settle, and a run with no job yet is its own row. A daemon green without verified waits for GitHub, turns red on GitHub's red, and times out when GitHub cannot confirm.
  • Local runs: npm test 1107 pass; swift test 126 pass; fixtures regenerated (checks-green gains verified).

Deploy

fleetd.js and lib/checks.js changed, so the daemon on ultra needs a restart via the idle-wait deploy. The CLI needs reinstalling on the laptop.

🤖 Generated with Claude Code

A wait that started 8 s after a push returned "1 of 1 green" at once from
the previous head's run: the fleet view learns a new head only from its
first run, and the new head's run (nine jobs, six behind `needs:`) was still
queued. GitHub's check list had the same gap: a needs-gated job has no check
run until its needs finish.

The daemon now asks GitHub for the head first (an open PR's branch ref wins
over the PR's lagging head.sha) and confirms every fleet green there:
every workflow run for the head (actions/runs?head_sha=, newest per
workflow) completed, every job of each and every other check passed, and
nothing new for 30 s. Until then the answer is pending with GitHub's reason;
GitHub not answering is not a yes. Greens carry `verified`. The CLI confirms
an older daemon's unverified green itself with gh, and its GitHub fallback
uses the same rule. Tests replay the race (a new head's run still queued,
a workflow_run run registering after the first run finished).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@addisdev
addisdev merged commit 4cf00c3 into main Oct 24, 2026
11 checks passed
@addisdev
addisdev deleted the fix/wait-green-needs-github-runs branch October 24, 2026 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant