Repository navigation
Honest green: skipped is a job status, every failure says why, registry hygiene - #94
Merged
Merged
Conversation
…ry hygiene LOOP-PLAN Phase 1, all four items. - The queue closes a job that ran nothing as `skipped` (one rule, skip.ts, shared with a one-time backfill that re-labels historical `done` skips in the migration that widens the status constraint). Status filters, facets, /metrics, the Overview, the dashboard chip and the morning digest know it. Waiters on a skipped dependency skip with "dependency X skipped". - A `schedule-skipping` alert after FLEET_ALERT_SKIP_STREAK (3) skipped runs. - ui-test reads the failing flow and step out of Maestro's JUnit report; every host row with ok:false gets a reason (fleet-client wrapper); a failed job's last_error carries it. GET /api/artifacts/:sha now 308s to the v1 content route instead of "no such endpoint". - Rokus register as platform roku; a serial-named Roku retires its IP-named duplicate; the sweep forgets declared simulators offline past FLEET_SIMULATOR_TTL_DAYS (7). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # CHANGELOG.md # collector/src/api/v1/routes.ts
# Conflicts: # CHANGELOG.md
…hecks Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
|
Self-review: skipped status + idempotent relabel (verified on a copy of the live brain's database: 28 historical skips relabelled, one mixed job correctly left done), stale-skip alert, failure reasons from Maestro JUnit, Roku platform and registry hygiene. Merge conflicts with #92/#93 resolved (one duplicate test import fixed). All checks green. Merging (LOOP-PLAN Phase 1). |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LOOP-PLAN Phase 1, all four items.
Summary
1.
skippedis a job status.collector/src/skip.tsholds the one rule for how a job closes:ok: falseon the final row isfailed; a final row whose error beginsskipped:isskipped; every device row skipped (none failed) under a plainok: truehost row isskipped; anything else isdone. It is the same rule the v1 summary already used, so status and verdict cannot disagree./resultscloses with it and writes the reason intolast_error(for failures too, from the failing device when the host row has none).db.ts: a rebuild widens the status CHECK (copying whatever columns exist, by name) and, inside the same transaction, re-labels historicaldonejobs with a"skipped:result row. Runs once per database; result rows untouched; idempotent.depends_on: a waiter on a skipped job skips withdependency X skipped, cascading; a failure outranks a skip; enqueue behind an already-skipped job is skipped on arrival, and validate warns.skippedwhen no child ran (all skipped, or skipped + cancelled)./api/jobsand/api/v1/jobsstatus filters, schemas (gen:apiregenerated), facets,/metrics, Overviewskipped_24h, dashboard grey chip + filter option + "skipped 24h" stat, digest batch wait, and a jobs line in the morning digest. CLI/MCP verdicts are unchanged (they read the summary, which already saidskipped).2.
schedule-skippingalert. One alert per enabled schedule whose lastFLEET_ALERT_SKIP_STREAK(3) finished runs all skipped, e.g.nightly-tvloop-hardware has skipped 3 runs in a row: no Roku developer password on this host. Runs are found by job id (<schedule>-<YYYYMMDDHHMM>, fan-out children grouped) so pre-provenance jobs count; in-flight runs are ignored; resolves on the first non-skip.3. Every failure says why. Diagnosis of the live case: job
nightly-fleet-ui-smoke-202610150230's artifact12de2ced…was uploaded and is in the store (in_store: true, 479-byte JUnit, served fine at/artifacts/<sha>)./api/artifacts/<sha>404'd because no such GET route existed (only DELETE and/pinlive there); the brain's catch-all answered "no such endpoint". Fix:GET /api/artifacts/:sha256now 308s to/api/v1/artifacts/:sha256/content. The missing reason was the executor never reading the report: it containedAssertion is false: "Fleet Runner" is visible.junit.tsparses failing testcases; ui-test (Maestro) putssmoke: Assertion is false: …inerrorandtest.failing_flow(added toresult.schema.json). No report: Maestro's exit code + last stderr line, never the command line (it carries the test password).result-reason.ts, wired intofleet-client.postResultso every host workload gets it: a device row withok:falseand no error gets its test counts; a final host row with none gets the failing devices' reasons. The dispatch catch reports uncaught exec failures as exit code + last stderr line instead of Node's message.4. Registry hygiene. (a)
describeRokuand the fallback sendplatform: "roku";os ~ 'roku'and tvloop (matches on the executor's Target.platform) unaffected. (b) A serial-namedroku-*registration withdescriptor.ipdeletesroku-ip-<ip>(not when locked/claimed). (c) The lease sweep deletes rows whose descriptor declareskindorsocsimulatorand that have been offline >FLEET_SIMULATOR_TTL_DAYS(7); not real devices, not Android emulators, not the looserisSimulator()heuristic (it matchesx86_64), not locked rows.POST /jobs/sweepreturnsforgotten.Docs: concepts.md (new "Skipped is not done", chains), api.md, dashboard.md, deploy/alerts.md, deploy/index.md, protocol.md (platform list,
skipped:convention, simulator TTL), lab.md, explore/findings.md (digest line), workloads/host.md (ui-test reasons), the lifecycle figure (re-rendered withnpm run assets -- --only lifecycle), CHANGELOG [Unreleased].Not verified
execFailureReason, not with a real failing Maestro.doneuntil Phase 2 upgrades it.Test evidence (on ultra)
collector
CI=true npm test:(1215
oklines; new:skip:×10,chain:skip ×5,fail:×1,artifact:×3,roku:registry ×3,sweep:×1,skipped:migration ×11,stale skip:×9,junit:×9,exec:×5,reasons:×7,hygiene:×10, Roku platform ×3, digest jobs ×3)fleet
CI=true npm test(with runner-machine deps installed so nothing skips):mkdocs build --strict: exit 0.🤖 Generated with Claude Code