Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,39 @@ a CI job fails when they disagree.
- **`explore-night.ts --dry-run`** opens the tunnel, runs the build and time
checks against the live brain and prints the plan, without enqueueing,
posting, warming or booting anything.
- **A job that ran nothing now has a status of its own: `skipped`.** A runner
that could not run a job says `ok: true` with an error beginning `skipped:`,
and the queue closed that as `done`, so the job list, the status counts and
the digest counted it as a pass: `nightly-tvloop-hardware` and three explore
jobs a night "passed" in two seconds for two weeks while testing nothing.
The queue now closes it as `skipped`, with the reason in `last_error`; a host
row of plain `ok: true` over device rows that all skipped counts the same,
and a fan-out family where no child ran is `skipped`. `skipped` is accepted
by the `status` filter of `GET /api/jobs` and `GET /api/v1/jobs`, counted in
the status facets, `GET /metrics` and the Overview (`skipped 24h`, beside
done rather than in it), drawn as a grey chip on the dashboard, and given its
own line in the morning digest. The v1 summary's verdict, `fleetr`'s exit 4
and the MCP verdict were already `skipped` and are unchanged. A database
opened by this release is migrated once: every `done` job with a `skipped:`
result is re-labelled by the same rule. Result rows are not touched.
- **A job waiting on a skipped job skips too.** A skipped build produced no
artifact, so the install behind it had nothing to install; it used to be
promoted and run against whatever was already on the phone. It now skips
with `last_error` `dependency <id> skipped`, the whole way down the chain,
and a job enqueued behind an already-skipped one is skipped on arrival. A
failed dependency still fails its waiters, and wins over a skipped one.
- **A `schedule-skipping` alert.** A schedule whose last three runs
(`FLEET_ALERT_SKIP_STREAK`) all skipped raises one alert naming the reason it
keeps giving, for example "nightly-tvloop-hardware has skipped 3 runs in a
row: no Roku developer password on this host", and resolves on the first run
that is not a skip. The tvloop nightly needed it for two weeks.
- **Simulator rows are forgotten after a week offline.** The lease sweep deletes
a row whose descriptor says `kind` or `soc` is `simulator` once it has been
offline for `FLEET_SIMULATOR_TTL_DAYS` (default 7), because simulators are
recreated under new UUIDs and never come back; the live brain listed thirteen
of them. Real devices and Android emulators are never forgotten, a row that
holds a lock is left alone, and results and beacons are kept. `POST
/jobs/sweep` reports what it forgot.

### Changed

Expand Down Expand Up @@ -160,6 +193,37 @@ a CI job fails when they disagree.
`fleet-explore-tv` night "passed" in about a minute. A tvOS simulator is
now `tvos-26.5` (watchOS and visionOS likewise). No TV night has run with
the fix yet.
- **A failing `ui-test` now says which step failed.** `nightly-fleet-ui-smoke`
failed on the Galaxy S8+ every night from 10-10 with `failed: 1` and no
error, while `Assertion is false: "Fleet Runner" is visible` sat in the JUnit
report it had uploaded. The executor now reads the failing flow and its
assertion out of the report into the device row's `error` and
`test.failing_flow`; when Maestro produces no report, the error is its exit
code and last stderr line, never its command line, which carries the test
account's password.
- **No host result says `ok: false` with nothing beside it.** Every host
workload's rows go through one wrapper in the executor: a device row with
only counts says so ("1 of 1 tests failed; the runner reported no failing
step"), and a final host row with no error of its own gets the failing
devices' reasons. A subprocess failure nobody caught is reported as its exit
code and last stderr line instead of Node's message. The collector also
copies a failure's reason into the job's `last_error`, so the job list and
the `job-failed` alert show it.
- **`GET /api/artifacts/<sha256>` answered "no such endpoint".** It is where
`DELETE` and `/pin` already live, so it is the URL a person reaches for, and
the 404 read as a missing artifact: the ui-smoke JUnit report was in the
store all along. It now redirects (308) to
`/api/v1/artifacts/<sha256>/content`, and says plainly when no artifact by
that hash was ever uploaded.
- **The executor registered Rokus as `platform: android`.** Its Roku
descriptor said `os: roku-15.3.4` but no platform, and the collector's
fallback for anything that is not iOS is Android. It now sends
`platform: "roku"`; `os ~ 'roku'` targeting and the tvloop workload are
unaffected.
- **A Roku seen first by address and later by serial was listed twice.** A
sweep where the box answered SSDP but not device-info left a
`roku-ip-192-168-50-218` row offline beside `roku-X01600EMFRJ9`. A
serial-named registration now retires the IP-named row at the same address.

## [0.8.0] — 2026-10-15

Expand Down
5 changes: 4 additions & 1 deletion collector/dash/src/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -166,7 +166,9 @@ export type Job = {
job_id: string;
workload: string;
executor: "device" | "host";
status: "waiting" | "queued" | "claimed" | "done" | "failed" | "cancelled";
/** `skipped`: claimed and closed without running anything (no device matched,
* the app was not installed, a dependency skipped). Not a pass. */
status: "waiting" | "queued" | "claimed" | "done" | "failed" | "cancelled" | "skipped";
/** Job ids this one waits for; it stays `waiting` until they are all done. */
depends_on?: string[] | null;
/** Whether a higher-priority job may ask this one to checkpoint and step aside. */
Expand Down Expand Up @@ -214,6 +216,7 @@ export type Overview = {
claimed: number;
done_24h: number;
failed_24h: number;
skipped_24h: number;
last_attempt: number;
oldest_queued_age_s: number | null;
};
Expand Down
3 changes: 2 additions & 1 deletion collector/dash/src/deviceKind.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,8 @@
* The model string comes first, because it is the one field that is right
* about every device on the shelf today. `platform` is not: agents that
* predate the field are reported as ios/android by the collector's fallback,
* so a Roku arrives as `android` and an Apple TV or Apple Watch as `ios`.
* so an Apple TV or Apple Watch arrives as `ios`, and a Roku registered by an
* executor older than 0.9 as `android` (newer ones say `roku`).
* `kind` is the agent naming its own shape and is believed — but only to pick
* the family; the model still chooses which member of it (a `tv` that says
* "Roku Streaming Stick" is drawn as the stick, not a bare panel).
Expand Down
2 changes: 1 addition & 1 deletion collector/dash/src/pages/JobDetail.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ function DependsOn({ job }: { job: Detail }) {
.{" "}
<span class="faint">
{statusOf(job) === "waiting"
? "It is queued the moment the last of them is done, with any ${jobs.…} references in its spec filled in from their final rows. If one fails or is cancelled, this job fails with it rather than waiting forever."
? "It is queued the moment the last of them is done, with any ${jobs.…} references in its spec filled in from their final rows. If one fails or is cancelled, this job fails with it rather than waiting forever; if one skips, this job skips too, since there is nothing for it to run on."
: "The chain has already resolved; this is what it hung off."}
</span>
</p>
Expand Down
2 changes: 1 addition & 1 deletion collector/dash/src/pages/Jobs.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,7 @@ export function Jobs() {
// 'waiting' leads because it is the one status a person goes
// looking for: a chain that has not started is invisible in
// every other view, including the overview's queue depth.
options={["waiting", "queued", "claimed", "done", "failed", "cancelled"].map((s) => ({
options={["waiting", "queued", "claimed", "done", "failed", "skipped", "cancelled"].map((s) => ({
value: s,
label: `${s} (${d.status_counts[s] ?? 0})`,
}))}
Expand Down
3 changes: 3 additions & 0 deletions collector/dash/src/pages/Overview.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,9 @@ export function Overview() {
<Stat label="running" value={d.queue.claimed} tone={d.queue.claimed > 0 ? "warn" : undefined} />
<Stat label="done 24h" value={d.queue.done_24h} tone="ok" />
<Stat label="failed 24h" value={d.queue.failed_24h} tone={d.queue.failed_24h > 0 ? "bad" : undefined} />
{/* Its own number, never added to done: a skip ran nothing. No tone,
like the chip -- it is not good news and not a failure. */}
<Stat label="skipped 24h" value={d.queue.skipped_24h ?? 0} />
<Stat label="oldest queued" value={duration(d.queue.oldest_queued_age_s)} />
<Stat
label="last attempt"
Expand Down
4 changes: 4 additions & 0 deletions collector/dash/src/style.css
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,10 @@ td.num { text-align: right; font-variant-numeric: tabular-nums; }
/* Cancelled is deliberately not styled like failed: a person stopped it,
nothing went wrong. */
.pill.cancelled { color: var(--ink-faint); border-color: var(--line); text-decoration: line-through; }
/* Skipped is neither green nor red: the job was claimed and nothing ran, so it
says nothing about the code. Grey and solid, so it does not read as a pass at
a glance, and does not read as a person's cancellation either. */
.pill.skipped { color: var(--ink-dim); border-color: var(--line); background: color-mix(in srgb, var(--ink-faint) 12%, transparent); }
.pill.claimed { color: var(--warn); border-color: color-mix(in srgb, var(--warn) 40%, transparent); }
.pill.done, .pill.online { color: var(--ok); border-color: color-mix(in srgb, var(--ok) 40%, transparent); }
.pill.failed, .pill.offline { color: var(--bad); border-color: color-mix(in srgb, var(--bad) 40%, transparent); }
Expand Down
10 changes: 6 additions & 4 deletions collector/openapi.json
Original file line number Diff line number Diff line change
Expand Up @@ -441,7 +441,7 @@
"name": "status",
"in": "query",
"required": false,
"description": "Comma-separated: waiting, queued, claimed, done, failed, cancelled.",
"description": "Comma-separated: waiting, queued, claimed, done, failed, cancelled, skipped.",
"schema": {
"type": "string"
}
Expand Down Expand Up @@ -2169,7 +2169,8 @@
"claimed",
"done",
"failed",
"cancelled"
"cancelled",
"skipped"
]
},
"ok": {
Expand Down Expand Up @@ -2458,7 +2459,8 @@
"claimed",
"done",
"failed",
"cancelled"
"cancelled",
"skipped"
]
},
"claimed_by": {
Expand Down Expand Up @@ -3571,7 +3573,7 @@
},
"status": {
"type": "string",
"description": "What the job would be on enqueue: queued, waiting or failed."
"description": "What the job would be on enqueue: queued, waiting, failed (a dependency failed) or skipped (a dependency skipped)."
},
"fanout": {
"type": "array",
Expand Down
4 changes: 3 additions & 1 deletion collector/schemas/result.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,9 @@
"properties": {
"passed": { "type": "integer" },
"failed": { "type": "integer" },
"artifacts": { "type": "array", "items": { "type": "string" }, "description": "sha256 refs of screenshots/videos/junit uploaded to the artifact store" }
"artifacts": { "type": "array", "items": { "type": "string" }, "description": "sha256 refs of screenshots/videos/junit uploaded to the artifact store" },
"skipped": { "type": "integer" },
"failing_flow": { "type": "string", "description": "The first failing flow or test case, by name; the step that broke is in the row's error" }
}
},
"shot": {
Expand Down
Loading
Loading