Repository navigation
Conversation
Wave 2. Chains, builds, energy, and five new workloads — plus a migration
defect this wave surfaced.
depends_on makes a chain one enqueue. A job names ids it waits for, is inserted
'waiting', and is promoted when the last of them closes; a failed or cancelled
dependency fails its waiters down the whole chain rather than parking them
forever. A promoted spec can carry ${jobs.<id>.artifact} and
${jobs.<id>.metrics.<key>}, substituted whole so a metric stays a number. That
is build → install → ui-test without a script watching the queue.
preemptible lets a twelve-hour drain step aside for a two-minute install. The
beacon answers preempt:true only when a strictly higher-priority queued job
would go to the same claimant; the runner checkpoints, uploads, and posts
preempted:true, which requeues with params.resume_from and hands the attempt
back. Stepping aside never counts against max_attempts, because it is not a
failure. Equal priority does not preempt.
build on the machine agent closes the loop the artifact store was built for:
check out a ref, build it, publish it with the headers that stamp publish_seq,
and a later job asking for "latest" resolves to it. It posts pending at claim
under its own status context rather than sharing the collector's, because two
writers on one GitHub context means the loser's verdict silently disappears.
Energy is measured honestly or not at all. energy_method is declared in config,
never inferred; a shared plug reports the pool's draw and refuses to divide it;
the integration skips gaps rather than bridging them and returns null rather
than zero when it has fewer than two samples. What it reports is watt-hours at
the wall above a measured idle baseline, including charging — stated everywhere
it appears, because a wall measurement cannot separate the two.
The Evals page counts what it excludes. The plant-ID write-up's accuracy rode in
decode_tok_s because vision had no field of its own, and those rows cannot be
queried today; a page that quietly dropped them would repeat the mistake it
exists to prevent.
Five workloads arrive with their metric names, their views and their refusals:
speech-eval and embed-eval on the phones, vantage from wherever an agent sits,
locale-shots, app-soak and a11y-audit on the host, self-check on the hosts
themselves. Every one of them refuses rather than approximates where a platform
cannot answer: no PSS on an iOS simulator, no locale on a physical iPhone, no
bold text below Android 12, no in-device shaping on an unrooted phone.
Three bugs found by doing the work.
Swift's convertToSnakeCase splits on capitals and a digit is not one, so
recallAt1 encoded as recall_at1 — one underscore short of the declared name,
which means a metric that silently never arrives. Explicit CodingKeys now.
The old runSoak ran simctl against a physical iPhone's UDID, so a real-device
soak reported the process DEAD at every check.
And the mono repo migration left .gitmodules inside runner-android/, where git
never looks. The gitlink survived with no URL registered against it, so
`git submodule update --init` in a fresh clone silently found nothing to do.
Moved to the root with its path rewritten; verified it registers.
Verified: the collector's suite passes with 552 assertions, the machine agent's
110 tests pass, the Android unit tests and debug build succeed, and the iOS app
builds from a clean xcodegen run off project.yml with zero xcframework
references in the committed project.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Wave 2: job chains, builds, energy, and five workloads that refuse to guess
Wave 3, and a partial one. A session limit killed three agents mid-flight; what is here is what survived being finished and verified by hand, and the gaps are named at the bottom rather than implied. The machine agent gains four workloads. model-convert turns a checkpoint into GGUF, Core ML or TFLite on the machine with the memory for it, uploading each output so a dependent job can name it. dataset-prep builds an eval set and carries its licence and source on the row, because a prepared set that loses its licence is one nobody can publish results from. serve hosts a model for the length of a lease and announces its endpoint, bound to loopback or the tailnet and never 0.0.0.0. shell is the one with a boundary. POST /jobs is unauthenticated by design, so "run this script" cannot take the collector's word for it. The machine's owner pins a sha by hand; the capability is declared only when that file exists and is non-empty, so a machine that has pinned nothing cannot be offered the job at all rather than claiming it and refusing afterwards. Refusing after the claim would still take the work off the queue. The sha is checked before the artifact is fetched, never after. The executor's handlers start becoming directories. install moved first, smallest first, with a loader that falls back to the existing chain for everything not yet moved — so the file that runs the real fleet is never half broken. Verified by driving the actual executor against a throwaway collector with seven host workloads queued: install ran from its new directory and not one job reported "not supported". The live mirror answers a different question from the artifact viewer. That one says what it looked like when it failed; this says what it is doing now, which is the question you have while a flow is running. Frames live in a bounded in-memory ring and are dropped when the job ends: the collector already stores every measurement the fleet has taken, and a video stream would trade a permanent cost for a transient benefit. MJPEG because an img tag renders it with no client code, and it degrades to a still frame rather than an error when the producer stalls. Real iPhones can finally be captured. The operations doc recorded the gap: the visual matrix advertises real phone screens while half the real phones have no capture path. A WKWebView inside the runner app closes it. The profile is named webkit and not safari, in a comment that says why, because a WKWebView has no reader mode and no content blockers and someone reading "safari" in a baseline matrix would believe a stronger claim than the capture supports. Three fixes to work the dying agents left behind. The machine's workloads referenced metric names that exist in the collector's schema but had never been mirrored into the agent's own protocol type, so none of them compiled; the four files were also never dispatched or capability-gated, which made them dead code. And one Swift expression mixed literals and Doubles densely enough that the type checker gave up and failed the build, so it is split into named steps. Verified: the collector's suite passes, the machine agent's 122 tests pass, and the iOS app builds from a clean xcodegen run off project.yml with zero xcframework references. Not done, and not pretended otherwise. push-latency needs FCM and APNs credentials. camera-eval needs a physical rig pointing a phone at a display. desktop-ui-test has no desktop app to test against. Only install has moved into the plugin layout. model-convert, dataset-prep and serve have never run against a real toolchain, because none of the three converters resolves on this machine and the probes correctly decline to declare them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Wave 3 (partial): pipelines, a live screen, and the shell trust boundary
A design pass proposed giving every job state a pill icon, and the icons are drawn and sitting in icons.tsx: one circle per state, differing only inside, so a column of them reads down cleanly. They are not wired up. ui.tsx already carried the opposite rule with a better argument behind it — device states are read across a whole shelf at a glance, job status is read one row at a time — and a pill that is already a coloured word does not get more legible for gaining a picture. Reverting the lookup and leaving the drawings behind the comment, so the next person to want them finds both the icons and the reason they were not used. Device pills keep their icons, unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A phone on the shelf and the collector in a browser are one product, and until now the Android runner was a title, two bare EditTexts, two buttons and a status line on a default AppCompat background. This gives it the dashboard's screen. The palette, theme and most shape drawables landed in 4ee7ab8 by accident — a commit about waiting jobs swept up work in progress. This is the half that makes them do anything: - activity_main is rebuilt as the dashboard's vocabulary at phone scale: a status hero over the mark's own pulse, the collector and device id as rows in a card rather than boxed inputs, three telemetry tiles, a job card that appears only when there is a job to name, and 48dp controls. - PulseTraceView draws the mark's trace left to right while the agent is in contact, and rests as a complete flat drawing when it is not. It honours ANIMATOR_DURATION_SCALE, so a device set to no animations gets none. - MainActivity binds the service's new phase and telemetry flows. The battery and thermal tiles read locally before the first beacon, because deciding whether to enrol a phone on 8% is a pre-start question, and read from the beacon afterwards, so they show what the collector was actually sent. - The launcher icon becomes adaptive: flat ink background, the phone and pulse inside the 66dp safe circle so no mask clips it, and a monochrome layer for themed icons. The status-bar icon becomes a stroke on the 24 grid; filled, it collapsed into a grey lozenge at notification size. No verdict badge on a finished job: the engines post their own final rows and this process never sees the outcome, so a green pill here would be a guess. The status TextView keeps its id and its text, so the Maestro flows that read the agent's state off this screen keep working. Verified by build only. assembleDebug links every layout and drawable reference, but three emulators wedged under this machine's load and the screen has not been looked at on a device. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The four component suites are path-filtered on purpose: a change to a phone runner should not spin up a Node matrix or a macOS runner. But that makes them unusable as required checks, because GitHub reports a check that never ran as pending forever -- so an iOS-only pull request would sit blocked on the collector's suite. The collector workflow's own header has warned about this since it was written; branch protection on main currently requires nothing at all, which is the other way to lose that argument. So the four become reusable workflows, and ci.yml calls them behind a change detector. `ci-ok` needs all four with `if: always()` and passes when each one is green or skipped. It is the only check branch protection has to name. The detector is plain git rather than a third-party action, and it fails toward running too much: an unusable diff base -- a branch's first push, a force push, a sha this checkout does not have -- runs every suite. Guessing the other way skips a suite silently, which is the failure this file exists to prevent. The changelog covers 0.1.0 and 0.2.0, and says why one version spans four components that share no code: the protocol is what changes, so the protocol is what the version tracks. Verified: actionlint clean on all five workflows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Android runner looks like the rest of the fleet
A merge gate that always reports, and a changelog
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two waves of work, the mono-repo migration, and the merge gate.
mainhasbeen describing a smaller project than the one that exists since 2026-09-04 —
its collector README lists 11 workloads where
developlists 17, and themono-repo layout it documents is not the one a visitor lands in.
Full notes in
CHANGELOG.md. The short version:Wave 2 — chains, builds, energy, and five workloads that refuse to guess
depends_onmakes build → install → ui-test a single enqueue, with${jobs.<id>.metrics.<key>}substituted whole so a metric stays a number.Preemption lets a twelve-hour
drainstep aside for a two-minuteinstalland resume, without the interruption counting as a failure. The
buildworkload closes the loop the artifact store was built for. Energy is reported
in watt-hours at the wall or not at all:
energy_methodis declared, neverinferred, and a shared plug reports the pool's draw rather than dividing it.
Wave 3 — pipelines, a live screen, and a trust boundary
model-convert,dataset-prepandserveon the machine agent.shellbehind a locally pinned sha256 allowlist, where the capability is absent
entirely unless the owner has pinned something — so an unpinned machine is
never offered the job rather than claiming it and refusing. A live MJPEG
mirror that answers "what is it doing now" without a video stream's permanent
cost. Real iPhone screen capture via a
WKWebView, labelledwebkitratherthan
safaribecause the two are not the same claim.Since those merged
The Android runner got the status screen and launcher icon the rest of the
fleet already had (#5), and CI got a
ci-okgate that always reports (#6).After this merges
Tag
v0.2.0, cut the release, and makeci-oka required check on bothmainanddevelop— which it cannot be until it exists onmain.🤖 Generated with Claude Code