Repository navigation
Findings reach the code: /api/v1, fleetr and MCP, and fixed findings become nightly regression tests - #96
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ings become nightly regression tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…essions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… findings list facets; deprecate the older findings routes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
addisdev
marked this pull request as ready for review
October 16, 2026 03:20
Owner
Author
|
Self-review: findings in /api/v1 sharing the store functions with the legacy routes; CLI/MCP/skill; regressions table with candidate→active→retired and reopen-on-recurrence; Findings page on the v1 client. Findings-table rebuild verified on a copy of the live brain's database (69 findings preserved, v1 list reads them). Checks green. Merging (LOOP-PLAN Phase 4). |
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LOOP-PLAN.md Phase 4, items 1–4, plus two route-table leftovers from Phase 3.
What changed
1. Findings in the control API. Three route-table entries (new
findingstag), thengen:api:GET /api/v1/findings: list envelope; filtersapp,state(open/triaged/all),verdict(comma-separated),check,since(ISO 8601 or a span:24h,7d,90m).GET /api/v1/findings/{id}: the finding pluscounts,evidence(role, sha256,/api/v1/artifacts/<sha>/contentlink, in_store),replay_flow(text inline up to 16 KiB),regression,history,links.POST /api/v1/findings/{id}/verdict:real | duplicate | not_a_bug | agent_mistake | fixed | null, optional note and duplicate_of. Scoperun(added toRUN_ROUTES), audited by the existing onResponse hook with the finding id as subject.fixedbeforerealis 409conflict./api/findingsroutes call the same functions (listFindings,findingDetail,parseVerdictBody,setVerdictinsrc/findings.ts); no logic is forked. The legacy verdict route keeps itsadminrequirement under tokens.2. CLI, MCP, skill.
fleetr findings,fleetr finding ID [--flow FILE|-],fleetr verdict ID real|duplicate|not-a-bug|agent-mistake|fixed|reopen [--note] [--duplicate-of]. MCPfleet_findings,fleet_finding(flow text inline),fleet_verdict(readOnlyHint false, scope run), each with an output schema. The MCP instructions mention them. SKILL.md has a "Reading a finding" section; the tool table is regenerated.3. A finding becomes a regression test. Owner's decision implemented as: Real, then Fixed.
regressions(keyed by finding id: flow text, kind, state candidate/active/retired, run counters) andfinding_verdicts(every verdict, plusregressedevents). The findings table is rebuilt once to allowfixedin its CHECK (same pattern as the jobs table forskipped), and each existing verdict gets one backfilled history row.regressionsas a candidate. fixed → active (a finding marked real before this release goes straight to active). duplicate / not_a_bug / agent_mistake → retired. real on a retired one → candidate again.src/findings.tsonly ever creates GitHub issues, never reads them back (by design, see its header), sofixedis the only trigger.ui-testacceptsparams.regressions("<app key>", ortrueforapp.name). At run time the executor pulls the app's active flows from a new protocol routeGET /findings/regressionsand runs each, after the suite's own flow, on devices of the finding's platform; each run is reported toPOST /findings/{id}/regression. I chose this over an auto-creatednightly-regressions-<app>schedule because a collector-made schedule would have to guess targets, the build, the suite's app_id and sign-in; the existing nightly already has all of those, and with the param set once every newly fixed finding joins without editing anything.suite.flowsmay be omitted for a regressions-only job (a device with nothing to run is skipped, not passed).appId→ the job'ssuite.app_idwhen set; a relativerunFlow(the mission's sign-in flow) is resolved under the flows dir, refusing escapes.logcat -b crash+am_anrread before and after the flow; a new crash of the package isreproduced. That fails the row and reopens the finding (verdict cleared, seen_count+1, build added, history lineregressed on build X (job, device): ...), once per job however many devices reproduce it; the test stays active. Every other check is "replays without error" only, and a failing flow fails the row without reopening; the docs and each run's detail say this. iOS: no crash buffer read, so crash findings are "replays without error" there too. tvloop flows are stored but skipped by ui-test.4. Morning brief.
fleetr findings --since 24h --jsonreturns the v1 list envelope of Finding objects; the shape and how to read it are in docs/agents.md ("A morning brief").Leftovers. (a)
POST /api/v1/jobs: the 400 response now namesno_matching_devices,no_buildand the other 400 codes; 422 namesno_capable_agentandunresolved_reference, matchingqueue.ts. (b) The v1 Artifact document haschannel: main|pr; thePOST /artifactsdescription namesx-artifact-channel.Dashboard. A Fixed button (
f, disabled until Real), a Regression test panel with state, last run and verdict history; the?shortcut list saysr d n a f. After merging #97, the Findings page's list, detail and verdict calls go through the generated v1 client (dash/src/client.ts) and the hand-written finding types are gone fromapi.ts. To give the page its filter menus, the v1 findings list carriesfacets(apps, checks, open/triaged counts) like the device and job lists.GET /api/findings,GET /api/findings/{id}andPOST /api/findings/{id}/verdictare now in the deprecation table; precision, digest and issues have no v1 counterpart and are not (the id pattern is\d+so they are not caught). The v1 verdict honours theFLEET_DASH_TOKENspeed bump the older route had.check-dash-requests: 56 requests, none deprecated.Part of #83, not closing it: the Findings page still reads
/api/findings/precision, and the issue's other dashboard-only routes (overview, visual, evals, system, ...) still need the add-to-v1-or-mark-internal decision it asks for.Docs: explore/findings.md (Fixed, a Regression tests section, the new routes), agents.md, api.md, workloads/host.md (ui-test regressions), dashboard.md. Changelog bullets carry #96. Workload count unchanged (34);
check-counts.mjspasses.Not verified
src/workloads/regressions.test.tsagainst fake crash readings and a fake Maestro; Maestro's handling of a rewritten absoluterunFlowpath andlogcaton a real phone are not exercised.vite buildpasses) but not looked at in a browser.Test evidence (ultra, Node 24)
After merging origin/develop (#95, #97), on ultra:
collector:CI=true npm test→ typecheck, dashboard build,the dashboard calls no deprecated route(56 requests, none deprecated), workloads ALL PASS (7), counts,openapi.json and the client match the route table, auth ALL PASS, lifecycle ALL PASS,smoke: ALL PASS, api v1 contract ALL PASS. 1377oklines, 0FAIL.fleet:npm test→ tests 83, pass 81, fail 0, skipped 2 (the twofleet upend-to-end tests skip on ultra: runner-machine deps not installed there, as before).mkdocs build --strict: exit 0.node scripts/check-counts.mjs: 7 marked counts agree, 34 workloads.🤖 Generated with Claude Code