feat(evalboard): replace the turn-budget signal with time per passed task

[bai-uipath]

The consumer half of Proposal: optimizing for wall clock time instead of expected turns. The producer is coder_eval_uipath#86, which derives each task's expected wall clock from run history and stamps expected_seconds into run.json.

Safe to merge in either order: runs without the stamp render as unscored rather than erroring.

Screenshots

---

Why the turn budget had to go

Problem	Evidence
The numbers were unmaintained	147 commits in skills touched expected_turns; six revised a value
The seed was a best-case extreme	Bulk-seeded from min_turns_success, the luckiest run ever seen; 44.7% of passing budgeted tasks exceed budget on a healthy night
Post-seed values are guesses	49% are multiples of 5; 204 tasks share a copy-pasted 300 against a p90 actual of 52, so they are permanently green
Coverage holes sit on the slow tasks	249 of 1002 tasks have no budget and hold 34.7% of task-seconds; of the 25 slowest, 14 have no budget
A turn is not a unit of time	Seconds per visible turn: p10 5.2s, p50 8.1s, p90 15.5s, max 156s. A Read and a 20-minute deploy both count 1
Not comparable across harnesses	On identical tasks Codex uses 0.50x Claude's turns and scores 98.3% within budget against Claude Code's 77.3%
Nothing consumed it	One evalboard chart, no Slack line, no gate

What changes on the dashboard

• Home page: the "Within Expected Turns (%)" card becomes Time per Passed Task, a per-harness trend of seconds per passed task. Hovering a point gives that run's seconds and its within-expected share. There is no headline figure over the chart: those numbers belong to one run and the chart spans many.
• Task grid: a new sortable vs Expected column carries the ratio and the tint. Duration stays untinted, so a long task is never mistaken for a slow one. Sorting by ratio is a genuinely different ranking than sorting by duration: on the 2026-08-18 codex nightly the two top-tens share zero tasks, and the ratio sort surfaces a uipath-troubleshoot cluster the duration sort cannot see.
• Task detail and trends: the ratio is printed beside the duration (7m29s 3.1×) and gets its own column in the per-task run history, so the number is readable without hovering.
• Watchlist: the third attention segment becomes time-overage; "turn-overage offenders" becomes slow-task offenders, reading 2m18s / 1m22s expected instead of turn counts.

Ratios render to one decimal. The second decimal is fake precision: the baseline is a min over a handful of runs (p10 over ten), and a task's own night-to-night spread is wider than the digit it would add.

The dashboard derives no baseline of its own. It reads the expected_seconds the runner stamped, so a number rendered here still matches the Slack ping that announced that run, and an unscored task (no passing run yet on that harness, or a run predating the stamp) reads as unscored everywhere rather than as "on target".

Two bugs that only running it against real data exposed

Both were invisible to the tests and are fixed here with regression coverage.

The board disagreed with the runner about its own headline. A codex nightly that stamps 3m12s rendered as 1m17s. timePerPassedTaskForTasks counted mature-skipped rows in the denominator while the runner excludes them from both sides, so it divided real seconds by 681 carried-forward passes that never ran. withinExpectedTimeRateForTasks had the same leak and survived only by accident, because those rows carry no stamped line.

The delta compared across harnesses. Consecutive nightlies alternate harness, so a codex run was being measured against the previous claude-code run and reported ▼ 75% vs prev, which described the schedule rather than a change in speed. That delta is now gone from the card entirely, for the reason above.

One semantic change worth reviewing

withinExpectedTimeRateForTasks scores passing tasks only. The turn rate it replaces counted a budgeted failure as over budget. Two reasons that does not carry over: a task that crashed in 10 seconds did not blow a time budget, and folding failures in would make a pass-to-timeout regression read as an efficiency gain once the slow pass stopped counting. Failure is already the pass rate's job, and the seconds burned failing are already in the headline's numerator (total seconds of every task that ran, over the number that passed).

expected_turns is deprecated, not deleted

The field stays on RunLimits, accepted and ignored, marked deprecated, with a CE031 exemption that states why. Deleting it now would fail every one of the ~930 skills task YAMLs that still declare it, since RunLimits sets extra="forbid", and would break any external suite that sets it (this package publishes to public PyPI and has an ADOPTERS.md). Removal belongs in its own release once those YAMLs are cleaned, and keeping the field here means no release-and-pin chain is on this PR's critical path.

Every consumer is gone: the orchestrator's one-shot warning and its two call sites, reports_stats.expected_turns_overage, the HTML header badge, the markdown Run-time Note, and the per-row expected_turns / expected_turns_overage report fields. Docs updated to describe the derived line instead.

Testing done

• pnpm vitest run: 523 passed, 43 files. tsc --noEmit: clean. next build: clean.
• Run end-to-end against real stamped data, not fixtures. Twelve real nightlies were replayed locally, each stamped using only the history that preceded it, and the board driven off them. That is what surfaced both bugs above, and it is the only check that compares what the board renders against what the runner wrote.
• New regression tests for both bugs: mature-skipped passes leaving both sides of the ratio, and a mature-skipped pass not counting as within expected.
• Component tests for the new column: the ratio rendering, the tint buckets, Duration staying untinted, and that sorting by ratio produces a different order than sorting by duration.
• uv run pytest -m "not live and not lint": 4175 passed. Four failures are pre-existing on main in this environment and untouched by this branch: test_litellm_route (us. vs eu. Bedrock profile, no litellm file in the diff) and three test_reports_stats_nonfinite cases (cohens_d → statistics.stdev on NaN; verified against 3.12.5 as well, so it is environment-local rather than a version issue, and cohens_d is byte-identical to main).
• ruff format --check + ruff check: clean. Custom CE lint: 344 passed.

Left to do

• Not seen on a deployed instance. Everything above is local. The chart, the new column and the hover have not been rendered against the live blob-backed data source.
• Not exercised on mobile. The grid's card layout carries the ratio under the duration rather than as a fifth stat; that path is tested but never viewed on a real device.
• The branch is behind main by a few non-conflicting commits and needs an update before merge.
• Manual deploy after merge: workflow_dispatch on coder_eval_uipath/.github/workflows/deploy-evalboard.yml with ref: main. That workflow does not fire on merge.
• Optional follow-up, not in this PR: the harness legend renders twice on the home page, once under each chart, with identical contents. De-duplicating it means coupling the two chart components, so it is left alone.

🤖 Generated with Claude Code

[github-actions]

Claude finished @bai-uipath's task in 1m 17s —— View job

---

Review in progress

Todo List

• Read .github/code_review.md for review standards
• Read CLAUDE.md for project conventions
• Analyze the full PR diff
• Review changed files with full context
• Perform cross-file consistency checks
• Analyze what's missing
• Design-level scrutiny
• Post comprehensive review

Starting review of PR #125: feat(evalboard): replace the turn-budget signal with time per passed task...