Run B is decided against: the power analysis, the 8-epoch cosine rung, and the gate applied (#135)

[jonfroehlich]

This PR has grown past its title and now closes #135. It carries three things in sequence: the power analysis below (no GPU), the 8-epoch cosine rung (pre-registered, run, and scored manual_gold-TIED at every epoch), and — added 2026-09-03 — the pre-registered Run B gate, applied. Verdict JUDGMENT CALL, and the judgment recorded is do not run Run B: primary +0.002994 at |z| ≤ 1.86, secondary shows no arrest, and Run B at n=1 could not be told apart from a seed draw at any length. Full reasoning and the reopening condition are in docs/stage2_cosine_rung_135.md's final section and at #135.

Merging this is what closes #135 — every artifact above exists only on this branch.

Answers the prerequisite #135 set before the 1,675–3,350 GPU-hour spend: can manual_gold resolve the effect Run B might produce, and does pooling the ten splits raise it?

No GPU time was spent. Run A was not re-run.

The answer

	s.e. on manual_gold max-F1	detectable at 80% power
Unpaired, as the 0.01 tie bar assumes	0.0042	0.0117
Paired, MEASURED — Run A epochs, median	0.0021	0.0059
Paired, MEASURED — range over 28 pairs	0.0016–0.0029	0.0045–0.0081
Pooling all ten splits, unpaired	0.0039	0.0109

Pairing is worth ~2× and it is free. Both checkpoints are scored on the same 1,000 panoramas against the same GT, so pano difficulty cancels. What governs a paired comparison is the discordance between the two detectors — measured at 2.5–6.6% across Run A's epochs — not n·p·(1−p).

Pooling is worth 7% (0.0117 → 0.0109). The nine city splits alone are worse than manual_gold alone (0.0219) despite more panoramas. #135's per-split instruction is right.

The benchmark is not a reason to cancel Run B. At MDE 0.0063 measured — ~0.008 extrapolated to Run B's larger epoch separation — it resolves an effect the size of #51's annealed tail (+0.024 here, 8σ) with room to spare.

The bracket was closed for no GPU at all

An earlier revision could only bracket the paired s.e. at [0.0043, 0.0135] using stand-in pairs, because Run A's committed artifacts are aggregate PR curves. Then the 2026-08-17 scoring cache turned out to have survived on makelab2 — 13 GB, all eight epochs, 1,000 panos each — and peak extraction from a cached heatmap is CPU-only numpy. So the measurement needed no model, no panorama images (they are not on that host), no GPU and no network, took 4 minutes, and covered all 28 epoch pairs instead of the two originally proposed.

The old bracket contained the answer, and the working assumption ("4–6% discordance ⇒ 0.006–0.009") was nearly exact.

Verification is the load-bearing part. Re-scoring each dump against manual_labels/ reproduces the committed summary.csv:

	agreement, all 8 epochs
max-F1	5×10⁻⁹ to 4×10⁻⁷
F1@0.30	1.2×10⁻⁴ to 1.4×10⁻⁴

max-F1 is a property of the whole PR curve, so agreement at 10⁻⁷ says the peak extraction is the same operation, not a similar one. The uniform 10⁻⁴ offset on F1@0.30 is the committed table's own 0.005-grid downsampling, which its provenance note already documents.

It corrects #84's curve shape

With each pair read against its own measured s.e. rather than a global bracket, epochs 7 and 8 are measurably below the plateau — 3 vs 7 at z = 4.0, 3 vs 8 at 3.0, 6 vs 7 at 3.0. The unpaired 0.01 bar could not see it.

So the curve is not "steps up once from epoch 1 to 2 and is then flat":

epoch 1 clearly low → epochs 2–6 a genuine plateau → epochs 7–8 measurably declining, though still above epoch 1.

The headline is unchanged — still no resolvable peak; 2, 3, 4, 5, 6 remain mutually indistinguishable (2 vs 6 identical to 4dp, 3 vs 6 at z = 1.4). What changes is that at constant LR the model begins to lose capability after ~epoch 6.

That sharpens what Run B tests: not "does a longer budget add to a flat curve" but "does cosine decay arrest a decline that is now measured." It cuts both ways — that decline is the classic signature of an LR left too high, which is what an anneal fixes; but a 30-epoch run whose decay does not bite could land below epoch 3.

Recommendation

SUPERSEDED 2026-09-03 — Run B is not being run at all. The paragraph below is the recommendation as written on 2026-08-18, kept verbatim because the reasoning that led to it is still the reasoning; what changed is that the pre-registered gate was then actually applied. Applying it returns JUDGMENT CALL, and the judgment is no: Run B is n = 1 and seed variance is the binding limit, so the arm is uninterpretable at any length. A readable Run B is three seeds, roughly $4,500-9,000. See docs/stage2_run_b_power_135.md.

Run the 30-epoch arm; not 60 on spec. The plateau ends at 6, so the extra 30 epochs buy schedule shape rather than useful steps, and a 60-epoch cosine spends proportionally longer at the high LRs the epoch 7–8 decline is evidence against.

Pre-register three free amendments: read Run B against Run A paired on max-F1 with the tie bar replaced by the measured MDE (0.008 at Run B's separation); per-split, not pooled; attribute cautiously below ~0.01.

Also flagged, not relitigated: the #84 amendment's 8-epoch cosine rung (~450 GPU-h, budget-matched to Run A) is more attractive than it was, because constant-LR is now known to peak at ~3 and decline by 7, so that rung isolates schedule from length — the confound the amendment already noted B − A carries.

The limit that binds is not the benchmark

Panorama sampling variance only, and Run B is n = 1. Nothing measures training-seed variance, so a gain below ~0.01 max-F1 is measurable but not attributable to the anneal without a seed control that doubles the bill.

Notes

• The epoch dumps live in docs/data/run_a_84_detections/, not benchmark/model_detections/ — rampnet/roster.py asserts every file there belongs to a registered challenger leg (#122), and these are internal checkpoints, not comparison entries. The suite caught the first attempt to put them there.
• They carry a 0.05 peak floor (Run A scored at 0.0, which retains ~511k predictions); AP is not recoverable from them and must be read from docs/data/run_a_84_manual_gold/.
• The 13 GB heatmap cache is the one input a clean clone cannot obtain, which is why the dumps are committed rather than left to be rebuilt. Stated beside them.
• Instrument check, committed: #132's exclude_border artifact puts 30% of the RampNet-vs-RampNet pair's disagreements at the seam against a 2.1% baseline; the pairs drawn from one source read the baseline exactly.
• metrics() under all-ones weights reproduces rampnet.detection_eval.aggregate to 1e-12 on four (split, model) combinations. Suite 1236 passed, 1 skipped.

Files

• docs/stage2_run_b_power_135.md — the write-up, caveats beside the numbers
• scripts/analysis/benchmark_power_135.py — the analysis, seeded and deterministic
• scripts/analysis/dump_peaks_from_cache.py — cache → per-pano detections, --verify
• docs/data/benchmark_power_135.json, docs/data/run_a_84_detections/ — LF-pinned
• tests/test_benchmark_power_135.py — 10 tests

Closes #135. This PR is the decision: Run B is decided against, and the cosine rung that tested its premise is scored and committed.

🤖 Generated with Claude Code (claude-opus-5[1m])

[jonfroehlich]

Deep review

Re-ran everything from the branch head (a6fb658) in a clean worktree. The central statistical argument is correct and the recommendation survives — pairing is the real lever, pooling is not, run 30 not 60. Two independent checks came back clean:

• Full suite: 1234 passed, 1 skipped, as claimed.
• docs/data/benchmark_power_135.json regenerates byte-for-byte identical from the documented command, on Windows, with no CRLF flip (126,832 bytes, matching sha256).

So everything below is a labeling/interpretation problem sitting on top of an artifact that is exactly what the script produces. One of them makes a headline number wrong by ~2×.

---

1. High — the nine city bundles are hard-truncated at conf 0.55, so "the #54 operating point of 0.30" is not what was measured on 9 of 10 splits

benchmark/<city>/records.jsonl holds zero detections below 0.55 (min 0.5501–0.5607, all nine). manual_gold reaches 0.0501, with 28.8% of its detections below 0.55.

Four consequences:

• keep = conf >= 0.30 is a no-op on the nine cities. The unpaired table's city rows and both POOLED rows are RampNet at an effective 0.55, printed under a header saying 0.30.
• max-F1 there is not max-F1 — the sweep runs off the end of a truncated curve. Bit-proof from the committed JSON: max_f1 == f1 exactly on 8 of 9 city splits and on POOLED cities; only manual_gold and sao_paulo differ.
• The PR's own artifact already showed this. self_pair records b=0, c=0, ΔF1 exactly 0.000000 for all nine cities at both +0.02 and +0.05, while manual_gold moves (c=8, c=23). A threshold shift that changes literally nothing on nine splits is the signature.
• The rampnet_1pass:rampnet city rows measure the truncation, not TTA: b=0 on 7 of 9 cities (b≤5 on the other two) against c=18–40 — the TTA asymmetry flips sign versus manual_gold's b=78/c=3. The doc reads the manual_gold row as confirming Experiment: flip-TTA at deployment operating points — the evaluated config used it, deployed inference doesn't #78's max-combine mechanism (correctly); the city rows in the same JSON say the opposite and carry no annotation.

The repo already documents the truncation at docs/model_comparison.md:931 — "RampNet's bundle detections were extracted at a 0.5 peak threshold, so its curve has no low-confidence tail at all… its AP is a lower bound." The caveat exists; it just did not travel to this doc, which CLAUDE.md explicitly requires.

Measured impact, re-running with --reference rampnet_1pass (analysis_out/op_cache, full curve to 0.05 on all ten splits):

	committed (cities cut at 0.55)	untruncated 0.30
manual_gold MDE	0.0117	0.0122
POOLED cities MDE	0.0219	0.0190
POOLED all MDE	0.0109	0.0105
pooling gain	7%	14%

"Pooling is worth 7%" is understated by ~2× — it is ~14%. The conclusion holds (14% is still not a lever; POOLED cities alone stays far worse than manual_gold alone), but the number in the summary table and the PR body is wrong.

Caveat on that check: rampnet_1pass is also single-pass rather than TTA and is missing seam detections per #132, so it bounds the size of the effect rather than being the corrected number. There is no clean uniform-0.30 RampNet arm in committed data — that is itself worth stating in the doc.

The headline paired numbers are all manual_gold and are unaffected.

2. Medium — the "LOWER BOUND" section cannot bound what it claims

metrics(s, weights, threshold) returns max_f1(s, weights, n_gt), which takes no threshold. So the self-pair block's max-F1 delta is identically zero by construction; the committed JSON records "max_f1": {"observed": 0.0, "se": 0.0} for all 24 entries.

That matters because the module docstring assigns this section exactly that job ("…so they give the lower bound"). For max-F1 — the metric the headline bracket and the whole recommendation are stated in — it cannot. No published number is wrong (the real 0.0015 comes from rampnet_1pass:rampnet), but se: 0.0 in a committed artifact reads as "zero uncertainty" rather than "statistic is not a function of the varied parameter." Combined with finding 1, two of 24 rows are live.

3. Medium — the doc's own reproducibility claim fails on its headline number

Line 281: "Every derived number in this document is in docs/data/benchmark_power_135.json." But the number the recommendation is stated in — "4–6% discordance… puts the MDE at roughly 0.006–0.009 max-F1" (line 129, repeated at 214 and in the PR body) — appears nowhere in the JSON or the script. It is a hand interpolation.

It errs safe (interpolating the three committed points to 4–6% gives about 0.0056–0.0068), but per the repo's replication rule it should be computed by the script, and that sentence should not stand as written.

Related: the stated mechanism "the standard error tracks the discordance" does not hold on its own three points — 8.1% gives 0.0029 but 9.1% gives 0.0048, a 12% rise in discordance against a 66% rise in s.e. For max-F1 there is no McNemar form, as the doc itself says.

4. Medium-low — the summary table mixes F1 and max-F1 standard errors under a max-F1 header

The headline table (and the PR body verbatim) is headed "s.e. on manual_gold max-F1". Rows 2–3 are max-F1. Rows 1 and 4 (0.0042, 0.0039) are F1 s.e.'s — unpaired.manual_gold.f1.se and unpaired["POOLED all"].f1.se. The actual max-F1 values in the same JSON are 0.0041 and 0.0039, giving MDE 0.0114 and 0.0108.

Numerically immaterial, but it is the one table people will quote, and "0.01 is very nearly the correct unpaired MDE of 0.0117" is stated about the wrong column. (The separate "1.8×–3.6× gain" claim is like-for-like — both sides F1.)

5. Medium-low — the #84 correction is not discoverable from #84's own doc

The PR's best secondary result is that the epoch-3 preference is now a measured result rather than a judgment call. docs/stage2_epoch_curve_84.md is untouched: 11 references to the 0.01 tie bar as the operative instrument (lines 28, 37, 399, 410, 412, 426, 463, 492, 516, 564, 570) and zero to #135. Anyone reading Run A's conclusions gets the superseded framing with no pointer.

6. One statistical caution

The load-bearing assumption is hedged in one place and dropped in another. The max-F1 bracket is carefully "bracketed rather than point-estimated." But the recall table prints a flat "resolvable" for gaps needing 20–34% discordance, justified by "real pairs on this benchmark run 2.1–9.1%" — where those pairs are two cross-architecture YOLO comparisons and one seam-contaminated self-pair, none of them epoch-vs-epoch.

Run A's own data suggests epoch-to-epoch calibration is unstable (max_f1_conf 0.268 at ep7, 0.581 at ep8, 0.376 at ep1), which at a fixed 0.30 threshold is precisely the high-discordance regime; its recall column already implies b+c of at least 3.7% between ep3 and ep7 before any noise. The proposed ~1-GPU-hour measurement is the right fix; until then those verdicts deserve the same conditional the max-F1 bracket gets.

7. Low

• 3v8 silently dropped from both doc tables — 7 pairs in the script's interesting list and in the JSON, 6 rows shown.
• --splits subsets do not reproduce the committed per-split s.e. One rng is threaded through every group in order, so the draw stream depends on which splits and pairs are present. "The bootstrap is seeded, so the run is deterministic" holds only for the exact full invocation.
• ~half the bootstrap work is redundant. observed_and_se runs 3 times per (pair, group), and the res_a/res_b calls exist only to produce the single scalar unpaired_se_f1; y11x_pano_h200's unpaired result is recomputed for both pairs it appears in. About 144 bootstraps of 20k reps where ~72 would do.
• Scored.gt_pano is dead — built in score_model, concatenated in stack, never read.
• test_committed_json_matches_the_doc does not check the headline numbers its docstring claims to guard. It pins three inventory integers and one inequality; none of 0.0042 / 0.0015 / 0.0048 / 0.0039 / 0.0117 / 0.0109. Given finding 4, that is exactly the gap it was meant to be.
• (prediction_confidence(p) or -1e9) in the test maps a legitimate 0.0 confidence to -1e9; if c is None is meant. Harmless at 0.25/0.30.
• observed_and_se(paired=…) assumes s and paired share pano order without asserting it, unlike mcnemar, which does.

---

What holds up

Stating this because a defect list on its own misrepresents the PR:

• The core claim is right. McNemar variance depending on b+c rather than n*p*(1-p); one weight matrix applied to both detectors so pano difficulty cancels; resampling panoramas rather than instances, with the design effect measured rather than assumed.
• The verification is real and reproduces. metrics() against aggregate to 1e-12 across four (split, model) combos spanning both GT schemes and both detection sources; match_detail against score_pano asserted per-pano at runtime.
• The seam control is the best part. 24/81 (30%) at seam vs a 2.1% baseline on the cross-source pair; 2.2% and 2.5% on the two same-source pairs. Every number checks against the JSON. A diagnostic with a working negative control, committed to run on future pairs.
• Clean-clone reproducibility is genuine — 10 op_cache files, 112 model_detections, summary.csv, all tracked.
• "Epochs 2 and 6 identical to four decimals" is a real measurement, not rounding: summary.csv carries six (0.916537 vs 0.916548).
• required_discordance inverts correctly, and rendering an unreachable requirement as >100% rather than a number is a good call.

The irony worth naming: finding 1 is exactly what "verify the instrument before judging" exists to catch, and the evidence was sitting in this PR's own committed JSON as 18 rows of 0.000000.

Fixes for 1–7 to follow on this branch.

🤖 Generated with Claude Code (claude-opus-5[1m])

[jonfroehlich]

The red CI was a real matcher divergence, and the seam is worth 0.000264 to this analysis

Provenance note first: this work landed inside commit fa02b37, whose message describes
only the resume/checkpoint fix. Two sessions were writing this worktree at once and a
git add -A swept the seam work in with it. Nothing was lost, but git log will not lead
anyone here, hence this comment.

What was failing

assert sum(1 for _, tp, _ in detail if tp) == ref.tp
AssertionError: ('manual_gold', 'run_a_epoch_2', '5wp4GvraXfsuAGujVx1X8w', 'tp')

Failing since 2026-08-18, three runs, both Python versions. Not a flake. This branch predates
bf64451 (#140), CI scores the branch merged with main, and the panorama it failed on is in
analysis_out/seam_roll_diagnostic.json.

score_pano had started wrapping x at the 360 seam; benchmark_power_135.py's match_detail
had not. Two matchers exist here exactly so that a divergence is an error rather than a
quietly different number, and that is what the assertion delivered. match_detail now wraps,
and both the greedy match and the ignore-point fallback go through rampnet.geometry.dist_sq
instead of an inline distance — the same defect #132 §4 found in score_pano itself.

What it moved: one number

Re-scoring all eight committed epoch dumps with the wrapping matcher:

epoch	max-F1 pre-#140	max-F1 post-#140	Δ
7	0.911009	0.910745	−0.000264
1–6, 8	unchanged	unchanged	0.000000

One prediction on one panorama now claims a ground-truth ramp across the seam instead of
scoring as a false positive on one side plus a miss on the other.

The conclusions hold, and sharpen slightly

0.000264 is 4% of the 0.0063 paired MDE the recommendation rests on, and it moves epoch 7
further below the plateau. With the point estimates re-scored consistently:

pair	z before	z after
3 vs 7	3.88	3.98
6 vs 7	2.84	2.97
1 vs 2	4.42	4.43
1 vs 3	5.14	5.13
5 vs 8	2.78	2.79

So "epoch 1 low → 2–6 a genuine plateau → 7–8 measurably declining" stands, with 3 vs 7 now
at z = 4.0 and 6 vs 7 at 3.0. Nothing in the recommendation changes.

One thing this exposed that was not the seam

The max-F1 table was mixing matchers: point estimates read from
docs/data/run_a_84_manual_gold/summary.csv (written 2026-08-18 14:24, about two hours before
#140 merged at 16:30) while the standard errors beside them were bootstrapped from re-scored
detections. Both now come from the epoch dumps under one matcher. summary.csv goes back to
being the provenance record of the #84 run and the regression test's target — it is not an
input to this analysis, and it is deliberately left as written, because only four of its ten
columns are exactly re-derivable from the committed detections (the max-F1 block is; the
operating-point and AP columns come from evaluate.py and use different conventions, which is
worth knowing separately — its AP differs from the shared scorer's by ~0.03).

Both matchers are now pinned. The historical curve is checked against the historical matcher at
1e-5 — a tolerance loose enough to absorb the seam would also absorb a real regression — and
the single post-#140 difference is asserted explicitly, so a second epoch starting to move
fails the build rather than passing quietly.

Spun out

Re-scoring the whole roster to check the blast radius turned up three committed YOLO
baseline cells that #140 did move and that were never regenerated → #148. RampNet's own
numbers are unaffected on every split.

🤖 Generated with Claude Code (claude-opus-5[1m])

[jonfroehlich]

Deep review, second pass: the ten commits since a6fb658

Re-ran everything from the branch head (82b9056) in a clean detached worktree on Windows. What reproduced:

• Full suite: 1361 passed, 1 skipped (the PR body's 1236 is stale).
• check_lr_schedule_135.py --events-dir stage_two/cosine_rung_135_events: PASS, output identical to the block quoted in the doc — 11 incarnations, 75,024 unique steps, 100.00% of the run, 0 non-decreasing violations, max deviation 4.547e-13 at step 15339. All 18 event files match SHA256SUMS.
• stage2_epoch_curve.py with the documented arguments regenerates docs/data/stage2_cosine_rung_135.csv byte-identical.
• Every cell of the rung's manual_gold table (F1@0.30, max-F1, AP, the −0.0073 / −0.0039 declines, the +4.62% / +1.43% auto-val rises) re-derives from the two committed summary.csv files.
• benchmark_power_135.json regenerated from the documented command (--bootstrap 20000 --matrix-bootstrap 5000, ~18 minutes on this desktop): byte-identical to the committed file — 162,252 bytes, sha256 d29923d4…, no CRLF flip on Windows. So every number below is the script's; the defects are in what was read out of it.
• git merge-tree against origin/main: clean.

The training-side work is sound and verified over the whole run. The problem is on the reading side: the rung's pre-registered test was not run, and the 2026-08-18 review's findings were mostly not fixed — the first review's "Fixes for 1–7 to follow on this branch" is still outstanding for five of the seven.

---

Status of the seven prior findings (review of 2026-08-18)

#	finding	status	evidence at 82b9056
1	nine city bundles truncated at 0.55; "pooling is worth 7%" is really ~14%	not fixed	JSON reference: "rampnet"; max_f1 == f1 exactly on 8 of 9 city rows and POOLED cities; benchmark/clovis/records.jsonl min confidence 0.5533, richmond 0.5519, 0.0% below 0.55; doc lines 34, 73–74, 103, 109, 336 still say 7% and "the #54 operating point of 0.30" with no truncation caveat; PR body repeats 7%
2	self-pair max-F1 se: 0.0 by construction	not fixed	metrics() (script lines 331–348) still returns max_f1(s, weights, n_gt) with no threshold; all 24 self_pair rows carry max_f1.se = 0.0
3	"0.006–0.009" hand-interpolated, not in the JSON	superseded	the measured matrix replaced it; line 149 keeps the old range as a historical remark, which is fine. But the sentence it was flagged under — "Every derived number in this document is in docs/data/benchmark_power_135.json" (lines 411–412) — is still false, see new finding 3
4	headline table mixes F1 and max-F1 s.e. under a max-F1 header	not fixed	rows 1 and 4 still 0.0042 / 0.0039 = unpaired.manual_gold.f1.se (0.00419) and POOLED all.f1.se (0.00390); the max-F1 values in the same JSON are 0.00406 and 0.00387, MDE 0.0114 and 0.0108
5	docs/stage2_epoch_curve_84.md does not point at #135	not fixed	git diff origin/main...HEAD -- docs/stage2_epoch_curve_84.md is empty; the file has 0 references to #135 and 11 to the tie bar as the operative instrument
6	recall-table "resolvable" stated flat where the max-F1 bracket is hedged	not fixed	lines 234–248 unchanged
7a	3v8 dropped from both doc tables	half fixed	now in the max-F1 table (line 262); the recall table still shows 6 of the 9 pairs the script computes — see new finding 4
7b	--splits subsets do not reproduce the committed per-split s.e.	not fixed	one rng (line 542) threaded through every group in order
7c	~half the bootstrap work redundant (res_a/res_b)	not fixed	lines 642–643
7d	Scored.gt_pano dead	not fixed	built (285, 294), stacked (316), never read
7e	test_committed_json_matches_the_doc pins none of the headline numbers	partly	now pins the 28-pair matrix shape and three inequalities; still none of 0.0042 / 0.0021 / 0.0059 / 0.0063 / 0.0117 / 0.0109
7f	(prediction_confidence(p) or -1e9)	not fixed	tests/test_benchmark_power_135.py:56
7g	observed_and_se(paired=…) assumes pano order without asserting	not fixed	function unchanged

---

New findings on the unreviewed delta

1. High — the rung's pre-registered test was not run; "tied at every epoch" is read against a different bar

The pre-registration (docs/stage2_cosine_rung_135.md lines 79–82, unchanged since 2026-08-18) says: paired against Run A's checkpoints via benchmark_power_135.py, each pair read against its own bootstrap standard error (B = 5,000), significance |Δ| / s.e. ≥ 1.96. The Results section (lines 341–343) instead compares the largest |Δ|, 0.0042, to #138's MDE of 0.0063 and calls every epoch a tie.

Those are different bars. The MDE is 2.80 × s.e. at 80% power; the pre-registered significance bar is 1.96 × s.e., 1.43× lower. An effect below the MDE is not thereby non-significant. Using the only paired s.e. this branch has measured — Run A epoch pairs, 0.0016–0.0029 — epoch 7's Δmax-F1 of +0.0042 gives z between 1.4 and 2.6, straddling 1.96; epoch 8's +0.0030 gives 1.0 to 1.9. The pre-registered verdict at epoch 7 is undetermined, not "tie".

Nothing on the branch can run the test as written. The comparison needs per-panorama detections for the cosine checkpoints, and none are committed: docs/data/cosine_rung_135_manual_gold/ holds downsampled PR curves only. The doc's own "Exact commands" (lines 175–177) prescribe dump_peaks_from_cache.py … --verify then benchmark_power_135.py --splits manual_gold for exactly this step, and neither was done (and the first would not have worked — finding 2).

Two more pre-registered items are missing for the same reason:

• Secondary test (lines 94–101): the cosine arm's own epoch 3 → 8 max-F1 change, "≥ 0, or a decline significantly smaller than Run A's". From summary.csv it is 0.920581 → 0.915423 = −0.0052, against Run A's −0.0066. That is a decline, so the first clause fails; the second is unanswerable without the paired s.e. The doc never states the outcome.
• The gate (lines 103–111): by the rule as written this is "tie on both", which the pre-registration says is a judgment call "recorded as a judgment call rather than dressed as a rule". The Results section does not record it either way; "What this does and does not settle for Run B" (lines 387–398) reads as the settled answer.

Also promised "alongside" and absent: the full 8-epoch paired matrix against Run A (line 113–116).

Fix: if the makelab2 eval cache for the rung survives (cosine_rung_135/evaluate_cache, the doc does not say), dump the eight epochs with a corrected dump_peaks_from_cache.py (finding 2), commit them under docs/data/cosine_rung_135_detections/, extend benchmark_power_135.py to score the cross-arm pairs (same epoch, and cosine 3 vs 8), and report primary, secondary and gate per the rule. If the cache is gone, say so next to the table, describe the MDE reading as a deviation from the pre-registration, and downgrade "tied at every epoch" to "below the 80%-power MDE at every epoch; the pre-registered per-pair test could not be run because …".

2. Medium — dump_peaks_from_cache.py cannot produce the rung's dumps as the doc says it will

• fingerprint_labels (line 91) hardcodes run_a_epoch_{N} whatever --summary-csv is passed. With the default summary the rung's fingerprints are unknown, so every rung dump would be labelled ckpt_<fingerprint>, land in docs/data/run_a_84_detections/ (the default --out-dir) beside Run A's, be skipped by --verify without a message (line 216, fingerprint in committed), and be invisible to benchmark_power_135.py, whose loader keys on the run_a_epoch_ prefix (RAMPNET_PREFIXES). Passing the rung's --summary-csv mislabels them run_a_epoch_N and overwrites Run A's committed dumps.
• The module docstring's usage (line 24) says --out-dir benchmark/model_detections — the directory the same file's --out-dir help (lines 146–149) and the doc explain must not be used, because rampnet/roster.py asserts on it.
• The signature stamps "exclude_border": False as a literal rather than reading it from the extractor.

Fix: a --label-prefix (or derive the label from a column in the summary), --verify fails loudly when a fingerprint is not in the summary, docstring corrected.

3. Medium — numbers in the doc that are not the committed JSON's

The doc claims every derived number is in benchmark_power_135.json. These are not:

doc	says	JSON
line 127, y11x − y26 ΔF1	+0.1119	0.111666 → +0.1117
line 127, ΔmaxF1	+0.1492	0.148884 → +0.1489
line 127, c	123	124
line 177, seam share	9 of 356	10 of 357
line 143, adjacent-epoch discordance	2.5–4.2%	2.50–4.26% (7v8)
line 152, widest pairs	6.5–6.6%	6.48% (1v8), 6.66% (1v7)
line 145, all 28	2.5–6.6%	6.66%
lines 25, 145, MDE upper	0.0081	2.8016 × 0.002926 = 0.0082

The first four are the pre-#140 values of the one proxy pair the seam wrap moved (the same mechanism as the #148 spin-out); the JSON was regenerated after the merge and the doc was not. In the rung doc, line 313 says 0.961% and line 314 says 3.979% where the committed CSV and the table beside them say 0.962% and 3.980% — the trap lines 286–288 of the same file warn about.

Fix: regenerate the affected rows from the JSON/CSV, and have test_committed_json_matches_the_doc pin the headline values (prior 7e) so the next regeneration cannot leave the prose behind.

4. Medium — the recall table shows six of nine pairs, and the missing three include the only "not resolvable"

The script computes nine recall pairs; the doc's recall table (lines 241–248) prints six. Omitted: 1v8 (+0.0043, required discordance 1.7%, not resolvable), 3v8 (−0.0153, resolvable) and 6v7 (−0.0199, resolvable). The max-F1 table directly below prints all nine. Whatever the reason, a table that drops the one verdict that cuts against the paragraph under it reads as selection. Print all nine.

5. Medium — the rung doc says nothing above Results was edited, and it was

Line 3: "Everything above Results is as written on 2026-08-18 and has not been edited." git diff 2cdb01c HEAD -- docs/stage2_cosine_rung_135.md shows two edits above that line: d9c4fcd (2026-08-20) rewrote lines 56–59 ("requeued twice" → "five times across its two job ids, measured with sacct -D"), and cb2fb59 filled the launch record. Both are factual and neither touches the decision rule, but a pre-registration's value is exactly that its no-edit claim is true. Fix: say what was edited above the line, and that the decision rule, comparison and outcome table were not.

6. Low — provenance and cost bookkeeping around the rung

• GPU-hours (560.9) and money ($0, ckpt-all) are recorded, as the repo rule requires. They are in no committed ledger: the compute ledger is on Record time as well as money for every non-free run (#143) #147's branch and not merged. Say "pending Record time as well as money for every non-free run (#143) #147's compute_log.jsonl" next to the number so the gap is visible.
• "21 restarts", "35.06 h" and "Run A was requeued five times" come from sacct -D with no dump committed here (Record time as well as money for every non-free run (#143) #147's sacct_klone_2026-08-19.txt ends two days before the rung finished). From the committed events a clean clone can count 18 incarnation files (11 with steps), not 21. State the source.
• The LR-verification block (lines 244–250) has no reproduce command. It is one line: python scripts/analysis/check_lr_schedule_135.py --events-dir stage_two/cosine_rung_135_events. I ran it; the output is identical.
• The one input finding 1 needs — the makelab2 evaluation heatmap cache for the rung — is not mentioned anywhere. The doc records where the checkpoints are (klone and makelab2, 8.6 GB) and that publishing them is an HF-storage decision; it should say the same for the cache.

7. Low — the epoch boundary is the one window the checkpoint-interval fix does not cover

latest_checkpoint.pth is refreshed every --checkpoint-interval-steps mid-epoch and, at epoch end, only after the validation pass, the 1 GB epoch checkpoint and the peek image (train.py lines 619–627). The committed events show what that costs: incarnations 5 and 6 both logged through step 9378 (end of epoch 1) and the next incarnation resumed at 9001 both times — the whole validation pass plus 378 steps lost, twice. fa02b37's 200-step interval does not shorten that window; one latest_checkpoint.pth write before validation would. Not a correctness defect; noting it because the commit message claims the livelock is fixed and the events show a second mechanism.

8. Low — the PR body's counts are stale

Suite 1236 → 1361 passed; "closes nothing on its own" and the 30-epoch recommendation predate the rung result. See decisions.

---

Decisions for Jon, not fixes

1. The PR body and title describe the first commit. The rung — a ~560 GPU-hour experiment with a pre-registered gate — is now the largest thing on the branch and the body does not mention it. The body should lead with the rung outcome and, per the pre-registration's own words, record the Run B gate decision as a judgment call: primary tie, secondary not met as written, so "proceed / do not proceed" is yours to state. The old recommendation ("run the 30-epoch arm, not 60") should be re-stated or withdrawn in light of it.
2. Whether to spend makelab2 time on finding 1. Dumping eight cached epochs is CPU-only and took 4 minutes for Run A; the question is whether the cache still exists. If it does, the pre-registered test costs nothing. If not, the deviation has to be written down.
3. Whether to publish the rung checkpoints (8.6 GB) alongside Run A's, which the doc already flags as the same open decision.

What holds up

• The stateless schedule is correct and was applied. step_index = global_step + i; the mid-epoch checkpoint stores step_index + 1 and resume_offset + i + 1; the epoch end adds len(train_loader) = epoch_length − skip, so global_step lands on (e + 1) × 9378 regardless of where the resume fell. test_resumed_epoch_lands_on_the_same_global_step pins that arithmetic. The applied rate reproduces from the committed events over 100% of the run to 4.5e-13, across ten resume boundaries at 0.993, 0.965, 0.694, 0.655, 0.166 and 0.00046 × peak — none at 1.000.
• The paper recipe is untouched. --lr-schedule defaults to constant, the param_group['lr'] write is guarded by != 'constant', and both are tested.
• ResumeSkipSampler is the right fix and the right test. epoch_length is frozen so the cosine horizon cannot shrink on resume; the equivalence test against the old discard loop is the property that matters; batch_idx_in_epoch is broadcast so every rank skips the same count.
• The rung's numbers re-derive. Every cell of the manual_gold table and the AP table from summary.csv; the auto-val CSV byte-identical from the committed events; both runs' event sets SHA-pinned; the eight rung checkpoint fingerprints tie rows to weights.
• The seam re-scoring is pinned properly. Both matchers checked, the single 0.000264 move asserted rather than tolerated, and the per-pair s.e. now come from the same matcher as the point estimates.
• The launcher changes one thing against Run A, guards world size, and takes the checkpoint interval explicitly with a test that the default did not drift.

Fix list

• [F1] High — Run the pre-registered rung test (new 1): dump the eight cosine epochs from the makelab2 cosine_rung_135/evaluate_cache with the corrected dump script, commit to docs/data/cosine_rung_135_detections/, extend benchmark_power_135.py with a cross-arm block (same-epoch pairs, cosine 3 vs 8, full 8×8 matrix), report primary/secondary/gate in docs/stage2_cosine_rung_135.md per lines 84–111. If the cache is gone, write the deviation next to the table and reword "tied" as "below the MDE; per-pair test not run because …". — files: scripts/analysis/dump_peaks_from_cache.py, scripts/analysis/benchmark_power_135.py, docs/stage2_cosine_rung_135.md, docs/data/cosine_rung_135_detections/, tests/test_benchmark_power_135.py
• [F2] High (prior 1) — Unpaired rows measure RampNet at an effective 0.55 on nine splits: switch the reference to an untruncated arm or state the truncation beside every affected number (unpaired table, both POOLED rows, the self_pair and rampnet_1pass:rampnet city rows), recompute the pooling gain, and replace "7%" in the doc and PR body. — files: scripts/analysis/benchmark_power_135.py, docs/stage2_run_b_power_135.md, docs/data/benchmark_power_135.json
• [F3] Medium (prior 4) — Headline table rows 1 and 4: use max_f1.se (0.0041 → MDE 0.0114; 0.0039 → 0.0108) or relabel the header. — files: docs/stage2_run_b_power_135.md
• [F4] Medium (prior 2) — self_pair max-F1 se: 0.0 is a non-function of the shift, not zero uncertainty: drop max-F1 from that block or make it threshold-aware, and say which in the JSON. — files: scripts/analysis/benchmark_power_135.py, docs/data/benchmark_power_135.json
• [F5] Medium (prior 5) — Add a status pointer in docs/stage2_epoch_curve_84.md to Run B (#84): the gate passed — decide whether to spend 1,675-3,350 GPU-hours on the annealed arm #135's paired re-read (plateau 2–6, decline at 7–8). — files: docs/stage2_epoch_curve_84.md
• [F6] Medium (prior 6) — Condition the recall-table verdicts on the discordance assumption the same way the max-F1 bracket is. — files: docs/stage2_run_b_power_135.md
• [F7] Medium (new 2) — dump_peaks_from_cache.py: label prefix from an argument or the summary, loud failure on --verify when a fingerprint is absent, docstring --out-dir corrected, exclude_border read from the extractor. — files: scripts/analysis/dump_peaks_from_cache.py
• [F8] Medium (new 3) — Regenerate the stale numbers from the committed JSON/CSV (proxy row 3, seam row 3, discordance ranges, MDE upper bound, 0.961/3.979) and pin the headline values in test_committed_json_matches_the_doc (prior 7e). — files: docs/stage2_run_b_power_135.md, docs/stage2_cosine_rung_135.md, tests/test_benchmark_power_135.py
• [F9] Medium (new 4) — Print all nine recall pairs, including 1v8 not resolvable. — files: docs/stage2_run_b_power_135.md
• [F10] Medium (new 5) — Replace the "has not been edited" sentence with what was edited above Results and what was not. — files: docs/stage2_cosine_rung_135.md
• [F11] Low (new 6) — Add the check_lr_schedule_135.py reproduce command under the verification block; "pending Record time as well as money for every non-free run (#143) #147" beside the 560.9 GPU-h; state that restarts/wall-clock come from sacct -D and are not re-derivable from the repo; state whether the rung's eval cache survives. — files: docs/stage2_cosine_rung_135.md
• [F12] Low (prior 7b–7g) — --splits per-group rng seeding; drop the res_a/res_b full bootstraps; remove dead gt_pano; if c is None instead of or -1e9; assert pano order in observed_and_se(paired=…). — files: scripts/analysis/benchmark_power_135.py, tests/test_benchmark_power_135.py
• [F13] Low (new 7) — Optional: write latest_checkpoint.pth once before validation at epoch end; if not done, note the window in the ResumeSkipSampler docstring. — files: stage_two/train.py
• [F14] Low (new 8) — Refresh the suite count and the stale claims in the PR body when Jon rewrites it (decision 1). — files: PR body

🤖 Generated with Claude Code (claude-fable-5-1)

[jonfroehlich]

Third pass: the second review's findings were not addressed, and the decision's own artifact contradicts the doc's headline

Re-ran everything from the branch head (4171a6e) in a clean detached worktree on Windows.

What reproduced. Full suite 1371 passed, 1 skipped. run_b_gate_135.py --check passes and the committed docs/data/run_b_gate_135.json matches a fresh run byte-for-byte. benchmark_power_135.json regenerates identically from the documented command (~35 min here). Every cell of the rung's manual_gold and AP tables re-derives from the two committed summary.csv files, and the auto-val table from stage2_cosine_rung_135.csv.

What did not. 4171a6e is a content commit, not a fixes pass, and it shows: of the second review's 14 items, one is partly addressed, one is partly addressed, and twelve are untouched. More seriously, the artifact 4171a6e added — run_b_gate_135.json — records "epoch": 7, "would_reach_significance_at_favourable_se": true, while the document three sections above it says "the two arms are tied at every epoch" and "No epoch separates the two arms." The commit's own message flags epoch 7 correctly; the document does not.

I checked each prior finding against the code rather than taking it on trust. None of the second review's findings was fabricated — every one reproduced, including the four-decimal drifts.

---

Status of the prior findings, at 4171a6e

id	finding	status	evidence
F1	rung's pre-registered test not run; "tied" read against the wrong bar	partly fixed	4171a6e applies the gate, states the s.e.-bracket deviation in the script docstring, and flags epoch 7 in the artifact and in "What is not being claimed". The prescribed reword did not happen: the status line, the Results status and the manual_gold headline all still say "tied at every epoch" against the 0.0063 MDE. See new finding 1.
F2 (prior 1)	nine bundles truncated at 0.55; "pooling is worth 7%"	not fixed	measured on the branch: all nine cities hold 0.0% of detections below 0.55 (minima 0.5501–0.5607); manual_gold reaches 0.0501 with 28.8% below. max_f1 == f1 exactly on 8 of 9 cities and on POOLED cities. Doc lines 34/103/109/336 still say 7% under a "0.30" header.
F3 (prior 4)	headline table mixes F1 and max-F1 s.e.	not fixed	rows 1 and 4 are unpaired.manual_gold.f1.se = 0.004190 and POOLED all.f1.se = 0.003897; the max-F1 values are 0.004060 and 0.003872 (MDE 0.0114, 0.0108).
F4 (prior 2)	self_pair max-F1 se: 0.0 by construction	not fixed	metrics() still returns max_f1(s, weights, n_gt), no threshold; 24 of 24 rows carry {"observed": 0.0, "se": 0.0}.
F5 (prior 5)	stage2_epoch_curve_84.md has no pointer to #135	partly fixed	4171a6e adds one line, at the Run B gate section. The status block at the top still states the superseded curve shape ("steps up once and is then flat") with no pointer, and that is the sentence anyone reads first.
F6 (prior 6)	recall-table verdicts stated flat	not fixed	unchanged.
F7 (new 2)	dump_peaks_from_cache.py cannot produce the rung's dumps	not fixed	fingerprint_labels hardcodes run_a_epoch_{N}; --verify skips silently on fingerprint in committed; the docstring still recommends --out-dir benchmark/model_detections, which the same file's --out-dir help forbids; exclude_border stamped as a literal.
F8 (new 3)	doc numbers that are not the artifact's	not fixed	verified all: +0.1119 / +0.1492 / c = 123 vs the JSON's 0.111666 / 0.148884 / 124; "9 of 356" vs 10 of 357; discordance 2.5–4.2 / 2.5–6.6 / 6.5–6.6 vs 2.50–4.26 / 2.50–6.66 / 6.48–6.66; MDE upper 0.0081 vs 0.0082; rung doc 0.961/3.979 vs 0.962/3.980.
F9 (new 4)	recall table prints 6 of 9 pairs	not fixed	the JSON holds all nine; 1v8 (not resolvable), 3v8, 6v7 omitted.
F10 (new 5)	"has not been edited" is untrue	not fixed	cb2fb59 and d9c4fcd both edit above the Results line; confirmed with git diff.
F11 (new 6)	provenance/cost bookkeeping	not fixed	rung doc has zero occurrences of check_lr_schedule, #147, ledger, compute_log, or the rung's eval cache.
F12 (prior 7b–7g)	rng, res_a/res_b, dead gt_pano, or -1e9, pano-order assert	not fixed	all five confirmed in place.
F13 (new 7)	epoch-boundary checkpoint window	not fixed	train.py still writes latest_checkpoint.pth after validation + the 1 GB epoch checkpoint + the peek image.
F14 (new 8)	PR body stale	not fixed	see new finding 2 — it is worse than stale.

---

New findings

High

1. docs/stage2_cosine_rung_135.md:3, :211, :342 — the document's headline claim is contradicted by the artifact committed alongside it, because it is read against a bar the pre-registration did not name.

The pre-registration (line 98) is |Δ| / s.e. ≥ 1.96. The Results section instead compares the largest |Δ| to #138's MDE of 0.0063, which is 2.80 × s.e. at 80% power — 1.43× looser. Using the s.e. bracket this branch measured (0.0016–0.0029), epoch 7's +0.0042 gives |z| = 1.45 to 2.64, and docs/data/run_b_gate_135.json records exactly that: "would_reach_significance_at_favourable_se": true. The primary (epoch 8, +0.0030, |z| = 1.02–1.86) is genuinely a tie at both ends, which is the reading the decision rests on and which is robust. Epoch 7 is not a tie; it is undetermined.

Failure scenario: the paper cites "the cosine rung found manual_gold tied at every epoch" as the evidence that annealing does nothing at 8 epochs. A reviewer opens the JSON in the same directory, finds a true flag on epoch 7, and the strongest claim in the document is the one that does not survive contact with its own data. The decision itself is unaffected — reason 3 (n=1 against unmeasured seed variance) is independent of the tie and would still bind if epoch 7 were significant — which is exactly why overstating the tie was unnecessary.

2. PR body — still recommends running Run B.

The body's ## Recommendation section reads "Run the 30-epoch arm; not 60 on spec." and closes with "Input to the #135 decision; closes nothing on its own." The header note added on top says the opposite of both. docs/stage2_run_b_power_135.md got a SUPERSEDED marker in 4171a6e; the body did not. It also still quotes the pre-#140 z = 3.9 / 3.0 / 2.8, the uncorrected 7% pooling figure, "discordance 2.5–6.6%", and "Suite 1236 passed" (measured: 1371).

Failure scenario: the PR body is what shows in the merge commit and in every GitHub notification. Someone budgeting cluster time reads a 1,675–3,350 GPU-hour recommendation from the very PR whose purpose was to decide against it. Left for you — I do not rewrite your PR bodies — but this is the highest-visibility surface the decision has.

Medium

3. docs/stage2_cosine_rung_135.md:378 and scripts/analysis/run_b_gate_135.py:70 — the Run A epoch-7 max-F1 used here is the value this same PR pins as superseded.

The rung table and the gate both read Run A's max-F1 from summary.csv, where epoch 7 is 0.911009. tests/test_benchmark_power_135.py:198 pins the post-#140 value as SEAM_FIXED_MAX_F1 = {7: 0.910745}, and docs/stage2_run_b_power_135.md prints 0.9107. Two committed documents in one PR give different values for the same checkpoint's same metric with nothing reconciling them. It is defensible — both arms were scored at dc7450e, so the comparison is internally consistent — but that reasoning is nowhere in the file, and the s.e. bracket standing beside those point estimates is post-#140, which is the mixed provenance the 2026-08-20 comment said had been eliminated.

Failure scenario: someone quotes Run A epoch 7 from whichever doc they opened, and two published numbers for one checkpoint differ by 0.000264 with no explanation — which is 4% of the MDE the whole analysis is stated in.

4. docs/stage2_cosine_rung_135.md:487 — the reopening condition points at a file that does not exist.

docs/seed_variance_51_135.md is not on this branch (ls docs/ | grep seed → nothing); the seed campaign lives on PR #155. The reopening condition is the load-bearing half of "not being run" rather than "will never be run", so a dead link there is worse than elsewhere.

5. docs/stage2_cosine_rung_135.md:461 — the decision's decisive argument turns on an unmeasured number, and the derivation is not given where it is used.

Reason 3 is the argument the commit message itself calls "THE ARGUMENT THAT WAS NOT AVAILABLE WHEN RUN B WAS SPECIFIED", and it works by putting the plausible ~0.003 effect below "~0.01 max-F1, measured but not attributable". That ~0.01 is not measured anywhere: stage2_run_b_power_135.md:352 sets it by analogy to the #51-scale ~0.02 effect, over a seed variance the same section says nothing in the repo measures. The doc is honest that seed variance is unmeasured, but the threshold itself reads as a quantity when it is an assumption, and it is the one the decision hangs on.

6. docs/stage2_cosine_rung_135.md:468 — "+0.115 F1, measured" is not re-derivable from this branch.

benchmark/ here holds ten bundles and no laurens_*; #151's arms are on another branch. Quoting it as measured opportunity cost, in a document whose job is to justify not spending money, without saying where it can be checked, is the gap the replication rule exists to close.

Low

7. docs/stage2_cosine_rung_135.md:25 — the motivation quotes pre-#140 z values (3.9 / 3.0 / 2.8) that docs/stage2_run_b_power_135.md now reports as 4.0 / 3.0 / 3.0. Keeping the pre-registration unedited is the right instinct; leaving two committed documents disagreeing in the third digit with no footnote is not.

8. docs/stage2_run_b_power_135.md:412 — "Every derived number in this document is in docs/data/benchmark_power_135.json" is false, still. Prior finding 3 flagged the sentence; the number it was flagged for is gone but the sentence stands, and there are now four classes of exception (the seam re-scoring table, the untruncated-pooling comparison, the 1.96-bar arithmetic, and the retained historical interpolation).

---

What holds up

Stating this because a defect list on its own misrepresents the work.

• The decision is right, and it is right for a reason that survives every finding above. Run B at n=1 cannot be told from a seed draw at any length; that argument does not depend on the tie, on the pooling figure, or on which end of the s.e. bracket you read. The three supporting reasons are independent of each other, and the reopening condition is concrete and already in flight.
• run_b_gate_135.py is the right shape for a decision artifact. It evaluates the pre-registered rule from the committed summaries instead of restating it in prose, it reports the branches that did not fire, it flags the post-hoc trap in its own output, and tests/test_run_b_gate_135.py exercises the three verdicts that did not occur — which is the only way an implementation that always emits one verdict can be tested at all.
• The deviation from the pre-registration is stated, not hidden. The script's docstring says plainly that a true cross-arm bootstrap is impossible without the cosine arm's per-panorama detections and that the Run A bracket is standing in. That is the honest form.
• The training-side work is verified over the whole run. The stateless schedule reproduces from the committed events to 4.5e-13 across ten resume boundaries, none at 1.000× peak; the event set is SHA-pinned; --lr-schedule defaults to constant and the paper recipe is untouched, with tests for both.
• The gate correctly refuses to make the call. "A tie on both is not an automatic cancellation" was written before the numbers and is honoured after them; the judgment is recorded as a judgment.

Fixes for everything above except new finding 2 follow in the next commit.

🤖 Generated with Claude Code (claude-opus-5)

[jonfroehlich]

Review fixes

cc94b34. Suite 1,373 passed, 1 skipped (was 1,371; two tests added). Everything in the list above except new finding 2 is fixed.

The two that mattered

The headline claim, corrected against the bar the pre-registration named. docs/stage2_cosine_rung_135.md now says what the numbers say: the pre-registered primary (epoch 8, +0.0030, |z| = 1.02–1.86) is a tie at both ends of the measured s.e. bracket, six of the other seven epochs are ties at both ends, and epoch 7 is undetermined — |z| = 1.45 to 2.64, clearing 1.96 at the favourable end, exactly as run_b_gate_135.json already recorded. The per-epoch z table is printed in the document rather than left in the JSON. The deviation from the pre-registration (no cross-arm paired bootstrap, because the cosine arm's per-panorama detections are not committed) is stated where the numbers are, together with the fact that the pre-registration's own "Exact commands" could not have run the test as written — dump_peaks_from_cache.py was broken for any arm but Run A, and benchmark_power_135.py still has no cross-arm block.

The decision is unchanged, and the correction shows why it was robust. Reason 3 — Run B at n=1 cannot be told from a seed draw — holds whether or not epoch 7 separates. Overstating the tie bought nothing.

The bundle truncation now travels with every number it qualifies. Measured and stated: all nine city bundles hold 0.0% of their detections below 0.55 (minima 0.5501–0.5607) against manual_gold's 0.0501 with 28.8% below. Re-ran the analysis against the one untruncated arm in committed data (--reference rampnet_1pass, ~35 min CPU) and put the comparison in the doc as a table:

	committed (cities cut at 0.55)	rampnet_1pass (untruncated)
manual_gold MDE	0.0117	0.0121
POOLED cities MDE	0.0219	0.0190
POOLED all MDE	0.0109	0.0105
pooling gain	7%	14% (13.6%)

rampnet_1pass is single-pass and missing seam detections, so 14% bounds the correction rather than being it — there is no clean uniform-0.30 RampNet arm in the repo, which is now said next to the number. The conclusion is unchanged on either reading: pooling is not a lever.

The artifact now records reference_min_confidence and protocol_threshold_binds per split and truncated_members per pooled row, the script prints a warning and marks affected rows, and test_committed_json_records_the_bundle_truncation pins it. The reason this sat undetected through two reviews is that the only evidence was indirect (max_f1 == f1 exactly, self_pair b = c = 0); it is now direct.

benchmark_power_135.json regenerates with ZERO changed values — 80 keys added, 48 removed (self_pair's degenerate max-F1), no number moved. Every figure both documents quote still stands, verified key-by-key against HEAD~1.

The rest of the list

id	what changed
F3	headline table rows 1 and 4 now max_f1.se: 0.0041/0.0114 and 0.0039/0.0108.
F4	self_pair max-F1 is null with a max_f1_note, not {"observed": 0.0, "se": 0.0}. The module docstring says the block bounds F1 only.
F5	stage2_epoch_curve_84.md's status block now carries the #135 amendment — plateau 2–6, not 2–8 — which is where a reader looks first. The gate-section pointer stays.
F6	recall verdicts carry the same conditional the max-F1 table has: the 2.1–9.1% range is three cross-detector pairs, none epoch-vs-epoch, and the two rows near the bottom of it (1v8, 5v8) sit inside the assumption rather than clear of it.
F7	--label-prefix added; --verify fails loudly on a fingerprint the summary does not contain instead of checking nothing; exclude_border read from the extractor; docstring no longer recommends the directory the same file's help forbids.
F8	all corrected: +0.1117 / +0.1489 / c = 124, 10 of 357, 2.50–4.26 / 2.50–6.66 / 6.48–6.66, MDE upper 0.0082, 0.962 / 3.980. test_committed_json_matches_the_doc_headline_numbers pins all eight headline values plus five MDEs, so the prose cannot drift again.
F9	all nine recall pairs print, including 1v8 not resolvable.
F10	replaced with what was edited above Results (cb2fb59 filled the launch record, d9c4fcd fixed the requeue count) and what was not (the rule, the tests, the gate, the outcome table). This review's own edit is listed too.
F11	check_lr_schedule_135.py reproduce command added; 560.9 GPU-h marked pending #147's compute_log.jsonl; the 21 restarts / 35.06 h marked as sacct -D with no committed dump (a clean clone can count 18 event files, 11 with steps); the rung's eval cache status stated as unknown, with the recovery cost if it is gone (~1.7 GPU-h).
F12	dead Scored.gt_pano removed; observed_and_se(paired=…) asserts shared pano order; if c is None instead of or -1e9.
F13	epoch-boundary window described in the ResumeSkipSampler docstring, with the two committed incarnations that show it.
new 3	Run A epoch-7's two values (0.9110 / 0.9107) reconciled in both documents: both arms scored pre-#140 so the comparison is consistent, and the wrap moves Run A down, so the epoch-7 Δ is if anything understated.
new 4	dangling docs/seed_variance_51_135.md → PR #155.
new 5	the "~0.01 measured but not attributable" threshold marked as a working assumption, with the note that the argument does not need it to be right.
new 6	#151's +0.115 F1 marked as not re-derivable from this branch.
new 7	pre-#140 z values in "Why this run" footnoted against the power doc's 4.0/3.0/3.0.
prior 3	"Every derived number is in benchmark_power_135.json" replaced with the four named exceptions.

Deliberately not fixed

• F12's --splits rng seeding and the redundant res_a/res_b bootstraps. Both are real. Both were left, with the reason written into the code: one Generator is threaded through the loop in order, so removing a draw or re-seeding per group shifts the stream and moves every standard error these documents quote — for no gain in correctness, in a PR whose output is a decision. The --splits caveat is documented in the doc's Reproduce section instead; the redundancy costs ~10 minutes of CPU on a run that happens when the analysis changes.
• F13's actual code change. Writing latest_checkpoint.pth before validation is a behavioural change to a training script no test on this branch exercises end to end, and the run it would have helped is finished. Documented as a known gap.
• F14 / new finding 2: the PR body. It still reads "Run the 30-epoch arm; not 60 on spec" and "closes nothing on its own", both opposite to the decision, plus the pre-Wrap the 360 seam: one geometry module, an adjudicated ground truth, and two retractions #140 z values, the uncorrected 7%, and a stale suite count. docs/stage2_run_b_power_135.md got its SUPERSEDED marker; the body is yours to rewrite. It is the highest-visibility surface the decision has.

One thing this pass could not do

The pre-registered per-pair test still has not been run, and cannot be from the repo. It needs the cosine arm's per-panorama detections, which need the rung's makelab2 evaluation heatmap cache — whose survival nobody has checked. If it is intact the test is CPU-only and free once benchmark_power_135.py grows a cross-arm block; if it is gone, ~1.7 GPU-hours regenerates it from checkpoints that do still exist. Either way epoch 7 stays undetermined until then, and the document now says so instead of calling it a tie.

🤖 Generated with Claude Code (claude-opus-5)