Repository navigation
Conversation
The bridge arm's separated-improvement vs G2a (-0.4385pp, p=0) is a single-seed result; the ledger's seed-variance caveat (2608.27729) applies. Seeds 2 and 3 run the identical recipe (same scan bridges, same 2.0 rung base, beta 0.1) as explicit independent draws: WO20_SEED seeds torch/numpy/python/DataLoader-workers; the unseeded seed-1 run stays as-is and is recorded as an uncontrolled draw. wo20_seed_confirm.py scores any seed's banked student preds against G2a's released preds (frontier-predictions-v1, sha-verified) with the identical evaluator + count-weighted paired bootstrap; the seed-1 calibration (must reproduce -0.4385pp CI [-0.65, -0.28]) precedes any seed-2/3 reading. Judgment pre-registered in interscript-models TODO.sota-2026/06 before launch.
…er got it transitively via transformers, the CPU scorer did not)
…agraph arrays are already pp; the paired delta/CI/p were always correct - the seed-1 calibration matched the published verdict exactly)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
train_wo20_tiny_hf.pygains theWO20_SEEDknob (torch/numpy/python/DataLoader-worker seeding) so seeds 2/3 of the WO20 bridge-arm recipe run as explicit independent draws; defaultWO20_SEED=0preserves the unseeded seed-1 behavior bit-for-bit. Seed recorded in the train log line andfinal_eval.jsonrecipe.wo20_seed_confirm.py— the WO20u M1 confirmation scorer (WO20u: M1 bridge-arm multi-seed confirmation (pre-registered) + M2 int8 release candidates staged interscript-models#444): scores a seed's banked student predictions against G2a's released predictions (frontier-predictions-v1/run-007-r7-muon-6ep-final_preds.jsonl, shaf6f60619…verified in-job) under the identical sadeed evaluator (word-count mismatches skipped pairwise) and the identical count-weighted paired bootstrap (documents = resampling unit, 10k, seed 7). Writeswo20u-m1/<run>.confirm.jsonto the isx-training bucket; resume-safe via that marker.Test plan
seed: WO20_SEED=2/3in-job log lines)