Skip to content

test(history): admit two-cycle, shared and learner-reference training logs and compare a log with a runtime run - #71

Merged
FeathBow merged 5 commits into
mainfrom
test/logcomparison
Oct 20, 2026
Merged

FeathBow merged 5 commits into
mainfrom
test/logcomparison

Conversation

@FeathBow

Copy link
Copy Markdown
Owner

Why

Training-log admission was tested on one cycle with engine reference scores only, and no test compared a training log with a runtime run over more than one update. The training driver that wrote these logs no longer exists, so a synchronous and a runtime execution cannot be produced side by side from this tree. Refs #30.

What

  • The Hugging Face test model runs a two-cycle runtime run at staleness zero. The test writes the training log of the same records from the run's transcripts, adding the publication and cycle records. invar compare histories compares the log with the run through the real codec. They are equal, including the update schedule, the learner state, gradients and probabilities of each generation, and the final inference. The two generations publish different adapters, so the comparison covers the policy sequence.
  • A second run repeats the first cycle and changes one seed in the second. Against the log, generation 1 compares equal and generation 2 unequal, with different learner states and gradients.
  • The training-log tests admit a log whose learner produces the reference words, which is refused when those words are removed. They also admit a shared log in which one owner carries the inference group and the learner, releases each in turn and closes once. Without the close, or with only one role declared shared, the log is refused. A two-cycle log admits two generations with different adapters and ends at the second generation's policy.
  • The learner fixture writes an adapter, gradients and learner state that depend on the scheduled update, so consecutive generations no longer share identities. The runtime publication test now expects the second generation to describe the second update's adapter.
  • The history guide and the loop design describe training logs as the output of runs that kept no journal. The guide also gives a complete invar compare histories command for two run directories and says where the final inference digests come from.

Validation

The core suite, the Hugging Face worker suite, the type fixtures, formatting and lint pass. With the comparison of generation 2 replaced by a comparison of a generation with itself, the changed second cycle is reported as equal and the new test fails.

… logs and compare a log with a runtime run

Signed-off-by: Feathbow <feathbow@gmail.com>
… show a run comparison

Signed-off-by: Feathbow <feathbow@gmail.com>
Signed-off-by: Feathbow <feathbow@gmail.com>
… two-cycle log with distinct generations

Signed-off-by: Feathbow <feathbow@gmail.com>
…whose second cycle differs

Signed-off-by: Feathbow <feathbow@gmail.com>
@FeathBow
FeathBow merged commit ad468b7 into main Oct 20, 2026
4 checks passed
@FeathBow
FeathBow deleted the test/logcomparison branch October 20, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant