eval: accept bounded asynchronous training against a synchronous reference #18

Motivation

Staleness is part of the training algorithm, so an asynchronous run is a different run whose result has to be judged. The benefit of overlap shows in throughput and total resource use. Both need runs on CUDA with rules fixed before the results are seen.

Proposal

On CUDA, train the same workload once with d = 0 and once with d = 1 on one host with the runtime from #16; this does not wait for the remote transport. A use contract compares the d = 1 checkpoint as the candidate with the d = 0 checkpoint as the reference. The decision covers these two checkpoints and the declared use. A claim that asynchronous training does not degrade in general would need independent training runs as its units, and more evaluation prompts cannot replace repeated training.

Before either run starts, the following are fixed and retained with the runs: the training budget and whether it counts updates, tokens or time; the evaluation protocol; the checkpoint selection rule; the quality margin and its reason; the confidence settings; the basis for isolating acceptance data from training; and what happens when a run fails. The performance comparison fixes the resources on both sides, and total cost includes waiting, retries, reference scoring and artifact transfer.

Acceptance

• Every setting listed above is fixed before the runs and retained with them.
• The use contract returns its decision for the d = 1 checkpoint against the d = 0 checkpoint: admitted, refused or insufficient evidence, reported as it comes out.
• Quality, throughput and total resource cost are reported for the recorded configurations under the fixed resources.
• The documentation states that the core fixes the logical assignment and use of samples, and that identical weights additionally require evidence of the workers' numerical invariance.