[umer12x]

Overview

This PR implements training stabilization, variable-binding contrast minibatches, token-weighted loss, reproducible experiments, exact match evaluation, inference gradient fixes, and Colab training workflows.

Summary of Changes (50 Commits)

1. Token-Weighted Loss (training/token_weights.py):

   • Dynamic token prefix classification (STRUCT:*: 1.0, COEF:*/EXP:*: 3.5, VAR:*/OPVAR:*: 4.0, [PAD]: 0.0).
   • TokenWeightedCrossEntropy with zero-weight masking and NaN safety fallback.
   • Comprehensive unit test coverage (tests/unit/test_token_weighted_loss.py).

2. Variable-Binding Contrast Minibatches (training/contrast.py, problem_generator.py):

   • Paired problem generation where expressions are identical while varying the target variable and derivative.
   • Generated paired splits data/splits/contrast_train.jsonl (16k) and data/splits/contrast_val.jsonl (800).
   • ContrastBatchSource deterministic cyclic pairing and batch merging.
   • Unit tests in tests/unit/test_contrast_pairs.py.

3. Verified Learning Rate Schedule (training/schedule.py):

   • Linear warmup (1,500 steps) and cosine decay to min_lr_ratio=0.05.
   • Preflight validation verify_schedule asserting expected total steps and bounds.
   • Unit tests in tests/unit/test_training_schedule.py.

4. Reproducibility & Run Manifests (training/reproducibility.py):

   • Automated environment metadata capture (OS, Python, PyTorch, CUDA device).
   • SHA256 checksums of datasets, vocabs, serializer, and configuration.
   • Seed management for deterministic training.
   • Unit tests in tests/unit/test_training_reproducibility.py.

5. Metrics & Exact Match Tracking (training/metrics.py):

   • Batched greedy sequence evaluation and stratified operation accuracy reporting.
   • Contrast pair joint accuracy metric.

6. Controlled Experiments & Publisher (training/experiments.py, scripts/):

   • Factorial 4-arm experiment runner (scripts/run_experiments.py).
   • Checkpoint selection and publishing script (scripts/publish_best_checkpoint.py).

7. Inference & Gradient Fixes (inference/solve.py):

   • Support explicit variables argument in _solve_gradient.
   • Direct gradient dictionary exposure in return payload.
   • Variable name vocabulary validation.

8. Colab Training Infrastructure:

   • Relaxed dependency pins in requirements-neural.txt for Python 3.12 / Colab GPU.
   • Interactive notebook at notebooks/colab_training.ipynb.

[vercel]

@umer12x is attempting to deploy a commit to the seno-quantum-coder's projects Team on Vercel.

A member of the Team first needs to authorize it.