This repository contains the scripts needed to reproduce the paper "You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories":
- Verify Finding 1: RLVR updates are extremely low-rank (rank-1) across the training trajectory.
- Verify Finding 2: the rank-1 coefficient evolves linearly with training step.
- Run the RELEX method end-to-end: fit a rank-1 + linear model on a partial-training prefix, extrapolate to a target step, build a predicted checkpoint, evaluate on both in-domain and out-of-domain benchmarks.
conda create -y -n relex python=3.11
conda activate relex
pip install -r requirements.txtEnd-to-end for any backbone:
bash run_qwen2.5-math-1.5b.sh # Qwen2.5-Math-1.5B
bash run_qwen3-4b-base.sh # Qwen3-4B-Base
bash run_qwen3-8b-base.sh # Qwen3-8B-BaseOr run the steps individually (Qwen2.5-Math-1.5B example below):
First materialize the first 75 step branches of the released RLVR trajectory (enough for the RELEX pipeline; raise --num_steps if you want more):
python scripts/download_rlvr_trajectory.py \
--hub_repo relex-rlvr/RLVR-Qwen2.5-Math-1.5B \
--num_steps 75 \
--output_dir ./rlvr_traj/Qwen2.5-Math-1.5BThen compute the per-step deltas:
python scripts/precompute_deltas.py \
--base_model Qwen/Qwen2.5-Math-1.5B \
--checkpoint_dir ./rlvr_traj/Qwen2.5-Math-1.5B \
--output_dir outputs/deltas/Qwen2.5-Math-1.5B \
--max_checkpoints 75 \
--num_workers 8python scripts/svd_extrapolation.py \
--base_model Qwen/Qwen2.5-Math-1.5B \
--delta_dir outputs/deltas/Qwen2.5-Math-1.5B \
--history_steps 75 \
--rank 1 \
--fit_method linear \
--mode extrapolate \
--target_steps 500 \
--output_dir outputs/relex/Qwen2.5-Math-1.5B/cutoff75 \
--svd_cache_dir outputs/svd_cache/Qwen2.5-Math-1.5B/cutoff75Note --svd_cache_dir is strongly recommended — it lets later calls with different --target_steps reuse the SVD cache without recomputation.
Tests whether the weight dynamics are truly low-rank — projects each observed delta into the rank-r subspace and back, with no temporal fitting. If a reconstructed checkpoint's eval accuracy matches the original, the rank-r subspace captures the essential training signal.
Note. Reconstruction over the full 500-step trajectory needs all
step_1..step_500branches ofrelex-rlvr/RLVR-Qwen2.5-Math-1.5B. Re-run Step 1 with both--num_steps 500(download script) and--max_checkpoints 500(precompute_deltas) to materialize the full trajectory and its deltas locally.
python scripts/svd_extrapolation.py \
--base_model Qwen/Qwen2.5-Math-1.5B \
--delta_dir outputs/deltas/Qwen2.5-Math-1.5B \
--history_steps 500 --rank 1 \
--target_steps 100,200,300,400,500 \
--mode reconstruct \
--output_dir outputs/reconstruct/Qwen2.5-Math-1.5B/rank1python scripts/plot_svd_coefficients.py \
--delta_dir outputs/deltas/Qwen2.5-Math-1.5B \
--history_steps 75 --rank 1 \
--output_dir plots/svd_coefficients/Qwen2.5-Math-1.5BIn-domain (MATH):
python scripts/eval.py \
--model_name relex-rlvr/RELEX-Qwen2.5-Math-1.5B \
--datasets TianHongZXY/MATH \
--output_dir eval_results/RELEX-Qwen2.5-Math-1.5B \
--temperature 0.0 --top_p 1.0 --top_k -1 \
--max_tokens 4096 --num_generation 1 \
--batch_size 1000Out-of-domain (AIME25, AIME26, HMMT25, AMC23, OlympiadBench):
for DS in TianHongZXY/AIME2025 MathArena/aime_2026 \
MathArena/hmmt_feb_2025 TianHongZXY/amc23 \
zwhe99/simplerl-OlympiadBench; do
python scripts/eval.py \
--model_name relex-rlvr/RELEX-Qwen2.5-Math-1.5B \
--datasets $DS \
--output_dir eval_results/RELEX-Qwen2.5-Math-1.5B \
--temperature 0.7 --top_p 0.8 --top_k -1 \
--max_tokens 4096 --num_generation 8 \
--batch_size 1000
doneAll paper artifacts are released on the 🤗 relex-rlvr Hub organization:
| Repo | Contents |
|---|---|
RLVR-Qwen2.5-Math-1.5B |
RLVR trajectory: main = step-500, branches step_1..step_500 |
RLVR-Qwen3-4B-Base |
RLVR trajectory: main = step-500, branches step_1..step_75 |
RLVR-Qwen3-8B-Base |
RLVR trajectory: main = step-500, branches step_1..step_100 |
RELEX-Qwen2.5-Math-1.5B |
RELEX prediction, T_cut=75, target step=500 |
RELEX-Qwen3-4B-Base |
RELEX prediction, T_cut=75, target step=500 |
RELEX-Qwen3-8B-Base |
RELEX prediction, T_cut=100, target step=750 |
Trajectory branches use the revision="step_N" convention, so you can load any intermediate RLVR checkpoint without downloading the entire trajectory:
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"relex-rlvr/RLVR-Qwen2.5-Math-1.5B",
revision="step_50", # any step in [1, 500]; omit revision to get final step-500
)If you have any questions related to the code or the paper, feel free to email Zhepei (zhepei.wei@virginia.edu). If you encounter any problems when using the code, or want to report a bug, feel free to open an issue! Please try to specify the problem with details so we can help you better and quicker!
If you find this codebase useful, please consider citing:
@article{wei2026relex,
title = {You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories},
author = {Wei, Zhepei and Zhu, Xinyu and Chen, Wei-Lin and Huang, Chengsong and Huang, Jiaxin and Meng, Yu},
journal = {arXiv preprint arXiv:2605.21468},
year = {2026}
}