Skip to content

Label the latest TabPFN-Rel versions on the leaderboard - #64

Merged
adrian-prior merged 2 commits into
mainfrom
adrian/tabpfn-rel-latest-labels
Sep 29, 2026
Merged

adrian-prior merged 2 commits into
mainfrom
adrian/tabpfn-rel-latest-labels

Conversation

@adrian-prior

Copy link
Copy Markdown
Collaborator

Generated by Claude Code

The README leaderboard and plot now label the current TabPFN-Rel versions tabpfn-rel-client-latest and tabpfn-rel-local-latest, using the aliases TabPFN-Rel 0.0.5 registers. The first local model is renamed from tabpfn-rel-local to tabpfn-rel-local-2026-08-15 to match the rename in TabPFN-Rel 0.0.5, and usage examples point at the -latest names. Ranks, Elo, and intervals are unchanged.

Details

Labels

  • The dev group installs base tabpfn-rel>=0.0.5, which registers the TabPFN-Rel methods without the inference backends. CI gets the labels through --group dev, so no workflow change is needed.
  • workflows/update_leaderboards.py fails when results.csv names a method that is not registered. Without that check, the labels would depend on which packages happen to be installed.
  • Regenerated README.md and docs/leaderboards.png. In the generated tables only the TabPFN-Rel names change.

Rename of the first local model

  • baseline_results/results.csv: the model value of the 63 tabpfn-rel-local rows becomes tabpfn-rel-local-2026-08-15. Checked with pandas: every other column matches main exactly.
  • Descriptions of that specific run (tuning table, baseline sources, per-model appendix, README models table) use the dated name. The pip extra tabpfn-rel-local keeps its name.
  • workflows/check_package_split.py checks the new registered name.

Usage examples
The README, docs/predictive-task.md, packages/relarena/README.md, the relbench_v1 docstring, and examples/olist_seller_churn.py now use tabpfn-rel-client-latest / tabpfn-rel-local-latest. Those examples now run the TabPFN 3.5 models instead of the 2026-08-15 ones.

Dependencies

  • tabpfn-rel extras floor: 0.0.4 → 0.0.5. The lock moves only tabpfn-rel 0.0.4 → 0.0.5 and is written with uv 0.12.20. uv sync --locked passes with uv 0.12.20 and 0.11.7.
  • The fixed fastdfs cutoff stays until 2026-10-01, when 1.2 clears the 3-day window.

Tests

  • A new test checks that the generator refuses unregistered methods and writes nothing.
  • The alias-move test keeps the installed aliases.
  • Full suite on macOS: 471 passed, 6 skipped, 1 failed. The failure is an RT test that fails only on macOS (torch supports only the file_system sharing strategy there); this PR changes no RT code. pre-commit run --all-files passes.

🤖 Generated with Claude Code

adrian-prior and others added 2 commits September 29, 2026 13:48
TabPFN-Rel 0.0.5 registers tabpfn-rel-client-latest and
tabpfn-rel-local-latest and renames its first local model to
tabpfn-rel-local-2026-08-15. The dev environment installs the plugin so the
README generator resolves those labels, and the generator now fails when a
result names an unregistered method. The stored results use the new dated
name for the first local model; scores are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@adrian-prior
adrian-prior marked this pull request as ready for review September 29, 2026 12:13
@adrian-prior
adrian-prior merged commit b12765a into main Sep 29, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant