Repository navigation
Support stable and RC toolchains with selective CFSG and Stacks caches - #670
Merged
Merged
Conversation
Collaborator
Author
|
Prerequisite for production use: leanprover/lean-eval-submissions#2182 restores library caches after solver files are overlaid and uses selective restoration when preparing the benchmark root. |
kim-em
marked this pull request as ready for review
November 13, 2026 13:04
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support submissions on Lean v4.35.0 and v4.36.0-rc2, each with a benchmark-owned, fully pinned dependency configuration. The root checkout defaults to rc2. The same problem sources must compile under both configurations; no comparison of their elaborated meanings is required. Portable formulations repair the projectivization, tensor-action and abelianization API differences.
Add isolated configuration preparation and source-archive toolchain selection. The closest
lean-toolchainabove each attempted workspace selects an approved configuration; missing pins use the default, and mixed or unsupported requests are rejected after archival. Configuration preparation keeps build outputs separate, and the runner regenerates selected pristine workspaces before overlaying solver code. Intake and evaluation retain the exact benchmark commit and selected toolchain.Allow solution imports from CFSG and Stacks while continuing to reject them in trusted statements. Parse import headers without elaborating solver code, register the required cache services, and restore explicit module targets with compilation disabled. Both public services are configured and verified: CFSG at
https://cfsg-downloads.axiommath.workers.dev/lake, Stacks athttps://stacks-cache.axiommath.workers.dev/lake. Empty endpoint overrides retain these defaults.CI requires catalog compilation on stable and RC, exercises cache restoration and sandbox/scoring on both, and preserves the existing required
verify, security andrestorecheck names. Committed default workspaces remain owned by the main-branch regenerator.Runner prerequisite completed: leanprover/lean-eval-submissions#2182 is merged at
8b0005d3and promoted through staging and production. It selects the archived configuration, regenerates workspaces, fetches solution-only caches, and records toolchain provenance in immutable verification receipts and archive sidecars.External cache compatibility: stable pins Mathlib
074063781836922cb02701cd12943c9f51731afaand TauCeti1f9b4414105d86e487a4ac4c3d3397076a44d0d2; rc2 pins Mathlib2ddf59c8f14245864e6c0ed227a2dab76b9efea2and TauCetibe86813dc92e6bac8614c2c2dd34572145df09c0. CFSG's current publication targets 4.35.0-rc3. Stacks targets 4.35.0 but Mathlib56a00585, so both libraries still need cache publications matching an approved full dependency set. Unused solution libraries do not block ordinary submissions.Validation:
Latest-head GitHub CI and cache restoration passed, including all stable/RC catalog shards, scoring jobs, inventory aggregation and the required
verifycheck. The classifier now parses headers without materializing package dependencies; its coverage checks pass on both toolchains. Benchmark CI materializes only pinned package metadata for the solution-only CFSG/Stacks dependencies, keeping ordinary validation within its deadlines; submission evaluation retains full source checkouts. Tests cover the metadata policy, exact revision and actual sparse Git materialization, and an isolated stable project passes manifest validation and workspace generation with these metadata-only packages.