Skip to content

Add a way for the test suite to test the output of a stable rustc #123404

Description

@estebank

Right now ui tests only target the output of the currently built rustc. As far as I can tell, there's no -Z flag to tell rustc "act as if you were a stable release". This means that when we customize the diagnostic output based on release channel, we cannot test what stable users will actually see. An example of this is this (as of now unmerged) change that on stable refers to unsized locals and unsized fn arguments with the same note telling the user that's not allowed, while on nightly it tells the user which feature flag is gating that being allowed, without the note stating it's not allowed. That kind of behavior is replicated in multiple diagnostics involving nightly features already. It's the same "problem" we'd have with editions if we didn't already have a stable flag to specify them.

Activity

  1. added
    A-testsuiteArea: The testsuite used to check the correctness of rustc
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
    C-feature-requestCategory: A feature request, i.e: not implemented / a PR.
    on Apr 3, 2024
  2. added
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Apr 3, 2024
  3. removed
    needs-triageThis issue may need triage. Remove when done. See docs forge.rust-lang.org/release/issue-triaging
    on Apr 3, 2024
  4. matthiaskrgr commented on Apr 3, 2024

    @matthiaskrgr
    Member

    Another usecase would be to use something like
    RUSTC_STABLE=1 rustc ice.rs to check if an ICE would happen on $todays nightly in several months after promotion to beta/stable has happened already. 🙂

  5. Mark-Simulacrum commented on Apr 3, 2024

    @Mark-Simulacrum
    Member

    RUSTC_BOOTSTRAP=0 feels like a reasonable mechanism/extension to me.

  6. estebank commented on Apr 4, 2024

    @estebank
    ContributorAuthor

    I'd be concerned with people having set RUSTC_BOOTSTRAP=0 globally in their machines and suddenly having issues with nightly, but at the same time "setting that flag has no stability guarantees".

  7. self-assigned this
    on Nov 13, 2024
  8. jieyouxu commented on Nov 13, 2024

    @jieyouxu
    Member

    FWIW I went with RUSTC_BOOTSTRAP=-1 to mean "force-stable" in #132993 instead of RUSTC_BOOTSTRAP=0 which can seem like "not bootstrap" which is not quite the same I feel like, and probably confusing.

    Implementation steps

    UI tests can use something like

    //@ rustc-env:RUSTC_BOOTSTRAP=-1
    //@ only-nightly

    to fool convince the unstable compiler under test to think it's very stable. We can of course also add a //@ pretend-stable-rustc but yeah.

  9. bjorn3 commented on Nov 13, 2024

    @bjorn3
    Member

    There is -Zallow-features= to disable support for all #![feature], but this doesn't affect commandline flags and doesn't work on stable either.

  10. added a commit that references this issue on Nov 18, 2024
  11. added a commit that references this issue on Nov 18, 2024
  12. added
    A-test-infraArea: test infrastructure (may span bootstrap/compiletest/more)
    on Nov 18, 2024
  13. jieyouxu commented on Nov 18, 2024

    @jieyouxu
    Member

    Update: #132993 has now merged to support RUSTC_BOOTSTRAP=-1 to mean "pretend I am a stable compiler".

  14. removed their assignment
    on Nov 18, 2024
  15. jieyouxu commented on Jan 2, 2025

    @jieyouxu
    Member

    For future reference, while you can force a given rustc (well, later than #132993 of course) to pretend to be stable for all intents and purposes, there will still be an important caveat:

    -> real stable rustc    <- actually stable
    |
    -> beta rustc
    |
    -> nightly rustc
    |
    -> `master` rustc       <- pretend to be stable
    

    the master or nightly or beta rustc will almost certainly contain changes that are not present in the actual stable rustc (unavoidable), which is something important to keep in mind.

    That being said, I think we can consider this feature request to be sufficiently satisfied via RUSTC_BOOTSTRAP=-1.

  16. Kobzol commented on Oct 2, 2026

    @Kobzol
    Member

    This didn't actually enable testing stable output in UI tests, because compiletest unconditionally passes certain -Z flags, and that breaks if you truly make rustc act like stable. We needed -2 all along (#163664), which acts as stable, but allows unstable compiler flags 😆

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-test-infraArea: test infrastructure (may span bootstrap/compiletest/more)A-testsuiteArea: The testsuite used to check the correctness of rustcC-feature-requestCategory: A feature request, i.e: not implemented / a PR.T-bootstrapRelevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions