Fix incremental compilation for fat LTO by bjorn3 · Pull Request #163317 · rust-lang/rust

bjorn3 · 2026-09-25T10:18:19Z

Previously we would copy post-LTO artifacts into the incr comp cache for fat LTO despite them never getting used. Also the pre-LTO bitcode didn't get tracked and thus determine_cgu_reuse believed it had to regenerate bitcode for all CGUs even when all CGUs would be green.

Also move incr comp session dirs out of CodegenContext. CodegenContext will in the future be serialized and deserialized into a different rustc instance when -Zno-link/-Zlink-only is used. A separate incr comp session has to be used for both rustc invocations.

Part of rust-lang/compiler-team#908

rustbot · 2026-09-25T10:18:22Z

rustc_codegen_gcc is developed in its own repository. If possible, consider making this change to rust-lang/rustc_codegen_gcc instead.

cc @antoyo, @GuillaumeGomez

rustbot · 2026-09-25T10:18:25Z

r? @mejrs

rustbot has assigned @mejrs.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

Owners of files modified in this PR: codegen, compiler
codegen, compiler expanded to 77 candidates
Random selection from 21 candidates

CodegenContext will in the future be serialized and deserialized into a different rustc instance when -Zno-link/-Zlink-only is used. A separate incr comp session has to be used for both rustc invocations.

Previously we would copy post LTO artifacts into the incr comp cache for fat LTO despite them never getting used. Also the pre LTO bitcode didn't get tracked and thus determine_cgu_reuse believed it had to regenerate bitcode for all CGUs even when all CGUs would be green.

bjorn3 · 2026-09-25T12:12:09Z

            if !self.check_config(cfg) {
                debug!("check_attr: config does not match, ignoring attr");
-                return;
+                continue;


This caused us to skip all assertions after the first one with a cfg that doesn't match. Surprisingly nothing regressed by the lack of test coverage since the introduction of this bug.

View changes since the review

mejrs

Not sure about this otherwise.

r? @nnethercote perhaps?

View changes since this review

Co-authored-by: Bruno Kolenbrander <59372212+mejrs@users.noreply.github.com>

nnethercote

r=me with the nits addressed.

View changes since this review

nnethercote · 2026-10-02T03:03:32Z

+                };
+
+                // FIXME include pre-LTO bitcode in workproduct tracking
+                // FIXME add separate incr comp session for post-LTO outputs to use during link step


How important are these FIXMEs? Are you planning to fix them?

I'm working on a follow up PR to do this.

bjorn3 · 2026-10-02T09:01:09Z

@bors r=nnethercote

rust-bors · 2026-10-02T09:01:12Z

📌 Commit ebdd423 has been tentatively approved by nnethercote

It will be put into the queue for this repository once PR CI succeeds.

Fix incremental compilation for fat LTO Previously we would copy post-LTO artifacts into the incr comp cache for fat LTO despite them never getting used. Also the pre-LTO bitcode didn't get tracked and thus `determine_cgu_reuse` believed it had to regenerate bitcode for all CGUs even when all CGUs would be green. Also move incr comp session dirs out of `CodegenContext`. `CodegenContext` will in the future be serialized and deserialized into a different rustc instance when `-Zno-link`/`-Zlink-only` is used. A separate incr comp session has to be used for both rustc invocations. Part of rust-lang/compiler-team#908

…uwer Rollup of 13 pull requests Successful merges: - #163317 (Fix incremental compilation for fat LTO) - #163582 (Reapply "bootstrap: Enable rustdoc mergeable CCI for std and internal docs") - #151793 (Add mul_add_relaxed methods for floating-point types) - #162782 (Fix rustdoc ICE caused by mishandling of ambiguity errors) - #163010 (Miri can do dirfd now) - #163535 (Improve `DocStrings` perf) - #163576 (Fix `TypeOutlives` fast-path) - #163587 (Several small span improvements) - #163612 (fix `ValidateBoundVars`) - #163632 (bump rustc-build-sysroot) - #163635 (Revert note about signum of NaN) - #163644 (Add mailmap entry) - #163651 (Remove variants from `feature-gate-autodiff-use` test)

JonathanBrouwer · 2026-10-02T14:07:42Z

💔 I suspect this PR failed tests as part of a rollup
@bors r-

After fixing the problem, consider running a try job for the failed job before re-approving.

Link to failure: #163650 (comment)

@bors try jobs=test-x86_64-mingw-1

rust-bors · 2026-10-02T14:07:51Z

This pull request was unapproved.

This PR was contained in a rollup (#163656), which was unapproved.

View changes since this unapproval

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

rust-bors · 2026-10-02T16:46:40Z

💔 Test for e4021fc failed: CI. Failed job:

try - test-x86_64-mingw-1 (web logs, enhanced plaintext logs)

rust-log-analyzer · 2026-10-02T16:46:42Z

The job test-x86_64-mingw-1 failed! Check out the build log: (web) (plain enhanced) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- [run-make] tests\run-make\lto-empty stdout ----

error: rmake recipe failed to complete
status: exit code: 1
command: "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake.exe"
stdout: none
--- stderr -------------------------------
"D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\stage2\\bin\\rustc.exe" "-L" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out" "lib.rs" "-Clto=fat" "-Copt-level=3" "-Cincremental=inc-fat" "--target=x86_64-pc-windows-gnu"
output status: `exit code: 0`
=== STDOUT ===



=== STDERR ===



command failed at line 16
"D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\stage2\\bin\\rustc.exe" "-L" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out" "lib.rs" "-Clto=fat" "-Copt-level=3" "-Cincremental=inc-fat" "--target=x86_64-pc-windows-gnu"
output status: `exit code: 1`
=== STDOUT ===



=== STDERR ===
error: linking with `x86_64-w64-mingw32-gcc` failed: exit code: 1
  |
  = note: "x86_64-w64-mingw32-gcc" "-Wl,D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out\\rustcnFGqkA\\list.def" "-fno-use-linker-plugin" "-Wl,--dynamicbase" "-Wl,--disable-auto-image-base" "-m64" "-Wl,--high-entropy-va" "<21 object files omitted>" "-Wl,-Bstatic" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out\\rustcnFGqkA/libstd-f19ec5cf82cf2e36.rlib" "<sysroot>\\lib\\rustlib\\x86_64-pc-windows-gnu\\lib/libcompiler_builtins-*.rlib" "-Wl,-Bdynamic" "-lkernel32" "-lkernel32" "-lntdll" "-luserenv" "-lws2_32" "-ldbghelp" "-lsynchronization" "-lgcc_eh" "-l:libpthread.a" "-lmsvcrt" "-lmingwex" "-lmingw32" "-lgcc" "-lmsvcrt" "-lmingwex" "-luser32" "-lkernel32" "-Wl,--nxcompat" "-L" "D:\\a\\rust\\rust\\build\\x86_64-pc-windows-gnu\\test\\run-make\\lto-empty\\rmake_out" "-o" "lib.dll" "-Wl,--gc-sections" "-shared" "-Wl,--out-implib=liblib.dll.a" "-Wl,-O1" "-nodefaultlibs"
  = note: some arguments are omitted. use `--verbose` to show all linker arguments
  = note: D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.object-748e56779e68aa6c.object.633ec871561b15de-cgu.0.rcgu.o.05j0x8c.rcgu.o:object.633ec871561b15de-cgu.0:(.text+0x45b): undefined reference to `__imp__RNvNvNtNtNtCsjJQlRGTNn6I_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.object-748e56779e68aa6c.object.633ec871561b15de-cgu.0.rcgu.o.05j0x8c.rcgu.o:object.633ec871561b15de-cgu.0:(.text+0x536): undefined reference to `__imp__RNvNvNtNtNtCsjJQlRGTNn6I_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x21519): undefined reference to `__imp__RNvNvNtNtNtCsjJQlRGTNn6I_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x215c0): undefined reference to `__imp__RNvNvNtNtNtCsjJQlRGTNn6I_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x21739): undefined reference to `__imp__RNvNvNtNtNtCsjJQlRGTNn6I_6memchr4arch6x86_646memchr10memchr_raw2FN'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x217ae): more undefined references to `__imp__RNvNvNtNtNtCsjJQlRGTNn6I_6memchr4arch6x86_646memchr10memchr_raw2FN' follow
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x267f6): undefined reference to `__imp__RNvNtNtNtCslNM2aqSgGGX_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x27af7): undefined reference to `__imp__RNvNtNtNtCslNM2aqSgGGX_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x27eef): undefined reference to `__imp__RNvNtNtNtCslNM2aqSgGGX_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x27ff8): undefined reference to `__imp__RNvNtNtNtCslNM2aqSgGGX_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x2ae06): undefined reference to `__imp__RNvNtNtNtCslNM2aqSgGGX_4core2io5error12os_functions12OS_FUNCTIONS'
          D:/a/rust/rust/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/14.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: lib.std-f19ec5cf82cf2e36.std.978b82965100b081-cgu.0.rcgu.o.05j0x8c.rcgu.o:std.978b82965100b081-cgu.0:(.text+0x2b89b): more undefined references to `__imp__RNvNtNtNtCslNM2aqSgGGX_4core2io5error12os_functions12OS_FUNCTIONS' follow
          collect2.exe: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link

error: aborting due to 1 previous error
---
Currently active steps:
test::RunMake { test_compiler: Compiler { stage: 2, host: x86_64-pc-windows-gnu, forced_compiler: false }, target: x86_64-pc-windows-gnu } at src\bootstrap\src\core\build_steps\test.rs:2024
test::Compiletest { test_compiler: Compiler { stage: 2, host: x86_64-pc-windows-gnu, forced_compiler: false }, target: x86_64-pc-windows-gnu, mode: run-make, suite: "run-make", path: "tests/run-make", compare_mode: None } at src\bootstrap\src\core\build_steps\test.rs:2024
Build completed unsuccessfully in 2:29:04
make: *** [Makefile:126: ci-mingw-x] Error 1
  local time: Fri Oct  2 16:46:25 CUT 2026
  network time: Fri, 02 Oct 2026 16:46:25 GMT
##[error]Process completed with exit code 2.
##[group]Run echo "disk usage:"
echo "disk usage:"

bjorn3 added 2 commits September 25, 2026 09:37

Remove unused method

1101121

Move some LLVM specific options from Linker::link to cg_llvm

e216dfe

rustbot assigned mejrs Sep 25, 2026

bjorn3 mentioned this pull request Sep 25, 2026

Move LTO from the codegen coordinator thread to link_binary rust-lang/compiler-team#908

Closed

14 of 22 tasks

This comment has been minimized.

Sign in to view

bjorn3 added 3 commits September 25, 2026 12:10

Move incr comp session dirs out of CodegenContext

5a01161

CodegenContext will in the future be serialized and deserialized into a different rustc instance when -Zno-link/-Zlink-only is used. A separate incr comp session has to be used for both rustc invocations.

Add test for incr comp fat LTO and fix assert_module_sources testing

980f03b

bjorn3 force-pushed the lto_refactors22 branch from bc690ff to 980f03b Compare September 25, 2026 12:10

bjorn3 commented Sep 25, 2026

View reviewed changes

bjorn3 changed the title ~~Fix incr comp for fat LTO~~ Fix incremental compilation for fat LTO Sep 27, 2026

mejrs reviewed Sep 30, 2026

View reviewed changes

Comment thread compiler/rustc_codegen_llvm/src/back/llvm_backend.rs

rustbot assigned nnethercote and unassigned mejrs Sep 30, 2026

Apply suggestion

52b68f2

Co-authored-by: Bruno Kolenbrander <59372212+mejrs@users.noreply.github.com>

nnethercote approved these changes Oct 2, 2026

View reviewed changes

nnethercote added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Oct 2, 2026

Fix typo

ebdd423

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Oct 2, 2026

JonathanBrouwer mentioned this pull request Oct 2, 2026

Rollup of 8 pull requests #163650

Closed

JonathanBrouwer mentioned this pull request Oct 2, 2026

Rollup of 13 pull requests #163656

Closed

rust-bors Bot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Oct 2, 2026

This comment has been minimized.

Sign in to view

rust-bors Bot pushed a commit that referenced this pull request Oct 2, 2026

Auto merge of #163317 - bjorn3:lto_refactors22, r=<try>

e4021fc

Fix incremental compilation for fat LTO try-job: test-x86_64-mingw-1

Uh oh!

Conversation

bjorn3 commented Sep 25, 2026

Uh oh!

rustbot commented Sep 25, 2026

Uh oh!

rustbot commented Sep 25, 2026

Uh oh!

This comment has been minimized.

bjorn3 Sep 25, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mejrs left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nnethercote left a comment • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nnethercote Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

bjorn3 Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

bjorn3 commented Oct 2, 2026

Uh oh!

rust-bors Bot commented Oct 2, 2026

Uh oh!

JonathanBrouwer commented Oct 2, 2026

Uh oh!

rust-bors Bot commented Oct 2, 2026

Uh oh!

This comment has been minimized.

rust-bors Bot commented Oct 2, 2026

Uh oh!

rust-log-analyzer commented Oct 2, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

bjorn3 Sep 25, 2026 •

edited by rustbot

Loading

mejrs left a comment •

edited by rustbot

Loading

nnethercote left a comment •

edited by rustbot

Loading