Deterministic encoding of `DefPathHashMap` by aerooneqq · Pull Request #162910 · rust-lang/rust

aerooneqq · 2026-09-17T15:10:45Z

This PR splits DefPathHashMap into two parts: deterministic (det_part) and non-deterministic (non_det_part). det_part is used while we sure that allocation and insertion order of def ids is deterministic, at the moment of writing it happens before we start parallel checks after prefetch of hir_crate_items in run_required_analysis. Up until this point of compilation the allocation of def ids and their insertion order into det_part should be the same between different compilations.

Next, when non-determinism starts due to parallel compilation we put all mapping between local hash and def ids into a SortedMap which gives us ready-to-use sorted by local hash slice of pairs to encode while encoding metadata.

Note, that this PR does not solve the problem of allocation of different def indices to same code entities (meaning same local hash), this problem is solved in #162809 by remapping needed local def indices.
Also note that we serialize det_part as a raw bytes sequence, so during remapping if we do not place all def indices that are needed to be remapped into separate container we will end up with copying det_part, changing some entries in it and then serialize it as a bytes sequence. With the approach in this PR we do not copy and modify det_part, instead we do all modifications in non_det_part.

r? @petrochenkov

petrochenkov · 2026-09-17T18:18:59Z

@bors try @rust-timer queue

…try> Deterministic encoding of `DefPathHashMap`

rust-bors · 2026-09-17T19:18:23Z

☀️ Try build successful (CI)
Build commit: df2b5a2 (df2b5a2f0e02a8844635d39578b4c1b338a8c0d0)
Base parent: c999cef (c999cef531ea9059e189e82fe0e82c5daf249bc9)

rust-timer · 2026-09-17T20:48:40Z

Finished benchmarking commit (df2b5a2): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.1%, 0.8%]	8
Regressions ❌ (secondary)	0.4%	[0.1%, 0.9%]	33
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.4%	[-0.9%, -0.3%]	9
All ❌✅ (primary)	0.5%	[0.1%, 0.8%]	8

Max RSS (memory usage)

Results (primary 3.1%, secondary 4.7%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.1%	[0.7%, 12.2%]	195
Regressions ❌ (secondary)	4.8%	[0.5%, 12.7%]	307
Improvements ✅ (primary)	-0.8%	[-0.8%, -0.8%]	1
Improvements ✅ (secondary)	-3.1%	[-3.1%, -3.1%]	1
All ❌✅ (primary)	3.1%	[-0.8%, 12.2%]	196

Cycles

Results (primary 10.3%, secondary 4.2%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	10.3%	[2.8%, 17.3%]	4
Regressions ❌ (secondary)	4.4%	[2.0%, 10.2%]	36
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-3.1%	[-3.1%, -3.1%]	1
All ❌✅ (primary)	10.3%	[2.8%, 17.3%]	4

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 500.132s -> 496.691s (-0.69%)
Artifact size: 406.81 MiB -> 409.64 MiB (0.70%)

petrochenkov · 2026-09-18T14:26:33Z

@bors try @rust-timer queue

…try> Deterministic encoding of `DefPathHashMap`

rust-bors · 2026-09-18T15:31:15Z

☀️ Try build successful (CI)
Build commit: a367041 (a36704158886f7496c764811537d3ff5babcbaf9)
Base parent: 420ed2a (420ed2a0c3d7225b1744266fd884d431b4d8cfe0)

rust-timer · 2026-09-18T17:00:11Z

Finished benchmarking commit (a367041): comparison URL.

Overall result: ❌ regressions - no action needed

Benchmarking means the PR may be perf-sensitive. Consider adding rollup=never if this change is not fit for rolling up.

@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	0.2%	[0.2%, 0.2%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Max RSS (memory usage)

Results (primary 3.6%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	3.6%	[3.6%, 3.6%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	3.6%	[3.6%, 3.6%]	1

Cycles

Results (secondary 3.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	3.1%	[2.5%, 4.2%]	3
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-	-	0

Binary size

Results (primary -0.1%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.1%	[-0.2%, -0.0%]	67
Improvements ✅ (secondary)	-0.1%	[-0.2%, -0.0%]	53
All ❌✅ (primary)	-0.1%	[-0.2%, -0.0%]	67

Bootstrap: 498.333s -> 497.372s (-0.19%)
Artifact size: 408.93 MiB -> 408.87 MiB (-0.02%)

rustbot · 2026-09-21T12:30:24Z

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

rustbot · 2026-09-24T13:40:44Z

Reminder, once the PR becomes ready for a review, use @rustbot ready.

…mode`

aerooneqq · 2026-09-25T06:03:53Z

@rustbot ready

petrochenkov · 2026-09-28T14:22:11Z

@bors r+

rust-bors · 2026-09-28T14:22:15Z

📌 Commit 2f77d20 has been approved by petrochenkov

It is now in the queue for this repository.

petrochenkov · 2026-09-28T14:22:48Z

@bors rollup=maybe

…uwer Rollup of 7 pull requests Successful merges: - #163327 (remove `MutTy`) - #162910 (Deterministic encoding of `DefPathHashMap`) - #163009 (dont store arbitrary parsed attributes in thir) - #162683 (Use attribute parser for `#[inline()]` attribute check) - #163429 (lint on `Ident::from_str_and_span` taking a string literal) - #163431 (Allow `#[repr(simd)]` with `f16b`) - #163433 (Support also `try-jobs:` to specify custom try jobs)

@petrochenkov

Rollup merge of #162910 - aerooneqq:det-def-path-hash-map-encoding, r=petrochenkov Deterministic encoding of `DefPathHashMap` This PR splits DefPathHashMap into two parts: deterministic (`det_part`) and non-deterministic (`non_det_part`). `det_part` is used while we sure that allocation and insertion order of def ids is deterministic, at the moment of writing it happens before we start parallel checks after prefetch of `hir_crate_items` in `run_required_analysis`. Up until this point of compilation the allocation of def ids and their insertion order into `det_part` should be the same between different compilations. Next, when non-determinism starts due to parallel compilation we put all mapping between local hash and def ids into a `SortedMap` which gives us ready-to-use sorted by local hash slice of pairs to encode while encoding metadata. Note, that this PR does not solve the problem of allocation of different def indices to same code entities (meaning same local hash), this problem is solved in #162809 by remapping needed local def indices. Also note that we serialize `det_part` as a raw bytes sequence, so during remapping if we do not place all def indices that are needed to be remapped into separate container we will end up with copying `det_part`, changing some entries in it and then serialize it as a bytes sequence. With the approach in this PR we do not copy and modify `det_part`, instead we do all modifications in `non_det_part`. r? @petrochenkov

…uwer Rollup of 7 pull requests Successful merges: - rust-lang/rust#163327 (remove `MutTy`) - rust-lang/rust#162910 (Deterministic encoding of `DefPathHashMap`) - rust-lang/rust#163009 (dont store arbitrary parsed attributes in thir) - rust-lang/rust#162683 (Use attribute parser for `#[inline()]` attribute check) - rust-lang/rust#163429 (lint on `Ident::from_str_and_span` taking a string literal) - rust-lang/rust#163431 (Allow `#[repr(simd)]` with `f16b`) - rust-lang/rust#163433 (Support also `try-jobs:` to specify custom try jobs)

…nathanBrouwer Rollup of 7 pull requests Successful merges: - rust-lang#163327 (remove `MutTy`) - rust-lang#162910 (Deterministic encoding of `DefPathHashMap`) - rust-lang#163009 (dont store arbitrary parsed attributes in thir) - rust-lang#162683 (Use attribute parser for `#[inline()]` attribute check) - rust-lang#163429 (lint on `Ident::from_str_and_span` taking a string literal) - rust-lang#163431 (Allow `#[repr(simd)]` with `f16b`) - rust-lang#163433 (Support also `try-jobs:` to specify custom try jobs)

rustbot assigned petrochenkov Sep 17, 2026

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Sep 17, 2026

aerooneqq force-pushed the det-def-path-hash-map-encoding branch from 11ec6a0 to 417b0b5 Compare September 17, 2026 15:13