Some refactorings around metadata encoding by aerooneqq · Pull Request #162900 · rust-lang/rust

aerooneqq · 2026-09-17T13:08:23Z

View all comments

Two refactorings that were extracted from #162809:

Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in Remap def indices for deterministic metadata encoding #162809,
Moves proc_macro_quoted_spans table into ProcMacroData as it is filled only when ProcMacroData is Some (meaning crate type is proc-macro), so it is reasonable to place it here, moreover all tables left in LazyTables have DefIndex as key, this simplifies refactorings and modifications that are connected with DefIndexes.

r? @petrochenkov

petrochenkov · 2026-09-17T13:54:04Z

r? @Zalathar , for example, I've seen you refactoring the metadata infra recently

rustbot · 2026-09-17T13:54:09Z

Zalathar is not on the review rotation at the moment.
They may take a while to respond.

Zalathar · 2026-09-17T15:00:46Z

Could you explain more about proc_macro_quoted_spans and why it's being moved?

From the PR description and commit message, I wasn't able to get a good sense of why this change is desirable, or whether it's even correct.

aerooneqq · 2026-09-17T15:19:20Z

Could you explain more about proc_macro_quoted_spans and why it's being moved?

It was the only table that has usize as key, so we can't use macros to record values in it, and values in this table are set only under is_proc_macro condition in encode_proc_macros, so if ProcMacroData is None (meaning is_proc_macro is false) then all queries to this table will panic, so we can move this table into ProcMacroData, I thought that it is more appropriate place for this table.

Zalathar · 2026-09-18T00:02:20Z

So if I understand correctly:

proc_macro_quoted_spans is only used in proc-macro crates, and ProcMacroData is specifically for holding data that is only used in proc-macro crates. So moving it to ProcMacroData is a reasonable change, regardless of why we want to move it out of LazyTables.
The reason we want to move it out of LazyTables is that it's the only one of those tables with a key type that isn't DefIndex. So having proc_macro_quoted_spans in LazyTables is getting in the way of changes to how DefIndex is treated.
The fact that proc_macro_quoted_spans happens to have a key type of usize is irrelevant, and prominently mentioning usize is misleading to future maintainers.

If so, could you please update the commit message and PR description to clearly explain the reasoning to future maintainers, and remove the misleading stuff about usize?

aerooneqq · 2026-09-18T09:09:05Z

If so, could you please update the commit message and PR description to clearly explain the reasoning to future maintainers, and remove the misleading stuff about usize?

Updated commit message and PR description.

Zalathar · 2026-09-18T10:00:46Z

Thanks.

I will also say that normally I would be against adding the macros, but I guess it makes sense if they’re specifically for enabling a tricky change to how def-indexes work. I do hope that we can remove the macros after those changes, though.

petrochenkov · 2026-09-29T11:59:12Z

This is now blocking #163321.

Zalathar · 2026-09-30T06:33:20Z

I don't have capacity to continue reviewing this.

petrochenkov · 2026-09-30T08:20:46Z

r? @petrochenkov
@bors r+

rust-bors · 2026-09-30T08:20:51Z

📌 Commit 3033e33 has been approved by petrochenkov

It is now in the queue for this repository.

bjorn3 · 2026-09-30T08:58:13Z

No need to block merging this PR on the above two comments. They are just minor style things.

@petrochenkov

…cros, r=petrochenkov Some refactorings around metadata encoding Two refactorings that were extracted from rust-lang#162809: - Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in rust-lang#162809, - Moves `proc_macro_quoted_spans` table into `ProcMacroData` as it is filled only when `ProcMacroData` is `Some` (meaning crate type is `proc-macro`), so it is reasonable to place it here, moreover all tables left in `LazyTables` have `DefIndex` as key, this simplifies refactorings and modifications that are connected with `DefIndex`es. r? @petrochenkov

aerooneqq · 2026-09-30T13:39:53Z

@bors r=petrochenkov

rust-bors · 2026-09-30T13:39:56Z

📌 Commit 1161c07 has been tentatively approved by petrochenkov

It will be put into the queue for this repository once PR CI succeeds.

@petrochenkov

…cros, r=petrochenkov Some refactorings around metadata encoding Two refactorings that were extracted from rust-lang#162809: - Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in rust-lang#162809, - Moves `proc_macro_quoted_spans` table into `ProcMacroData` as it is filled only when `ProcMacroData` is `Some` (meaning crate type is `proc-macro`), so it is reasonable to place it here, moreover all tables left in `LazyTables` have `DefIndex` as key, this simplifies refactorings and modifications that are connected with `DefIndex`es. r? @petrochenkov

…uwer Rollup of 18 pull requests Successful merges: - #163532 (rustc_codegen_cranelift subtree update) - #163534 (miri subtree update) - #163279 (const and NonZero impl for clamp_magnitude()) - #162900 (Some refactorings around metadata encoding) - #163455 (Don't use the metadata based crate_hash for rustdoc runs) - #163483 (Bump bootstrap compiler to 1.100.0 beta) - #159798 (Attribute documentation for cfg_attr) - #163368 (Don't build format string suggestions from `concat!` offsets) - #163375 (Update expect messages in library/std/src/os/unix/net/ following Rust's `expect` guidance) - #163405 (Remove some #[linkage] options) - #163470 (Miscellaneous attr error stuff) - #163489 (expose Rc::is_unique) - #163492 (x86 and x86_64: cleanup some callconv code) - #163496 (cycle handling: mirror old solver) - #163509 (Use more default field values in `Resolver`) - #163519 (Forbid `Reborrow` impls for types with destructors) - #163520 (Cast cleanups) - #163524 (Document `Result` case for the `arena_cache` query modifier) Failed merges: - #163547 ([beta] rustfmt backport)

@petrochenkov

…cros, r=petrochenkov Some refactorings around metadata encoding Two refactorings that were extracted from rust-lang#162809: - Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in rust-lang#162809, - Moves `proc_macro_quoted_spans` table into `ProcMacroData` as it is filled only when `ProcMacroData` is `Some` (meaning crate type is `proc-macro`), so it is reasonable to place it here, moreover all tables left in `LazyTables` have `DefIndex` as key, this simplifies refactorings and modifications that are connected with `DefIndex`es. r? @petrochenkov

…uwer Rollup of 18 pull requests Successful merges: - #163532 (rustc_codegen_cranelift subtree update) - #163534 (miri subtree update) - #163279 (const and NonZero impl for clamp_magnitude()) - #162900 (Some refactorings around metadata encoding) - #163455 (Don't use the metadata based crate_hash for rustdoc runs) - #159798 (Attribute documentation for cfg_attr) - #162921 (make mips64 `Complex` GCC-compatible) - #163368 (Don't build format string suggestions from `concat!` offsets) - #163375 (Update expect messages in library/std/src/os/unix/net/ following Rust's `expect` guidance) - #163405 (Remove some #[linkage] options) - #163470 (Miscellaneous attr error stuff) - #163489 (expose Rc::is_unique) - #163492 (x86 and x86_64: cleanup some callconv code) - #163496 (cycle handling: mirror old solver) - #163509 (Use more default field values in `Resolver`) - #163519 (Forbid `Reborrow` impls for types with destructors) - #163520 (Cast cleanups) - #163524 (Document `Result` case for the `arena_cache` query modifier)

@petrochenkov

…cros, r=petrochenkov Some refactorings around metadata encoding Two refactorings that were extracted from rust-lang#162809: - Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in rust-lang#162809, - Moves `proc_macro_quoted_spans` table into `ProcMacroData` as it is filled only when `ProcMacroData` is `Some` (meaning crate type is `proc-macro`), so it is reasonable to place it here, moreover all tables left in `LazyTables` have `DefIndex` as key, this simplifies refactorings and modifications that are connected with `DefIndex`es. r? @petrochenkov

…uwer Rollup of 9 pull requests Successful merges: - #163483 (Bump bootstrap compiler to 1.100.0 beta) - #161380 (only rerun const eval in next-solver if the const actually references opaques) - #162900 (Some refactorings around metadata encoding) - #163580 (Provide better doc code example for `UnixDatagram::bind_addr` and `UnixListener::bind_addr`) - #163584 ([triagebot] Ping me for debugger visualizer changes) - #162782 (Fix rustdoc ICE caused by mishandling of ambiguity errors) - #163314 (move `#[macro_export]` on declarative macro check to `rustc_attr_parsing`) - #163405 (Remove some #[linkage] options) - #163581 (do not suggest precise capturing when the opaque span is in a macro expansion)

@petrochenkov

…cros, r=petrochenkov Some refactorings around metadata encoding Two refactorings that were extracted from rust-lang#162809: - Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in rust-lang#162809, - Moves `proc_macro_quoted_spans` table into `ProcMacroData` as it is filled only when `ProcMacroData` is `Some` (meaning crate type is `proc-macro`), so it is reasonable to place it here, moreover all tables left in `LazyTables` have `DefIndex` as key, this simplifies refactorings and modifications that are connected with `DefIndex`es. r? @petrochenkov

@petrochenkov

…cros, r=petrochenkov Some refactorings around metadata encoding Two refactorings that were extracted from rust-lang#162809: - Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in rust-lang#162809, - Moves `proc_macro_quoted_spans` table into `ProcMacroData` as it is filled only when `ProcMacroData` is `Some` (meaning crate type is `proc-macro`), so it is reasonable to place it here, moreover all tables left in `LazyTables` have `DefIndex` as key, this simplifies refactorings and modifications that are connected with `DefIndex`es. r? @petrochenkov

@petrochenkov

…cros, r=petrochenkov Some refactorings around metadata encoding Two refactorings that were extracted from rust-lang#162809: - Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in rust-lang#162809, - Moves `proc_macro_quoted_spans` table into `ProcMacroData` as it is filled only when `ProcMacroData` is `Some` (meaning crate type is `proc-macro`), so it is reasonable to place it here, moreover all tables left in `LazyTables` have `DefIndex` as key, this simplifies refactorings and modifications that are connected with `DefIndex`es. r? @petrochenkov

Rollup of 11 pull requests Successful merges: - #163483 (Bump bootstrap compiler to 1.100.0 beta) - #161380 (only rerun const eval in next-solver if the const actually references opaques) - #162900 (Some refactorings around metadata encoding) - #163580 (Provide better doc code example for `UnixDatagram::bind_addr` and `UnixListener::bind_addr`) - #163584 ([triagebot] Ping me for debugger visualizer changes) - #162904 (Fix ICE for ambiguous candidates on method probing) - #163281 (Add `f16` inline ASM support to `spirv.rs`) - #163314 (move `#[macro_export]` on declarative macro check to `rustc_attr_parsing`) - #163405 (Remove some #[linkage] options) - #163530 (`const impl PartialEq` for `f16b`) - #163581 (do not suggest precise capturing when the opaque span is in a macro expansion)

…uwer Rollup of 20 pull requests Successful merges: - #163483 (Bump bootstrap compiler to 1.100.0 beta) - #161380 (only rerun const eval in next-solver if the const actually references opaques) - #162900 (Some refactorings around metadata encoding) - #163461 (Improve diagnostic deduplication) - #163580 (Provide better doc code example for `UnixDatagram::bind_addr` and `UnixListener::bind_addr`) - #163584 ([triagebot] Ping me for debugger visualizer changes) - #159021 (windows-gnu: enable native TLS) - #161467 (wfcheck: name the item that discards an unused type parameter) - #162618 (trait_selection: Preserve eager normalization failures) - #162904 (Fix ICE for ambiguous candidates on method probing) - #163064 (Avoid computing overflowed goal chains for crate dependencies) - #163281 (Add `f16` inline ASM support to `spirv.rs`) - #163314 (move `#[macro_export]` on declarative macro check to `rustc_attr_parsing`) - #163360 ([rustdoc] Correctly handle rustc_allow_incoherent_impl on primitive methods) - #163385 (GVN transmutes of Immediate::Uninit to Immediate::Uninit) - #163405 (Remove some #[linkage] options) - #163530 (`const impl PartialEq` for `f16b`) - #163581 (do not suggest precise capturing when the opaque span is in a macro expansion) - #163590 (Make `AllocatorNightly` less clever) - #163599 (Add union pattern reference change to relnotes)

@petrochenkov

Rollup merge of #162900 - aerooneqq:record-tables-through-macros, r=petrochenkov Some refactorings around metadata encoding Two refactorings that were extracted from #162809: - Use macros to record values in all tables, this unifies writing to tables and makes it much easier to modify or refactor writing process as it was needed in #162809, - Moves `proc_macro_quoted_spans` table into `ProcMacroData` as it is filled only when `ProcMacroData` is `Some` (meaning crate type is `proc-macro`), so it is reasonable to place it here, moreover all tables left in `LazyTables` have `DefIndex` as key, this simplifies refactorings and modifications that are connected with `DefIndex`es. r? @petrochenkov

…uwer Rollup of 20 pull requests Successful merges: - rust-lang/rust#163483 (Bump bootstrap compiler to 1.100.0 beta) - rust-lang/rust#161380 (only rerun const eval in next-solver if the const actually references opaques) - rust-lang/rust#162900 (Some refactorings around metadata encoding) - rust-lang/rust#163461 (Improve diagnostic deduplication) - rust-lang/rust#163580 (Provide better doc code example for `UnixDatagram::bind_addr` and `UnixListener::bind_addr`) - rust-lang/rust#163584 ([triagebot] Ping me for debugger visualizer changes) - rust-lang/rust#159021 (windows-gnu: enable native TLS) - rust-lang/rust#161467 (wfcheck: name the item that discards an unused type parameter) - rust-lang/rust#162618 (trait_selection: Preserve eager normalization failures) - rust-lang/rust#162904 (Fix ICE for ambiguous candidates on method probing) - rust-lang/rust#163064 (Avoid computing overflowed goal chains for crate dependencies) - rust-lang/rust#163281 (Add `f16` inline ASM support to `spirv.rs`) - rust-lang/rust#163314 (move `#[macro_export]` on declarative macro check to `rustc_attr_parsing`) - rust-lang/rust#163360 ([rustdoc] Correctly handle rustc_allow_incoherent_impl on primitive methods) - rust-lang/rust#163385 (GVN transmutes of Immediate::Uninit to Immediate::Uninit) - rust-lang/rust#163405 (Remove some #[linkage] options) - rust-lang/rust#163530 (`const impl PartialEq` for `f16b`) - rust-lang/rust#163581 (do not suggest precise capturing when the opaque span is in a macro expansion) - rust-lang/rust#163590 (Make `AllocatorNightly` less clever) - rust-lang/rust#163599 (Add union pattern reference change to relnotes)

rustbot assigned petrochenkov Sep 17, 2026

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Sep 17, 2026

aerooneqq commented Sep 17, 2026

View reviewed changes

Comment thread compiler/rustc_metadata/src/rmeta/encoder.rs Outdated

rustbot assigned Zalathar and unassigned petrochenkov Sep 17, 2026

aerooneqq force-pushed the record-tables-through-macros branch from cc26ffb to 020ffe1 Compare September 18, 2026 09:04

aerooneqq mentioned this pull request Sep 25, 2026

Use LocalDefId as an index in metadata encoding #163321

Open

petrochenkov mentioned this pull request Sep 28, 2026

Promoting Parallel Front End rust-lang/goals#121

Open

petrochenkov approved these changes Sep 29, 2026

View reviewed changes

rustbot assigned petrochenkov Sep 30, 2026

rust-bors Bot added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Sep 30, 2026

rustbot unassigned Zalathar Sep 30, 2026

rust-bors Bot removed the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Sep 30, 2026

bjorn3 reviewed Sep 30, 2026

View reviewed changes

Comment thread compiler/rustc_metadata/src/rmeta/decoder.rs Outdated

bjorn3 reviewed Sep 30, 2026

View reviewed changes

Comment thread compiler/rustc_metadata/src/rmeta/encoder.rs

JonathanBrouwer mentioned this pull request Sep 30, 2026

Rollup of 14 pull requests #163540

Closed

rust-bors Bot added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Sep 30, 2026

JonathanBrouwer mentioned this pull request Sep 30, 2026

Rollup of 18 pull requests #163560

Closed

JonathanBrouwer mentioned this pull request Sep 30, 2026

Rollup of 18 pull requests #163566

Closed

aerooneqq mentioned this pull request Oct 1, 2026

Remap def indices for deterministic metadata encoding #162809

Draft

JonathanBrouwer mentioned this pull request Oct 1, 2026

Rollup of 9 pull requests #163595

Closed

JonathanBrouwer mentioned this pull request Oct 1, 2026

Rollup of 9 pull requests #163596

Closed

GuillaumeGomez mentioned this pull request Oct 1, 2026

Rollup of 10 pull requests #163600

Closed

GuillaumeGomez mentioned this pull request Oct 1, 2026

Rollup of 11 pull requests #163601

Closed

JonathanBrouwer mentioned this pull request Oct 1, 2026

Rollup of 20 pull requests #163609

Merged

rust-bors Bot merged commit 1232437 into rust-lang:main Oct 2, 2026
13 checks passed

rustbot added this to the 1.101.0 milestone Oct 2, 2026

Uh oh!

Conversation

aerooneqq commented Sep 17, 2026 • edited by rustbot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

petrochenkov commented Sep 17, 2026

Uh oh!

rustbot commented Sep 17, 2026

Uh oh!

Zalathar commented Sep 17, 2026

Uh oh!

aerooneqq commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Zalathar commented Sep 18, 2026

Uh oh!

aerooneqq commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Zalathar commented Sep 18, 2026

Uh oh!

petrochenkov commented Sep 29, 2026

Uh oh!

Zalathar commented Sep 30, 2026

Uh oh!

petrochenkov commented Sep 30, 2026

Uh oh!

rust-bors Bot commented Sep 30, 2026

Uh oh!

Uh oh!

Uh oh!

bjorn3 commented Sep 30, 2026

Uh oh!

aerooneqq commented Sep 30, 2026

Uh oh!

rust-bors Bot commented Sep 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

aerooneqq commented Sep 17, 2026 •

edited by rustbot

Loading

aerooneqq commented Sep 17, 2026 •

edited

Loading

aerooneqq commented Sep 18, 2026 •

edited

Loading