Compiler OOM when creating extremely large array of String using const with const evaluation

I tried this code:

pub fn other() {
    const S: String = String::new();
    let _ = [S; usize::MAX >> 5];
}

I expected to see this happen: Either an error about the value being too large for the current architecture, or nothing (because errors about too large arrays don't seem to happen for values bound to _)

Instead, this happened:

# rustc main.rs        
memory allocation of 13835058055282163688 bytes failed
zsh: IOT instruction (core dumped)  rustc main.rs

Meta

rustc --version --verbose:

rustc 1.61.0-nightly (532d3cda9 2022-02-23)
binary: rustc
commit-hash: 532d3cda90b8a729cd982548649d32803d265052
commit-date: 2022-02-23
host: x86_64-unknown-linux-gnu
release: 1.61.0-nightly
LLVM version: 14.0.0

Valgrind with trace-children=yes gives me the following location for the error

Backtrace

==2782416== Argument 'size' of function malloc has a fishy (possibly negative) value: -4611686018427387928
==2782416==    at 0x4845899: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==2782416==    by 0x6F4982E: _RNvMs_NtCsc8QzWtcr2xm_5alloc7raw_vecINtB4_6RawVecINtNtCsd3w0T8MxtOr_12rustc_middle3mir14ProjectionElemNtBO_5LocalNtNtBQ_2ty2TyEE11allocate_inCs3FhhpBtephy_19rustc_mir_transform (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x50D8F5F: _RNvXNtNtCsc8QzWtcr2xm_5alloc3vec14spec_from_iterINtB4_3VecTNtNtCsd3w0T8MxtOr_12rustc_middle3mir5PlaceINtNtCsffm2ZuYuepl_4core6option6OptionuEEEINtB2_12SpecFromIterBU_INtNtNtNtB1G_4iter8adapters3map3MapINtNtNtB1G_3ops5range5RangeyENCNvMs0_NtCsdKNwjzyVHJ1_18rustc_mir_dataflow15elaborate_dropsINtB3O_8DropCtxtNtNtCs3FhhpBtephy_19rustc_mir_transform4shim18DropShimElaboratorE19open_drop_for_array0EE9from_iterB4Z_ (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x6F08403: _RNvMs0_NtCsdKNwjzyVHJ1_18rustc_mir_dataflow15elaborate_dropsINtB5_8DropCtxtNtNtCs3FhhpBtephy_19rustc_mir_transform4shim18DropShimElaboratorE19open_drop_for_arrayB1f_ (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x6F05E89: _RNvMs0_NtCsdKNwjzyVHJ1_18rustc_mir_dataflow15elaborate_dropsINtB5_8DropCtxtNtNtCs3FhhpBtephy_19rustc_mir_transform4shim18DropShimElaboratorE14elaborate_dropB1f_.llvm.4668469828716773882 (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x6F4343A: _RNvNtCs3FhhpBtephy_19rustc_mir_transform4shim9make_shim.llvm.8089105803349852858 (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x71DB720: _RINvNtNtCsjJfV0yDsyeG_18rustc_query_system5query8plumbing17try_execute_queryNtNtCsgV5xg9Y7H18_16rustc_query_impl8plumbing9QueryCtxtINtNtB4_6caches10ArenaCacheNtNtNtCsd3w0T8MxtOr_12rustc_middle2ty8instance11InstanceDefNtNtB2C_3mir4BodyEEB1g_.llvm.2504427033025671314 (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x7265451: _RINvNtNtCsjJfV0yDsyeG_18rustc_query_system5query8plumbing9get_queryNtNtCsgV5xg9Y7H18_16rustc_query_impl7queries9mir_shimsNtNtB17_8plumbing9QueryCtxtEB17_ (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x72763B3: _RNvXs8A_CsgV5xg9Y7H18_16rustc_query_implNtB6_7QueriesNtNtNtCsd3w0T8MxtOr_12rustc_middle2ty5query11QueryEngine9mir_shims (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x6C322F1: _RNvMsE_NtCsd3w0T8MxtOr_12rustc_middle2tyNtNtB5_7context6TyCtxt12instance_mir (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x61A1915: _RNvNtCsl186E7FnW1R_18rustc_monomorphize9collector18collect_neighbours (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416==    by 0x619CAC3: _RNvNtCsl186E7FnW1R_18rustc_monomorphize9collector17collect_items_rec (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)
==2782416== 
memory allocation of 13835058055282163688 bytes failed
==2782416== 
==2782416== Process terminating with default action of signal 6 (SIGABRT): dumping core
==2782416==    at 0x8BFE34C: __pthread_kill_implementation (in /usr/lib/libc.so.6)
==2782416==    by 0x8BB14B7: raise (in /usr/lib/libc.so.6)
==2782416==    by 0x8B9B533: abort (in /usr/lib/libc.so.6)
==2782416==    by 0x88A8876: std::sys::unix::abort_internal (mod.rs:259)
==2782416==    by 0x8862E85: std::process::abort (process.rs:2003)
==2782416==    by 0x889B12D: rust_oom (alloc.rs:330)
==2782416==    by 0x88EB146: __rg_oom (alloc.rs:411)
==2782416==    by 0x88EB0C6: alloc::alloc::handle_alloc_error::rt_error (alloc.rs:377)
==2782416==    by 0x88EAE96: core::ops::function::FnOnce::call_once (function.rs:227)
==2782416==    by 0x88EB5C5: core::intrinsics::const_eval_select (intrinsics.rs:2366)
==2782416==    by 0x8864585: alloc::alloc::handle_alloc_error (alloc.rs:381)
==2782416==    by 0x6F49870: _RNvMs_NtCsc8QzWtcr2xm_5alloc7raw_vecINtB4_6RawVecINtNtCsd3w0T8MxtOr_12rustc_middle3mir14ProjectionElemNtBO_5LocalNtNtBQ_2ty2TyEE11allocate_inCs3FhhpBtephy_19rustc_mir_transform (in /home/jess/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/librustc_driver-920796763ff499ad.so)

Which I traced down to probably this line. I'm assuming size is the array size here, but I have not verified that.

rust/compiler/rustc_mir_dataflow/src/elaborate_drops.rs

Lines 737 to 751 in c8a49fc

    
           let fields: Vec<(Place<'tcx>, Option<D::Path>)> = (0..size) 
        
               .map(|i| { 
        
                   ( 
        
                       tcx.mk_place_elem( 
        
                           self.place, 
        
                           ProjectionElem::ConstantIndex { 
        
                               offset: i, 
        
                               min_length: size, 
        
                               from_end: false, 
        
                           }, 
        
                       ), 
        
                       self.elaborator.array_subpath(self.path, i, size), 
        
                   ) 
        
               }) 
        
               .collect();

Cc @rust-lang/wg-const-eval

Triage: This seems fixed now:

$ cat src/main.rs
pub fn main() {
    const S: String = String::new();
    let _ = [S; usize::MAX >> 5];
}
$ cargo build
   Compiling minimal v0.1.0 (/home/martin/src/bin)
error: values of the type `[String; 576460752303423487]` are too big for the current architecture

error: could not compile `minimal` (bin "minimal") due to 1 previous error

I'm going to assume a regression test has been added when this was fixed, because I would be very surprised if that wasn't the case. Closing.

I think that just indicates that we now properly skip const-eval if the type is truly unrepresentable.

But for a type that's big but within the size limit, I don't think anything got fixed here. Should be easy to reproduce by making the array size smaller until the "too big for the current architecture" error disappears.

Oh right, this isn't actually a huge const, just a huge array of consts.

Still, there may or may not have been a regression test added.

Ah, yes that looks very similar. Thanks for digging this out!
I agree this can be closed then.

Navigation Menu

Repository navigation

Compiler OOM when creating extremely large array of `String` using `const` with const evaluation #94643

Meta

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	let fields: Vec<(Place<'tcx>, Option<D::Path>)> = (0..size)
	.map(\|i\| {
	(
	tcx.mk_place_elem(
	self.place,
	ProjectionElem::ConstantIndex {
	offset: i,
	min_length: size,
	from_end: false,
	},
	),
	self.elaborator.array_subpath(self.path, i, size),
	)
	})
	.collect();

Compiler OOM when creating extremely large array of String using const with const evaluation #94643

Description

Meta

Activity

RalfJung commented on Mar 5, 2022

Enselic commented on Mar 1, 2024

RalfJung commented on Mar 1, 2024

RalfJung commented on Mar 1, 2024

RalfJung commented on Mar 1, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Compiler OOM when creating extremely large array of `String` using `const` with const evaluation #94643