ICE in sparc64 `fn_abi_of_instance`

[RalfJung]

I tried the following code:

#[repr(transparent)]
#[derive(Default)]
struct Wrapper2<T>((), T);

type T = (i8, i16, f32);

extern "C" fn test(x: Wrapper2<T>) {}

fn main() {
    test(Default::default());
}

Building this for the sparc64-unknown-linux-gnu target causes an ICE:

thread 'rustc' panicked at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/compiler/rustc_abi/src/lib.rs:563:13:
Size::sub: 8 - 12 would result in negative size
stack backtrace:
   0:     0x7f514596316c - std::backtrace_rs::backtrace::libunwind::trace::h7d5a50c97105e9c9
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x7f514596316c - std::backtrace_rs::backtrace::trace_unsynchronized::hf283bd0ba71b8b19
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7f514596316c - std::sys_common::backtrace::_print_fmt::hbc3f1af55ab433e1
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x7f514596316c - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h662df30e888949cd
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x7f51459c941c - core::fmt::rt::Argument::fmt::hf59806e96303ebc5
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/core/src/fmt/rt.rs:138:9
   5:     0x7f51459c941c - core::fmt::write::hf7279be296576ae3
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/core/src/fmt/mod.rs:1094:21
   6:     0x7f5145955c8e - std::io::Write::write_fmt::h1ecf2bec14816818
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/io/mod.rs:1714:15
   7:     0x7f5145962f54 - std::sys_common::backtrace::_print::hceca1ed09536a7dd
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x7f5145962f54 - std::sys_common::backtrace::print::hb3d0e53175a9dc58
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x7f514596604a - std::panicking::panic_hook_with_disk_dump::{{closure}}::hb5593ac8317ecfc8
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/panicking.rs:280:22
  10:     0x7f5145965d45 - std::panicking::panic_hook_with_disk_dump::hd03ff9ecbda8604b
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/panicking.rs:314:9
  11:     0x7f5148b43ae9 - rustc_driver_impl[8296f4cfca142f1b]::install_ice_hook::{closure#0}
  12:     0x7f5145966913 - <alloc::boxed::Box<F,A> as core::ops::function::Fn<Args>>::call::h70ed5b57462ef04a
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/alloc/src/boxed.rs:2021:9
  13:     0x7f5145966913 - std::panicking::rust_panic_with_hook::h7bf02c396cdadbfd
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/panicking.rs:757:13
  14:     0x7f5145966681 - std::panicking::begin_panic_handler::{{closure}}::hecf382f929251efa
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/panicking.rs:631:13
  15:     0x7f5145963696 - std::sys_common::backtrace::__rust_end_short_backtrace::hc87b776526608b83
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/sys_common/backtrace.rs:170:18
  16:     0x7f51459663c2 - rust_begin_unwind
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/panicking.rs:619:5
  17:     0x7f51459c57a5 - core::panicking::panic_fmt::hab5931093cddd316
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/core/src/panicking.rs:72:14
  18:     0x7f5149824c2c - rustc_target[342b34254fdb499e]::abi::call::sparc64::classify_arg::<rustc_middle[d24a3bf7d1107f60]::ty::Ty, rustc_middle[d24a3bf7d1107f60]::ty::layout::LayoutCx<rustc_middle[d24a3bf7d1107f60]::ty::context::TyCtxt>>
  19:     0x7f5149825183 - rustc_target[342b34254fdb499e]::abi::call::sparc64::compute_abi_info::<rustc_middle[d24a3bf7d1107f60]::ty::Ty, rustc_middle[d24a3bf7d1107f60]::ty::layout::LayoutCx<rustc_middle[d24a3bf7d1107f60]::ty::context::TyCtxt>>
  20:     0x7f51472693c6 - <rustc_target[342b34254fdb499e]::abi::call::FnAbi<rustc_middle[d24a3bf7d1107f60]::ty::Ty>>::adjust_for_foreign_abi::<rustc_middle[d24a3bf7d1107f60]::ty::layout::LayoutCx<rustc_middle[d24a3bf7d1107f60]::ty::context::TyCtxt>>
  21:     0x7f5147260c3e - rustc_ty_utils[eb4842348932a0d7]::abi::fn_abi_new_uncached
  22:     0x7f514725ebbf - rustc_ty_utils[eb4842348932a0d7]::abi::fn_abi_of_instance
  23:     0x7f5146c0f03f - rustc_query_impl[758f86c15081ebe2]::plumbing::__rust_begin_short_backtrace::<rustc_query_impl[758f86c15081ebe2]::query_impl::fn_abi_of_instance::dynamic_query::{closure#2}::{closure#0}, rustc_middle[d24a3bf7d1107f60]::query::erase::Erased<[u8; 16usize]>>
  24:     0x7f5147320c19 - rustc_query_system[9e084a0952f60d7c]::query::plumbing::try_execute_query::<rustc_query_impl[758f86c15081ebe2]::DynamicConfig<rustc_query_system[9e084a0952f60d7c]::query::caches::DefaultCache<rustc_middle[d24a3bf7d1107f60]::ty::ParamEnvAnd<(rustc_middle[d24a3bf7d1107f60]::ty::instance::Instance, &rustc_middle[d24a3bf7d1107f60]::ty::list::List<rustc_middle[d24a3bf7d1107f60]::ty::Ty>)>, rustc_middle[d24a3bf7d1107f60]::query::erase::Erased<[u8; 16usize]>>, false, false, false>, rustc_query_impl[758f86c15081ebe2]::plumbing::QueryCtxt, true>
  25:     0x7f514732036a - rustc_query_impl[758f86c15081ebe2]::query_impl::fn_abi_of_instance::get_query_incr::__rust_end_short_backtrace
  26:     0x55ff00dfab6d - <rustc_const_eval[2a965be9d5d21d19]::interpret::eval_context::InterpCx<miri[7af406f97ef40a48]::machine::MiriMachine> as rustc_middle[d24a3bf7d1107f60]::ty::layout::FnAbiOf>::fn_abi_of_instance
  27:     0x55ff00e089c2 - <rustc_const_eval[2a965be9d5d21d19]::interpret::eval_context::InterpCx<miri[7af406f97ef40a48]::machine::MiriMachine> as miri[7af406f97ef40a48]::concurrency::thread::EvalContextExt>::run_threads
  28:     0x55ff00d44434 - miri[7af406f97ef40a48]::eval::eval_entry
  29:     0x55ff00ca29f7 - <rustc_middle[d24a3bf7d1107f60]::ty::context::GlobalCtxt>::enter::<<miri[1b66db6e0a3da615]::MiriCompilerCalls as rustc_driver_impl[8296f4cfca142f1b]::Callbacks>::after_analysis::{closure#0}, ()>
  30:     0x55ff00c9a914 - <miri[1b66db6e0a3da615]::MiriCompilerCalls as rustc_driver_impl[8296f4cfca142f1b]::Callbacks>::after_analysis
  31:     0x7f51480920ad - <rustc_interface[d452972be578bd9f]::interface::Compiler>::enter::<rustc_driver_impl[8296f4cfca142f1b]::run_compiler::{closure#1}::{closure#2}, core[540f925e73d2d0a2]::result::Result<core[540f925e73d2d0a2]::option::Option<rustc_interface[d452972be578bd9f]::queries::Linker>, rustc_span[4781cfcf330d4c16]::ErrorGuaranteed>>
  32:     0x7f514808f3b8 - std[81d5a1178575a592]::sys_common::backtrace::__rust_begin_short_backtrace::<rustc_interface[d452972be578bd9f]::util::run_in_thread_with_globals<rustc_interface[d452972be578bd9f]::interface::run_compiler<core[540f925e73d2d0a2]::result::Result<(), rustc_span[4781cfcf330d4c16]::ErrorGuaranteed>, rustc_driver_impl[8296f4cfca142f1b]::run_compiler::{closure#1}>::{closure#0}, core[540f925e73d2d0a2]::result::Result<(), rustc_span[4781cfcf330d4c16]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[540f925e73d2d0a2]::result::Result<(), rustc_span[4781cfcf330d4c16]::ErrorGuaranteed>>
  33:     0x7f514808eb35 - <<std[81d5a1178575a592]::thread::Builder>::spawn_unchecked_<rustc_interface[d452972be578bd9f]::util::run_in_thread_with_globals<rustc_interface[d452972be578bd9f]::interface::run_compiler<core[540f925e73d2d0a2]::result::Result<(), rustc_span[4781cfcf330d4c16]::ErrorGuaranteed>, rustc_driver_impl[8296f4cfca142f1b]::run_compiler::{closure#1}>::{closure#0}, core[540f925e73d2d0a2]::result::Result<(), rustc_span[4781cfcf330d4c16]::ErrorGuaranteed>>::{closure#0}::{closure#0}, core[540f925e73d2d0a2]::result::Result<(), rustc_span[4781cfcf330d4c16]::ErrorGuaranteed>>::{closure#1} as core[540f925e73d2d0a2]::ops::function::FnOnce<()>>::call_once::{shim:vtable#0}
  34:     0x7f5145971375 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h2aa91af32e72c49a
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/alloc/src/boxed.rs:2007:9
  35:     0x7f5145971375 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h470b309ec186d141
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/alloc/src/boxed.rs:2007:9
  36:     0x7f5145971375 - std::sys::unix::thread::Thread::new::thread_start::h6e3be0b07716d9c0
                               at /rustc/59a8294849358a878a72358aa6d5fe5b9d312867/library/std/src/sys/unix/thread.rs:108:17
  37:     0x7f51455dd3ec - start_thread
                               at ./nptl/pthread_create.c:444:8
  38:     0x7f514565da1c - __GI___clone3
                               at ./misc/../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
  39:                0x0 - <unknown>

error: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/miri/issues/new

note: rustc 1.74.0-nightly (59a829484 2023-08-30) running on x86_64-unknown-linux-gnu

note: compiler flags: --crate-type bin -C embed-bitcode=no -C debuginfo=2 -C incremental=[REDACTED]

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [fn_abi_of_instance] computing call ABI of `test`
end of query stack

Miri caused an ICE during evaluation. Here's the interpreter backtrace at the time of the panic:
note: the place in the program where the ICE was triggered
  --> src/main.rs:10:5
   |
10 |     test(Default::default());
   |     ^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: inside `main` at src/main.rs:10:5: 10:29

(I used Miri for this because I don't have cross-builds set up for that target, but I'm fairly sure the ICE would also happen with a regular rustc build.)

I am fairly sure that it is this subtraction that is overflowing:

rust/compiler/rustc_target/src/abi/call/sparc64.rs

Line 187 in 1bd0430

let mut rest_size = arg.layout.size - data.last_offset;

How do I find out who's maintaining that target so that I can ping the right people?

[RalfJung]

Cc @psumbera (based on git blame, not sure who is maintaining the sparc64 target -- it is sadly not documented)

[RalfJung]

This also affects the netbsd and openbsd targets.

[glaubitz]

FWIW, there are sparc64 machines available at the GCC Compile Farm which can be used for testing.

See: https://gcc.gnu.org/wiki/CompileFarm

The sparc64 Linux machines are currently offline but are expected to return shortly.

[psumbera]

I think I see the same on Solaris SPARC. The first bad version is 1.74. Version 1.73 seems to work.

[RalfJung]

It is possible that this did not ICE before #115277. However, that just means that the latent bug in the sparc ABI handling was hidden by the way our layout code chose to order zero-sized fields. We have no invariant on layouts saying that zero-sized fields can only occur at the beginning, so the bug needs to be fixed in sparc ABI adjustments.

[RalfJung]

In particular, here it is walking the fields in source declaration order:

rust/compiler/rustc_target/src/abi/call/sparc64.rs

Line 116 in 1bd0430

for i in 0..layout.fields.count() {

However, source order can be different from layout order. So this will not visit the fields in increasing order of offsets!

Maybe that's a problem, maybe it isn't. Unfortunately the ABI code has a bunch of very arbitrary-looking logic and basically no comments. (arg_scalar in particular has no comments at all and would need a lot of them to explain all the cases that are covered here. Also the exact meaning of the fields of Sdata is unclear.) So I can't really help figure out what is going on despite being intimately familiar with the Rust layout type, probably one needs to know the sparc ABI by heart to understand this.

[psumbera]

Do you think you can provide some example code which would demonstrate this SPARC64 ABI zero-sized fields issue? Right now I'm lost. I think I need something to play with.

(And yes. Someone with both Rust internals and SPARC64 ABI knowledge would be great to have here.)

[RalfJung]

The issue description contains an example that ICEs. It involves a zero-sized type, (). I think that is likely relevant but I have not investigated. I also have not tried whether reverting #115277 helps.