pthread_exit crashes on threads created by std::thread::spawn in 1.84, not 1.83, breaking pyo3-log

Workaround to the production problem

In your Cargo.toml that is compiling your Python module, set

[profile.release]
debug = 0
lto = false

This prevents the 1.84 crashes.

However, there is still UB going on even with this setting.

The Production Problem

The crate pyo3-log installs a bridge that makes log functions call into Python. This means that all calls to logging::info! etc will take the GIL.

Python has a stage during interpreter shutdown where attempts to take the GIL will cause a pthread_exit. Python 3.14 (still Alpha today, targeted to be released by the end of this year) will change this in python/cpython#87135 - but that will take some time to reach people.

This means that if you have a Python program that uses a Rust library and pyo3-log, that spawning a Rust thread, that is calling logging::info! in a way unsynchronized with interpreter exit, you'll have unpredicatable crashes in 1.84.

Minified Program

This program:

use std::ffi::c_void;

extern "C" {
    fn pthread_exit(retval: *const c_void);
}
fn main() {
    std::thread::spawn(|| {
        unsafe { pthread_exit(std::ptr::null()); }
    });
    std::thread::sleep(std::time::Duration::from_secs(1));
}

when compiled with the following options

rustc +1.84 d.rs -Cpanic=abort -Cdebuginfo=limited

crashes with this confusing error

thread '<unnamed>' panicked at core/src/panicking.rs:223:5:
panic in a function that cannot unwind
stack backtrace:
   0: rust_begin_unwind
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869/library/std/src/panicking.rs:665:5
   1: core::panicking::panic_nounwind_fmt::runtime
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869/library/core/src/panicking.rs:119:22
   2: core::panicking::panic_nounwind_fmt
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869/library/core/src/intrinsics/mod.rs:3535:9
   3: core::panicking::panic_nounwind
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869/library/core/src/panicking.rs:223:5
   4: core::panicking::panic_cannot_unwind
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869/library/core/src/panicking.rs:315:5
   5: std::sys::pal::unix::thread::Thread::new::thread_start
             at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869/library/std/src/sys/pal/unix/thread.rs:99:9
   6: start_thread
   7: clone
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread caused non-unwinding panic. aborting.
Aborted

This crash happens:

Only on 1.84, not on 1.83
Only when debuginfo is enabled, but even if the binary is stripped.

When using -C panic=unwind instead, on all versions of the compiler, you get this error:

FATAL: exception not rethrown
Aborted (core dumped)

I seen the claim in Zulip (https://rust-lang.zulipchat.com/#narrow/channel/122651-general/topic/pthread_exit.20from.20a.20Rust-spawned.20thread) that this is undefined behavior, but I'll rather not break pyo3-log

See #129582 and #74990

Relevant discussions: rust-lang/unsafe-code-guidelines#404

Even if it's bad, I believe we need to have a working way to write Python libraries in Rust, even if the libraries spawn Rust threads that can potentially call into Python. debuginfo = 0 is a working way, but it feels too ugly.

The other argument is that Rust 1.84 has the MSRV-aware resolver, so e.g. reverting the change just for 1.84, leaving people stuck in Rust 1.84, is much better than leaving them stuck in 1.83.

I believe it would be possible to make pyo3-log marshal everything to a Python thread, but that would not work if there are non-pyo3-log causes of the problem.

@vorner

cc @vorner for the pyo3-log problem

As expected:
Bad/regressed (new) commit: 06d261d Auto merge of #129582 - nbdd0121:unwind, r=nnethercote
Good (old) commit: dd51276 Auto merge of #131796 - cuviper:no-waker-waker, r=Mark-Simulacrum

hang instead of pthread_exit during interpreter shutdown

@davidhewitt

@davidhewitt what do you think of PyO3/pyo3#4874 ?

Thanks, I've seen you've pushed that, I'm very short of time at the moment so please allow me a few days to find a moment to sit down and review fully.

Cool.

hang instead of pthread_exit during interpreter shutdown

hang instead of pthread_exit during interpreter shutdown (#4874)

hang instead of pthread_exit during interpreter shutdown

hang instead of pthread_exit during interpreter shutdown (#4874)

We handled a fix in PyO3 side for this, should we close this issue?

Yea, closing

tests: the interpreter-exit check opens its window, and says whether …

Navigation Menu

Repository navigation

pthread_exit crashes on threads created by std::thread::spawn in 1.84, not 1.83, breaking pyo3-log #135929

Meta

Workaround to the production problem

The Production Problem

Minified Program

9 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

pthread_exit crashes on threads created by std::thread::spawn in 1.84, not 1.83, breaking pyo3-log #135929

Description

Meta

Workaround to the production problem

The Production Problem

Minified Program

Activity

arielb1 commented on Jan 23, 2025

jieyouxu commented on Jan 23, 2025

arielb1 commented on Jan 23, 2025

arielb1 commented on Jan 23, 2025

arielb1 commented on Jan 23, 2025

9 remaining items

arielb1 commented on Jan 27, 2025

davidhewitt commented on Jan 28, 2025

arielb1 commented on Jan 28, 2025

davidhewitt commented on Apr 8, 2025

arielb1 commented on Apr 8, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions