macOS: opt out of AppKit window restoration, catch SIGTRAP

[Grantim]

Since 2026-09-29 13:35 UTC, MeshInspectorCode's "Run Start-and-Exit Test" hangs on every job that runs on the self-hosted runner MACBOOK-PRO-16-2021-TBI-2 (macOS 15, arm64), on PRs and master alike, until the step's 3-minute timeout. The same commits pass on the other macOS runners. The log stops inside glfwInit(): Log file: ... is printed, glfwInit succeeded never is, and the process sits at 0% CPU. MeshViewer, which is not an app bundle, still gets through glfwInit() on the same runner.

The hangs started right after UI tests on two MeshInspectorCode branches killed MeshInspector.app with SIGTRAP (rc=-5), five times in each run, and both times the next launch hung. The crashes came from the macOS atexit bug that #6864 introduced and #6969 already fixed; this PR is about the hang they left behind. glfwInit() calls [NSApp run] to finish launching, and inside that call AppKit's window restoration (NSPersistentUIManager) can block. After an abnormal exit it may show a "reopen windows?" alert that nobody answers, or wait on the restoration service (talagentd). Each hung run is then killed by the timeout, which is another abnormal exit, so the runner likely stays stuck.

• MRMacOSOpenDocumentsHandler.mm: its +load, which already runs before main, now registers ApplePersistence = NO, so AppKit skips window restoration at launch. GLFW creates every window with setRestorable:NO, so nothing is lost. registerDefaults: only sets an in-memory fallback, the same way GLFW sets ApplePressAndHoldEnabled. An Electron app (zero-abd/teamree, PR 170) fixed a launch hang in the same AppKit code with the same registration.
• MRStacktrace.cpp: printStacktraceOnCrash now also handles SIGTRAP. On arm64, __builtin_trap() raises SIGTRAP (on x86-64 it is SIGILL, which is already handled), so those crashes skipped the handler. TBI-2's logs have no "Crash signal" line or stack trace for them, while the other runners logged theirs for the same bug. Debuggers get breakpoint traps before the process does, so debugging is unaffected.

Side effects

• SIGTRAP crashes get an exit code. They now end in std::exit(5), so the exit code is 5 instead of death by signal (rc=-5). MeshInspectorCode's UI tests only check for 0, so they fail the same way; only the printed code changes.
• Cleanup now runs where the process used to die at once. std::exit runs static destructors and atexit handlers. In MeshInspector, ~EmbeddedPython then finalizes the embedded interpreter, which runs Python atexit callbacks such as the viewer shutdown from Python viewer: live at the macOS prompt through PyOS_InputHook, shut down on interpreter exit #6864. SIGSEGV, SIGABRT and SIGILL already take this path, and it can crash again inside the handler and hang: on 2026-09-29, MacBook-Pro-Daniil went 11 → 6 → 11 in the handler and then sat until the 500 s timeout.
• Handling SIGTRAP may be enough by itself. In the runs checked, crashes that ended with an exit code never blocked the next launch. The restoration opt-out is still needed for exits the handler can't catch, such as a CI timeout's SIGKILL. It also means a green TBI-2 after the next bump proves the opt-out only if the machine is still stuck when the bump lands. If TBI-2 is cleaned first, a green run shows nothing until the app crashes there again, and even then it won't say which change prevented the hang.
• Host apps lose AppKit window restoration. The +load runs in every process that loads MRViewer, including Python with meshlib's viewer, so restoration is off for the whole process, not only for GLFW windows. It is in-memory only, and a host that wants restoration back can register its own ApplePersistence value before launch. The same +load already sets NSTreatUnknownArgumentsAsOpen for every host, and that one is saved to disk.

Test plan

• After the next regular MeshLib bump in MeshInspectorCode, "Run Start-and-Exit Test" on TBI-2 no longer hangs after the app crashes in UI tests.
• macOS jobs here stay green.

🤖 Generated with Claude Code