Refuse native addons built for the standard V8 ABI on pointer-compression Node runtimes, instead of crashing on first use

[kriszyp]

⊙ Problem

V8 pointer compression changes V8's C++ ABI (object layouts in the public headers) without changing NODE_MODULE_VERSION. A pointer-compression (pc) Node 24 still reports ABI 137. So on the harper-pro pc image (harperfast/harper-pro:<ver>-pc), which Fabric is starting to run, an addon that links V8's C++ API (NAN or direct v8::) and was built for the standard ABI passes Node's own check and loads. It then SIGSEGVs the whole process, every thread with it, the first time it is used. On a pc Node 24.18 the stock uws_linux_x64_137.node loads, creates an App(), and crashes (exit 139) on its first served request.

Application components can bring such prebuilds. Today's guards miss them: one checks uWS only (server/serverHelpers/uwsServer.ts), and the other is a build-time nm scan of the image's own tree in harper-pro. Node-API addons are ABI-stable and unaffected. Refs #919.

❓ Your call: Is the requirement warranted? As specified, yes. The load is the only point that sees what a component installs: deploy-time npm install, a vendored node_modules, harper dev, mounted directories. The adjudicator's do-less alternative was to extend the uWS startup check into "a pc runtime may only boot from the marked harper-pro image". That does not help, because components install their own addons at runtime inside that image.

💡 Solution

On a runtime with process.config.variables.v8_enable_pointer_compression === 1, Harper wraps process.dlopen in every thread before any component code runs. process.dlopen is the call behind every .node load: CJS require, createRequire, and ESM addon import alike. A .node loads only when one of these holds:

1. Node-API: its .dynsym has no undefined V8 C++ symbols (_ZN2v8… / _ZNK2v8…). A pure-JS ELF64 reader does the check, so there is no binutils dependency.
2. Marked: its package directory carries .pointer-compression-build. The pc image swaps in pc-built uWS and @datadog/pprof binaries and marks them.
3. Built here under pc: it sits under a build/ whose node-gyp config.gypi records "v8_enable_pointer_compression": 1, and it is strictly newer than that config.gypi. re2 and segfault-handler in the image qualify, as does anything rebuilt with npm rebuild <pkg> --build-from-source.

Anything else is refused before dlopen runs, and the error says which file and package and how to fix it:

IncompatibleNativeAddonError: Native addon …/components/bad-addon/native/uws_linux_x64_137.node (package bad-addon) links V8's C++ API built for the standard Node.js ABI, which this pointer-compression Node.js runtime does not share; loading it would crash the process on first use. Install a Node-API version of bad-addon or a build compiled for pointer compression, rebuild it from source under this Node.js (npm rebuild bad-addon --build-from-source), or run on a standard Node.js build.

The error carries code: 'ERR_DLOPEN_FAILED' and statusCode: 500, so code that already falls back to JS when an addon cannot load treats a refusal the same way. On a standard runtime nothing is installed: the cost is one property read when the module loads.

Companion docs: HarperFast/documentation#725 adds a "Native Addons on Pointer-Compression Runtimes" section to the Module Loading reference and a 5.4 release-notes entry.

⚠️ Look hardest: install order. The guard must be the first module of each thread. On the main thread, harper_logger.ts requires segfault-handler (a V8 C++ API addon) inside its module-scope initLogSettings(). That is why the guard is the first import of the entry points rather than of manageThreads.js, where the existing workerProcessGuard precedent sits. In workers it is the first --require, which runs before threads.preload and threads.preloadRequire.

⚖️ Alternatives

Planning review: Framing-Verdict: better-alternative-exists (codex). I resolved it rather than clearing it. I overruled the proposed alternative, replacing rule 3 with a per-binary "build completed" marker that the producer writes: npm rebuild <pkg> --build-from-source, the fix the error tells users to run, writes config.gypi and no marker, so every user source rebuild would stay refused. I adopted its narrowings:

• config.gypi counts only for a binary inside that build/ tree, not a sibling prebuilds/;
• the binary must be newer than config.gypi;
• an ELF64 the reader cannot inspect is refused;
• only admissions are cached, revalidated against the binary's identity;
• diagnostics use builtins only.

Approach	Why not
Scan apps at deploy time (host-manager, deploy_component)	Components run npm install inside the container, and can arrive by other routes. Only the loader sees every load.
Give the pc Node its own NODE_MODULE_VERSION	node-gyp builds against the nodejs.org headers for process.version (ABI 137), so re2, segfault-handler and the pc uWS would all be refused. It would also protect only our image, and Node's message would point users at the wrong fix.
Force npm_config_build_from_source in the image	@datadog/pprof's npm package has no sources, uWS has no npm package, and a component can ship a prebuilt node_modules.
Shell out to nm at load time	Slim images have no binutils, and it would spawn a process per addon per thread inside the synchronous loader.

❓ Your call: Rule 3 trusts timestamps. A standard-ABI binary copied into build/Release/ after a pc configure, with a fresh mtime, is admitted. So is one carrying a future mtime from an archive. node-gyp's own configure → compile → link order and npm's extraction-time mtimes both fail safe, including the failed-compile case, which is tested. The alternative is a per-binary attestation written by producers: it is stricter, but it changes harper-pro's producer contract and does not recognise a plain npm rebuild. Switching later is a local change to isPointerCompressionBuild.

❓ Your call: Files that are not ELF64 (Mach-O, PE) are admitted, as the task specified, because pc images are Linux-only and a 64-bit Linux dlopen rejects them itself. On a self-built macOS or Windows pc Node, the global guard therefore protects nothing. The uWS check still refuses an unmarked uWS binary there, because uWS always links V8's C++ API. Refusing unreadable formats instead would also refuse rocksdb-js, so Harper could not start there at all. A Mach-O/PE symbol reader is the real fix if that platform ever matters.

❓ Your call: Refusals reuse Node's ERR_DLOPEN_FAILED, so existing fallbacks keep working; cbor-x and msgpackr catch any error. The cost is that callers cannot tell "would have crashed" apart from "missing or corrupt" by code; the class name still differs. This is cheap to change before release.

❓ Your call: Refuse rather than redirect. Off-image, node-gyp-build-optional-packages prefers an abi137 prebuild over a napi one. On a pc Node 24 it therefore picks cbor-extract's V8-API build, the guard refuses it, and cbor-x falls back to its JS decoder: slower, but correct. The pc image is unaffected, because its Dockerfile already deletes node.abi*.node. A fix belongs in that loader (follow-up), not in a guard that rewrites load paths.

❓ Your call: Install mechanism. The guard is the first import of each entry point plus a worker --require. The alternative is NODE_OPTIONS=--require <guard> set by the pc image: that would also cover forked children and run before an operator's own NODE_OPTIONS preloads, but it is image-level config, and core running on any other pc Node would be unprotected. NODE_OPTIONS preloads are out of scope here: they run in the main thread before Harper's entry module, where no Harper code can precede them.

🔧 Changes

Product and architecture tour

Which native addons may load on a pointer-compression runtime

On a runtime whose process.config.variables.v8_enable_pointer_compression is 1, no .node that links V8's C++ API without evidence of a pointer-compression build is dlopened by Harper or by anything Harper loads after its entry module. On a standard runtime nothing is wrapped.

Addon	Verdict	Evidence
rocksdb-js, argon2, lmdb/msgpackr-extract napi builds	loads	no undefined V8 C++ symbols
pc-built uWS, @datadog/pprof in the pc image	loads	.pointer-compression-build in the package directory
re2, segfault-handler built from source under the pc Node	loads	build/config.gypi records pointer compression and is older than the binary
A component's standard-ABI V8 prebuild (any NAN addon)	refused: IncompatibleNativeAddonError	none
cbor-extract abi137 build (picked off-image on Node 24)	refused; cbor-x uses its JS decoder	none
An ELF64 whose dynamic symbols cannot be read	refused unless marker or build evidence	none
Mach-O / PE (not loadable by a 64-bit Linux dlopen)	left to dlopen	not inspected

• nativeAddonAbi.ts:144 — the rules, in cost order: symbol scan, marker, node-gyp build, then non-ELF files left to dlopen
• nativeAddonAbi.ts:91 — exactly 24-byte entries and a non-empty table, or the file is uninspectable
• nativeAddonAbi.ts:165 — rule 3: the nearest build/, a regular config.gypi, strictly older than the binary

Where the guard installs, and why first

The guard precedes every native load in each thread

The main thread's logger loads segfault-handler at module scope, and worker preloads may bring APM natives, so the guard sits ahead of both.

flowchart TD
    A[bin/harper.ts or index.ts] --> B[nativeAddonGuard wraps process.dlopen]
    B --> C[harper_logger loads segfault-handler]
    B --> D[startWorker]
    D --> E[worker: --require nativeAddonGuard]
    E --> F[--require threads.preloadRequire]
    F --> G[--import threads.preload]
    G --> H[threadServer and components]

Loading

Each worker has its own process, so the main thread's wrapper does not reach it, and argv --requires run in order and before every --import. These were verified on the pc Node 24.18; there a threads.preloadRequire of a stock V8 prebuild was refused in every worker. NODE_OPTIONS preloads are out of scope: they run in the main thread before Harper's entry module.

• nativeAddonGuard.ts:13 — the wrapper: once per target, exact argument list forwarded
• manageThreads.js:1261 — the guard leads every worker's execArgv

What a component author sees

Before	After
A component that requires a standard-ABI V8 prebuild loads, then the first call SIGSEGVs the process and every thread with it	The component fails to load with IncompatibleNativeAddonError naming the file, the package and the fixes; Harper keeps serving

• nativeAddonAbi.ts:187 — the error: the nearest package.json name, the four fixes, ERR_DLOPEN_FAILED, statusCode 500

• utility/nativeAddonAbi.ts (new, Node builtins only, reusable by harper-pro's build-time script):
  • scanUndefinedV8Symbols reads only the ELF header, the section-header table, .dynsym and its string table. Reads are exact-length and positional, every offset is bounds-checked, each table is capped at 64 MiB, symbol entries must be 24 bytes, and either byte order is handled.
  • checkDescriptor applies the rules in cost order. isPointerCompressionBuild reads config.gypi only when it is a regular file older than the binary, because reading a FIFO would block inside dlopen.
  • assertNativeAddonLoadable opens the file once and takes the cache identity from that descriptor. It caches admissions only, and leaves a missing, unreadable or non-regular file to dlopen.
  • IncompatibleNativeAddonError builds its message from the nearest package.json.
  • isPointerCompressionRuntime is the single runtime predicate.

❓ Your call: Section-header trust. The reader takes .dynsym from the section headers, which nm reads, rather than from the PT_DYNAMIC/DT_SYMTAB view the loader uses. A binary built so that the two views disagree would be admitted. That is in line with the goal of preventing accidental crashes, not resisting hostile code, which can already run arbitrary JS. Reading PT_DYNAMIC is a bounded change if the threat model widens.

• server/threads/nativeAddonGuard.ts (new):
  • wraps process.dlopen once per thread, keyed by Symbol.for so a second copy of the module sees the existing wrapper;
  • forwards the exact argument list, because Node's DLOpen turns an explicit undefined flags argument into 0, which dlopen(3) rejects;
  • nativeAddonGuardExecArgv() returns ['--require', <this module>] on pc runtimes;
  • installs itself when loaded.
• Install points: the first import of bin/harper.ts, index.ts and launchServiceScripts/launchHarperDB.js. In server/threads/manageThreads.js it is also first in startWorker's execArgv, which covers every worker (HTTP, job, replication pool) and safe mode.
• server/serverHelpers/uwsServer.ts: the inline marker check becomes the shared rules, plus the existing HARPER_UWS_HTTP/HARPER_UWS_UDS hint.
  • The check stays at this site because uws.js rethrows any load error as a misleading "supports only Node.js versions" message.
  • A non-ELF uWS binary needs evidence too, because uWS always links V8's C++ API.
  • A missing or non-regular binary is left to uws.js.
• server/DESIGN.md, indexed from DESIGN.md: the invariant, the three rules, the producer obligation that comes with the marker, the install points, and the scope boundaries (NODE_OPTIONS, section-header trust).

✅ Verification

End to end on a real pointer-compression Node (v24.18.0, v8_enable_pointer_compression: 1, ABI 137), running this branch's dist. Docker is not available on this machine, so the harper-pro image itself was not built; the rules were checked against the image per the task.

Scenario	Result
Stock uWS prebuild, no guard	require and App() succeed; the first request → SIGSEGV, exit 139
harper run with 2 threads: REST PUT/GET, operations search_by_value	204 / 200 / 200
In each worker (threadId 1 and 2), the AddonProbe resource requires the stock uWS prebuild	IncompatibleNativeAddonError, ERR_DLOPEN_FAILED in both workers; argon2 (Node-API) loads
.node files mapped in /proc/<pid>/maps	rocksdb-js, argon2, bufferutil, lmdb napi, msgpackr-extract napi and the pc-built segfault-handler. cbor-extract's abi137 build was refused (cbor-x decodes through JS)
A component whose top-level code requires the stock prebuild	The component fails to load with the error above and never binds its port; Harper stays up and GET /Dog/1 → 200 after the restart
A threads.preloadRequire module that requires the stock prebuild	Refused in every worker before the preload body runs; no SIGSEGV. Harper then stops on the existing "exhausted restarts" path that any throwing preload takes
re2 (86 V8 symbols) and segfault-handler (34) built from source under the pc Node	Admitted by rule 3; new RE2('a(b+)').exec('xabbb') → bbb, and registerHandler works
pc-built uWS (71 symbols) and pprof (117) with the marker	Admitted; uWS serves HTTP 200; pprof exports TimeProfiler and heapProfiler
JS scanner vs nm -D --undefined-only on all 27 ELF addons in node_modules	Identical counts

Unit tests, all passing on standard Node 26.2 and on the pc Node 24.18:

• unitTests/utility/nativeAddonAbi.test.js covers:
  • the scanner on real argon2 (0 symbols) and uWS (above 0) prebuilds, which both packages ship for every OS;
  • synthetic ELF64 files built by unitTests/utility/fixtures/syntheticElf64.js: _ZN2v8/_ZNK2v8 counted, defined, napi_* and node:: symbols not counted, big-endian, no section headers, truncated, stride 48 or 2^32, empty table, ELF32, and Mach-O;
  • every admission rule, including the failed-compile, equal-mtime, sibling-prebuilds/ and config.gypi-as-a-directory refusals;
  • the error contents, refusal retry, revalidation after replacement, symlinks, and missing, unreadable and directory paths.
• unitTests/server/threads/nativeAddonGuard.test.js (with nativeAddonGuard-fixture.cjs) covers:
  • standard runtimes left untouched;
  • refusal before dlopen, exact-argument and receiver forwarding, and idempotence;
  • Node's real .node loader intercepted in a Worker;
  • the execArgv path, plus a pc-only test that installs from execArgv alone (it passes on the pc Node and is skipped on CI).
• unitTests/server/threads/preloadSafeMode.test.js now checks the configured preload paths rather than the bare --require flag, so it holds on a pc runtime, where the guard is always present.

Full gates at b8505e288, run locally on Node 26.2. The last commit only adds two regular-file checks, after which the guard's own test files and the pc-Node uWS refusal were re-run.

• test:unit:main: 6875 passing, 208 pending.
• test:unit:resources: 4132 passing, 54 pending.
• lmdb test:unit:bin: 326 passing (run because bin/ changed).
• test:integration:all: 2338 pass, 0 fail, 18 skipped, 6 cancelled.

CI on 1e0487fe5: 51 checks success, 5 skipped. The full Node 22/24/26 integration matrix, dispatched by hand because a PR push runs Node 24 only, passed 41/41 jobs: run 38072589594.

The 6 cancellations are all in ollama-backend.test.ts: it imports Harper sources as ESM, and utility/globalSchema.ts imports JSON without an import attribute (ERR_IMPORT_ATTRIBUTE_MISSING). That path is untouched by this diff, and the suite cancels the same way on this machine without it.

_{🤖 Generated by Claude Opus 5.5 (Claude Code); posted via @kriszyp.}

🤖 Generated with Claude Code

https://claude.ai/code/session_011m4iSgKNWY8toCDWmuoh9a

Release target: v5.4 — new feature (the pointer-compression rollout harper#919/#3072 targets), ships from main only, no backport
Related PRs: none found
Complexity: complicated

Origin — the dispatch brief this PR was written from

On a pointer-compression runtime, refuse native addons built for the standard V8 ABI instead of segfaulting

_{Dispatch: task harper-pc-native-addon-guard · queued by unknown · ran by claude/opus/xhigh · worker kzyp-xps-1}

_{Review-Coverage: authored=claude; ran=cursor-muse,gemini,codex; adjudicated=domain; declined=cursor-grok,cursor-composer,cursor-kimi; rounds=5; full=1 @ 4a74245}

_{Review-Attention: study ~11m (hot: manageThreads.js, nativeAddonGuard.ts; decisions: global-dlopen-wrapper, refusal-code, library-import-installs-guard, non-elf64-admitted, config-gypi-evidence, do-less-alternative) @ 4a74245}

[gemini-code-assist]

Code Review

This pull request introduces a native addon guard mechanism to prevent standard-V8-ABI addons from being loaded on Node.js runtimes with V8 pointer compression enabled, which would otherwise cause a segmentation fault. It implements a custom ELF64 parser in utility/nativeAddonAbi.ts to scan for undefined V8 C++ symbols in .node binaries, wraps process.dlopen via server/threads/nativeAddonGuard.ts in both the main thread and worker threads, and updates createUwsServer to perform early compatibility checks on the uWebSockets.js binary. Comprehensive unit tests and design documentation are also added. There are no review comments, and I have no feedback to provide.