Skip to content

Intermittent segfault (0xC0000005) during process teardown on Windows with file::memory:?cache=shared under concurrent processes #231

Open

Description

@ariasr47

Summary

On Windows, a Node process that has used a file::memory:?cache=shared libsql database intermittently dies with an access violation during teardown — after all JavaScript work has completed and after process.on("exit") handlers have already run. The process reports exit code 3221225477 (0xC0000005, STATUS_ACCESS_VIOLATION); spawned from a POSIX shell the same crash surfaces as SIGSEGV / exit 139.

Because it happens after JS shutdown, nothing observable from JavaScript sees it: no unhandledRejection, no uncaughtException, and an exit handler installed in the same process reports code 0 immediately before the crash.

It is concurrency-sensitive: it grows more frequent with more simultaneous processes and does not disappear at low concurrency.

Environment

OS Windows 11 Pro, 10.0.26200.9168, x64 (32 logical cores)
Node v26.7.0
libsql 0.5.29
@libsql/win32-x64-msvc 0.5.29
@libsql/client 0.15.15
drizzle-orm 0.45.2

Not a recent regression. Reproduced on @libsql/client 0.15.15, 0.15.4, and 0.14.0 at comparable rates, so it spans at least the 0.14 → 0.15 range.

How it shows up

The original symptom was a test suite (node --test, --test-concurrency=4, ~160 files) going red roughly 10–15% of runs, on a different file each time. In every case the file's own assertions all passed; only the file-level result failed. Node's TAP reporter carries the distinguishing detail:

not ok 76 - tests\rank-truth\write-table.test.ts
  ---
  duration_ms: 522.4409
  type: 'test'
  failureType: 'testCodeFailure'
  exitCode: 3221225477
  signal: ~
  error: 'test failed'
  code: 'ERR_TEST_FAILURE'
  ...

A genuine assertion failure carries code: 'ERR_ASSERTION' and no exitCode, so the two are cleanly distinguishable.

Reproduction

Reduced to this, which crashes roughly 1 run in 48 with 4 processes at a time:

// child.mjs — run several of these concurrently, repeatedly
const { ensureDb, getDb } = await import("./db.js"); // creates the schema, wraps the client in drizzle
await ensureDb();
const db = getDb();
await db.select().from(someTable).limit(5);
// client deliberately not closed — matches a worker process exiting
for r in $(seq 1 12); do
  for c in 1 2 3 4; do node child.mjs & done
  wait   # any nonzero exit here is the crash
done

db.js is a thin module that creates one client with createClient({ url: "file::memory:?cache=shared" }), caches it on globalThis, executes a multi-statement schema (~34 CREATE TABLE IF NOT EXISTS plus indexes) once, and wraps it with drizzle().

What I ruled out, with measurements

These are recorded because each one narrows where the fault is not:

  1. Escaping promise rejection. unhandledRejection and uncaughtException handlers armed across ~50 full runs: zero fired.
  2. The client never being closed. Closing the cached client explicitly on beforeExit — verified to actually execute, by confirming the client is present at exit without the change and absent with it — did not change the rate (2 crashed runs / 16 vs 2 / 14 baseline).
  3. A version regression. See the version list above; 0.14.0 crashed on its first run.
  4. Pure @libsql/client with a comparable workload. A script that creates the same file::memory:?cache=shared client, issues ~34 CREATE TABLE + 68 CREATE INDEX statements via executeMultiple, runs a write transaction and a read, and exits without closing did not crash in 60 runs at the same concurrency. A simpler version (50 inserts + one transaction) did not crash in 160 runs.

Point 4 is the interesting one: the crash needs something beyond the raw client usage. In my reproduction the additional ingredients are drizzle-orm wrapping the client and a schema-creation path that runs before it. I was not able to reduce it further, so the pure-client negative result may just mean I have not found the right shape rather than that the client is uninvolved.

Why I think it is native teardown

  • All JavaScript completes successfully; the process's own exit handler observes code 0.
  • The failing exit code is only visible to the parent process.
  • No JS-level handler can intercept it.
  • It is timing/contention sensitive rather than deterministic.

That pattern points at a native destructor or N-API finalizer running after JS shutdown, likely racing on the shared-cache in-memory database, but I have not captured a native stack trace and cannot name the frame.

What would help

If there is a supported way to deterministically release the native handle before process exit (beyond client.close(), which did not change the rate), that would be a usable workaround. Happy to run instrumented builds or capture a crash dump on this machine if that is useful.

Activity

  1. lucaspaniago commented on Sep 21, 2026

    @lucaspaniago

    Same crash here, and I can add a few data points that may narrow it down: it is not specific to file::memory:?cache=shared. It reproduces with a plain file-backed database, with no ORM, and calling client.close() does not prevent it.

    Environment

    OS Windows 11 Pro 10.0.26200, x64
    Node v22.15.0
    libsql 0.5.29
    @libsql/win32-x64-msvc 0.5.29
    @libsql/client 0.17.3

    Not reproducible on macOS (arm64) with the same code.

    Minimal reproduction (file-backed, client closed)

    // repro.js
    'use strict';
    const fs = require('node:fs');
    const os = require('node:os');
    const path = require('node:path');
    const { pathToFileURL } = require('node:url');
    const { createClient } = require('@libsql/client');
    
    async function oneDb(i) {
      const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'libsql-repro-'));
      const file = path.join(dir, `b${i}.db`);
      const client = createClient({ url: pathToFileURL(file).href });
      await client.execute('CREATE TABLE IF NOT EXISTS t (id INTEGER PRIMARY KEY, v TEXT)');
      const tx = await client.transaction('write');
      for (let n = 0; n < 20; n += 1) await tx.execute({ sql: 'INSERT INTO t(v) VALUES (?)', args: [`row-${n}`] });
      await tx.commit();
      await client.execute('SELECT COUNT(*) AS n FROM t');
      client.close();
      try { fs.rmSync(dir, { recursive: true, force: true }); } catch { /* file may still be held on Windows */ }
    }
    
    (async () => {
      for (let i = 0; i < 12; i += 1) await oneDb(i);
      console.log('REPRO_OK');
    })().catch(err => { console.error(err); process.exitCode = 1; });

    Run it 1000 times, 2 processes at a time, and count exit codes. Every crashing run prints REPRO_OK first, has an empty stderr, and then exits with 3221225477 (0xC0000005).

    Results (1000 runs per variant, 2 concurrent processes)

    Variant Crashes (0xC0000005)
    @libsql/client only (script above) 7 (0.7 %)
    Same workload with node:sqlite only (control) 0
    @libsql/client + node:sqlite on the same file 6 (0.6 %)
    Same, but never calling close() 6 (0.6 %)

    So the crash follows the libsql native binding; node:sqlite doing the same work never crashed, and closing or not closing the client makes no measurable difference.

    In a real test suite (each file opens ~18 temporary databases) the per-file rate is higher, 2–5 %, which matches the "different file goes red each time" pattern described above.

    Two things I tried that did not fix it

    1. Forcing GC before exit (--expose-gc, three gc() + setImmediate rounds in a root after() hook): made it worse in my measurements — 16/300 crashes vs 6/300 without it, same direction in a second smaller sample (7/100 vs 2/100). That suggests the native finalizers themselves are involved, rather than merely running too late.
    2. Skipping Node's environment teardown (process.reallyExit(code) from an exit listener): inconclusive — 18/300 vs 14/300 baseline on one test file, but 1/300 vs 6/300 on another.

    Note for anyone measuring this

    node --test <file> runs the file in a subprocess, so the parent exits with code 1 and the native crash is only visible as exitCode: 3221225477 inside the TAP output. Running the file directly (node file.test.js) exposes the real exit code.

    Happy to run further experiments on this Windows machine (e.g. a debug build, or collecting a crash dump) if that helps.

  2. maritonx commented on Oct 17, 2026

    @maritonx

    Another data point from CI, in case it helps narrow this down.

    • Where: GitHub Actions windows-latest, Node 22 and 24, libsql 0.5.29, @libsql/client 0.18.0, no ORM in the affected packages' tests.
    • How it runs: Vitest 5 with the default forks pool, so each test file runs in a child process that exits when the file is done.
    • Frequency: 5 crashes in about 200 CI runs over two weeks. Each was exit code 3221225477 (Worker exited unexpectedly … during started state) in a file whose tests had all passed. It hit three different packages, and never macOS or Linux.
    • What we checked: every client is closed (client.close()) before the file ends.

    We now rerun a package once when its worker exits with that code: https://github.com/maritonx/easy-cms/blob/main/.github/scripts/test-retry-crashes.mjs

  3. hyunjun6928-netizen commented on Oct 19, 2026

    @hyunjun6928-netizen

    Same crash here, reduced to plain libsql with nothing else: in-memory databases that are never closed. No @libsql/client, ORM, shared cache, file or SQL statement is needed. The process exits with 3221225477 (0xC0000005) and an empty stderr when those databases are finalized, after a garbage collection or at exit.

    This matches the double sqlite3_close_v2 in tursodatabase/libsql#2251 ("root cause 2" in #235). Its fix, tursodatabase/libsql#2261, is in the libsql crate 0.10.0 (2026-10-13), but no libsql-js release has it yet. The crate paths embedded in the Windows binaries show @libsql/win32-x64-msvc 0.5.29 built with libsql 0.9.30 (Neon 1.0.0) and 0.6.0-pre.43 with libsql 0.10.0-pre.4 (napi 2.16.17), and both crash below. I have no Rust toolchain on this machine, so I could not build against 0.10.0 to confirm the fix.

    Environment

    OS Windows 11 Pro 10.0.26200, x64, AMD Ryzen 9 9950X (16 cores, 32 threads)
    Node v24.21.0 (npm 11.19.0)
    libsql 0.5.29 with @libsql/win32-x64-msvc 0.5.29, and 0.6.0-pre.43 (install note at the end)
    @libsql/client 0.18.0 (last table only)

    Smallest repro

    min.mjs:

    import Database from 'libsql';
    
    // 200 in-memory databases that are never closed; the process then exits normally.
    const dbs = [];
    for (let i = 0; i < 200; i++) dbs.push(new Database(':memory:'));

    A single run usually exits 0, so run it many times and count the exit codes: node count.mjs 1000 16 min.mjs (count.mjs is at the end). Crashed runs in two batches of 1000, 16 at a time (every other run exited 0):

    batch 1 batch 2
    libsql 0.5.29 133 / 1000 125 / 1000
    libsql 0.6.0-pre.43 176 / 1000 201 / 1000

    Variants

    repro.mjs opens N in-memory databases (12 by default), then either drops them and runs gc(), or keeps them until a natural exit. It writes marks to stderr synchronously, so a crashed run shows where it died: in the gc() loop (after opened, before collected), or after the 'exit' event, in Node's environment teardown.

    // libsql on Windows: in-memory databases that were never closed crash the process (0xC0000005, nothing on stderr)
    // when they are finalized, by the garbage collector or at exit. Run it many times; a single run usually exits 0.
    //   node --expose-gc repro.mjs                 12 databases, dropped and collected
    //   REPRO_KEEP=1 node repro.mjs                12 databases still referenced at a natural exit
    //   REPRO_CLOSE=1 node --expose-gc repro.mjs   each database closed right after it is opened
    //   REPRO_DBS=1 node --expose-gc repro.mjs     one database
    // REPRO_SQL=1 also creates a table, inserts a row and reads it back on each database.
    // REPRO_REALLY_EXIT=1 calls process.reallyExit(0) inside 'exit', which skips Node's environment teardown.
    import Database from 'libsql';
    import {writeSync} from 'node:fs';
    
    const env = process.env, n = +(env.REPRO_DBS ?? 12);
    const mark = (s) => writeSync(2, s + '\n'); // synchronous, so a crashed run shows how far it got
    
    let dbs = [];
    for (let i = 0; i < n; i++) {
      const db = new Database(':memory:');
      if (env.REPRO_SQL === '1') {
        db.exec("CREATE TABLE t (id INTEGER PRIMARY KEY, v TEXT); INSERT INTO t (v) VALUES ('x');");
        db.prepare('SELECT * FROM t').all();
      }
      if (env.REPRO_CLOSE === '1') db.close();
      else dbs.push(db);
    }
    mark('opened');
    if (env.REPRO_KEEP !== '1') {
      dbs = null;
      for (let i = 0; i < 4; i++) {
        globalThis.gc();
        await new Promise((r) => setImmediate(r));
      }
      mark('collected');
    }
    process.on('exit', () => {
      mark('exit-event');
      if (env.REPRO_REALLY_EXIT === '1') process.reallyExit(0);
    });

    Crashed runs, 16 processes at a time. Every failed run exited with 0xC0000005, and its stderr held only the marks; "gc" and "exit" say where it died. Rows without REPRO_KEEP=1 run as node --expose-gc repro.mjs.

    Settings 0.5.29 0.6.0-pre.43
    12 dbs, dropped + gc() (defaults) 20 / 3000 (16 gc, 4 exit) 22 / 3000 (17 gc, 5 exit)
    12 dbs, kept: REPRO_KEEP=1 0 / 3000 20 / 3000 (exit)
    12 dbs with SQL, dropped + gc(): REPRO_SQL=1 6 / 2000 (exit) 7 / 2000 (gc)
    12 dbs with SQL, kept: REPRO_SQL=1 REPRO_KEEP=1 0 / 2000 2 / 2000 (exit)
    2 dbs, dropped + gc(): REPRO_DBS=2 3 / 2000 (exit) not run
    50 dbs, dropped + gc(): REPRO_DBS=50 71 / 1000 (55 gc, 16 exit) not run
    200 dbs, dropped + gc(): REPRO_DBS=200 226 / 1000 (183 gc, 43 exit) 180 / 1000 (162 gc, 18 exit)
    200 dbs, kept: REPRO_DBS=200 REPRO_KEEP=1 153 / 1000 (exit) 122 / 1000 (exit)
    1 db, dropped + gc(): REPRO_DBS=1 0 / 2000 0 / 2000
    1 db, kept: REPRO_DBS=1 REPRO_KEEP=1 0 / 2000 not run
    12 dbs, each closed right after it is opened: REPRO_CLOSE=1 0 / 2000 not run
    12 dbs with SQL, each closed: REPRO_SQL=1 REPRO_CLOSE=1 0 / 2000 not run
    200 dbs, each closed: REPRO_DBS=200 REPRO_CLOSE=1 0 / 1000 0 / 1000
    200 dbs with SQL, each closed: REPRO_DBS=200 REPRO_SQL=1 REPRO_CLOSE=1 0 / 1000 0 / 1000
    200 dbs, kept, reallyExit in 'exit': REPRO_DBS=200 REPRO_KEEP=1 REPRO_REALLY_EXIT=1 0 / 1000 not run
    200 dbs, dropped + gc(), reallyExit in 'exit': REPRO_DBS=200 REPRO_REALLY_EXIT=1 181 / 1000 (all gc) not run

    More databases means more crashes, and a single database never crashed. Closing each database before it is dropped avoided the crash in every setting, with or without SQL. process.reallyExit(0) inside 'exit', which skips Node's environment teardown, removed the crashes at exit (153 to 0 of 1000 with 200 databases kept) but not the ones in gc() (181 of 1000 left), so it happens wherever the finalizers of unclosed databases run. One difference between the versions: 12 databases kept to a natural exit never crashed on 0.5.29 but did on 0.6.0-pre.43.

    Through @libsql/client 0.18.0

    12 createClient({url: ':memory:'}), each with a table, an insert and a select, left open:

    Settings crashed runs
    natural exit 7 / 1000 (exit)
    each client closed after use 0 / 1000
    process.reallyExit(0) inside 'exit' 0 / 1000

    In our own test suite (node:test, about 140 files with in-memory @libsql/client databases), the two files that failed most often went from 5 crashed runs in 480 to 0 once each test closed its databases when it ended. Closing all of a file's databases together at its end was not enough: 6 of 360 runs still crashed inside the last close(), which fits a double close that also runs from close(). Closing each database as soon as it is done with is the workaround that held for us.

    Ask

    Could the 0.6 line pick up libsql 0.10.0, and could tursodatabase/libsql#2261 also go into a 0.5.x release? @libsql/client 0.18.0 depends on libsql ^0.5.28, so 0.5.29 is what most users get.

    count.mjs
    // Runs a Node script many times, some at once, and counts the exit codes in hex (0xc0000005 is the access violation).
    // usage: node count.mjs <runs> <parallel> [node args...] <script>
    import {spawn} from 'node:child_process';
    
    const [runs, parallel, ...args] = process.argv.slice(2);
    const codes = {};
    let started = 0, done = 0;
    const next = () => {
      if (started++ >= +runs) return;
      spawn(process.execPath, args, {stdio: 'ignore'}).on('exit', (code, signal) => {
        const key = signal ?? '0x' + (code >>> 0).toString(16);
        codes[key] = (codes[key] ?? 0) + 1;
        if (++done === +runs) console.log(codes);
        else next();
      });
    };
    for (let i = 0; i < +parallel; i++) next();

    0.6.0-pre.43 on Windows: npm i libsql@0.6.0-pre.43 installs no native binary there, and require('libsql') throws Cannot find module 'libsql-win32-x64-msvc'. That optional dependency is not on npm; the binary is published as @libsql/win32-x64-msvc@0.6.0-pre.43. I installed it with npm i libsql-win32-x64-msvc@npm:@libsql/win32-x64-msvc@0.6.0-pre.43.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions