From 627a3723f8f42dc40f95a7a0e3b4f210f13ae942 Mon Sep 17 00:00:00 2001 From: robobun <117481402+robobun@users.noreply.github.com> Date: Sat, 31 Oct 2026 16:49:15 +0000 Subject: [PATCH 1/3] [JSC] $vm.callWithTerminationAtTrapCheck(fn, n): take a termination at the n-th trap check A termination begins at a trap check. Native code has one at each RETURN_IF_EXCEPTION, so a termination can end an operation between any two of its steps. $vm.callWithTimeLimit() reaches only the steps that are slow enough for a timer. $vm.callWithTerminationAtTrapCheck(fn, n) calls fn, and the n-th trap check of the call requests a termination and takes it. The call withdraws that termination when it ends (VM::cancelTermination()), as an embedder does for a time limit, and returns whether the count ended. A test walks n from 1 until the call is not cut, and so stops the operation at each of its exception checks in turn. Before the n-th check no termination is pending, as when a timer thread requests one at that moment. Code that looks for a termination by itself (Atomics.wait(), VM::hasPendingTermination()) sees none. To bring each trap check to VMTraps::handleTraps(), the call keeps the trap bit of the watchdog set. handleTraps() counts the check and does not pass the bit on to the watchdog, so the call is refused in a VM that has a watchdog. A check does not count where a termination cannot begin (DeferTermination), where another termination is requested and not taken yet, or where one unwinds. So the termination of the call is not taken for the termination of a time limit, and the call withdraws only its own: a time limit around the call ends it, and a time limit inside fn ends what it limits while the count goes on. $vm.callWithTimeLimit() requests the termination of an enclosing count again where it withdrew it with its own. The count is side data of the VM (VM::ensureSideData()). VMTraps and VM keep their layout. VMTraps::handleTraps() looks for the count only while a call is in progress in the process (one counter). The places of the trap checks are fixed for native code and for the interpreter. The optimizing JITs poll less or not at all, so a walk runs with --useJIT=0. A garbage collection drops what the VM compiled, so the loads behind it have more trap checks and a walk goes back some places. Tests: JSTests/stress/vm-termination-at-trap-check.js, JSTests/stress/vm-termination-at-trap-check-with-watchdog.js --- JSTests/BUN-TEST-DIFFERENCES.md | 1 + ...termination-at-trap-check-with-watchdog.js | 18 + .../stress/vm-termination-at-trap-check.js | 324 ++++++++++++++++++ Source/JavaScriptCore/runtime/VMTraps.cpp | 68 ++++ Source/JavaScriptCore/runtime/VMTraps.h | 17 + Source/JavaScriptCore/tools/JSDollarVM.cpp | 77 +++++ 6 files changed, 505 insertions(+) create mode 100644 JSTests/stress/vm-termination-at-trap-check-with-watchdog.js create mode 100644 JSTests/stress/vm-termination-at-trap-check.js diff --git a/JSTests/BUN-TEST-DIFFERENCES.md b/JSTests/BUN-TEST-DIFFERENCES.md index e45f9b6a61329..094c21619ee79 100644 --- a/JSTests/BUN-TEST-DIFFERENCES.md +++ b/JSTests/BUN-TEST-DIFFERENCES.md @@ -236,6 +236,7 @@ Not upstream's: a sync neither brings nor changes them. Most are named `stress/b | `stress/bun-return-address-with-wrong-signature-is-not-returned-to.js` | with `ENABLE(RETURN_ADDRESS_SIGNING)` a function does not return once a bit of the signature of its saved return address was flipped, in each of the four tiers: the run must crash. Where nothing is signed the test crashes by itself | | the fr-CA block of `stress/intl-date-pattern-includes-literal-text.js` (the file is upstream's) | keeps the file's coverage on CLDR 49, where fr's pattern no longer has literal text: an `h` inside literal text is not mistaken for the hour field | | the last block of `stress/regexp-test-sticky-first-char-filter.js` (the file is upstream's; nothing of upstream's text changes) | the first-character filter of a sticky `test()` comes from the first sets of the alternatives (`yarr/YarrFirstFollowSets.*`): behind a leading lookbehind, forward reference or lookahead, and in groups nested more than eight deep, a byte that passes the filter still reaches the matcher | +| `stress/vm-termination-at-trap-check.js` and `-with-watchdog.js` | `$vm.callWithTerminationAtTrapCheck(fn, n)` requests a termination at the n-th trap check of a call and withdraws it when the call ends: each larger n lets the call get at least as far, native code is stopped too, no termination is pending before that check (`Atomics.wait()` goes to its timeout), each trap check counts once, none counts while terminations are deferred, a time limit around the call or inside its function still ends what it limits, and nothing is left pending. A VM with a watchdog and script that runs with terminations deferred are refused. The test hook for what a termination leaves behind in the middle of an operation | `jsc` cannot compile a program ahead of time by itself (the compiler's input comes from the front end, in Bun's repository), so the programs that test ahead-of-time compilation are in Bun's `test/aot`. diff --git a/JSTests/stress/vm-termination-at-trap-check-with-watchdog.js b/JSTests/stress/vm-termination-at-trap-check-with-watchdog.js new file mode 100644 index 0000000000000..4c3b3684ff994 --- /dev/null +++ b/JSTests/stress/vm-termination-at-trap-check-with-watchdog.js @@ -0,0 +1,18 @@ +//@ runDefault("--useDollarVM=1", "--useJIT=0", "--watchdog=600000") + +// $vm.callWithTerminationAtTrapCheck() counts trap checks with the trap bit of the watchdog. A VM that has a watchdog +// refuses the call: the watchdog would get none of its checks while the count runs. + +let error; +let started = false; +try { + $vm.callWithTerminationAtTrapCheck(() => { started = true; }, 1); +} catch (e) { + error = e; +} +if (!(error instanceof TypeError) || error.message != "$vm.callWithTerminationAtTrapCheck() needs a VM without a watchdog") + throw new Error(`expected the TypeError of the refusal, got ${error}`); +if (started) + throw new Error("the function started"); +if ($vm.hasPendingTermination()) + throw new Error("something is still pending"); diff --git a/JSTests/stress/vm-termination-at-trap-check.js b/JSTests/stress/vm-termination-at-trap-check.js new file mode 100644 index 0000000000000..3bb7f57020de4 --- /dev/null +++ b/JSTests/stress/vm-termination-at-trap-check.js @@ -0,0 +1,324 @@ +//@ runDefault("--useDollarVM=1", "--useJIT=0") + +// $vm.callWithTerminationAtTrapCheck(fn, n): the n-th trap check of a call requests a termination and takes it. The +// termination is withdrawn when the call ends (VM::cancelTermination(), as an embedder does for node:vm's `timeout`). +// Native code polls traps at every RETURN_IF_EXCEPTION. The interpreter polls them where a function is called and +// where a loop goes around. So a loop over n stops a call at each of its trap checks in turn. +// Without the JIT: optimized code polls less, or not at all, so there the n-th check is not a fixed place. + +function shouldBe(actual, expected, what) { + if (actual !== expected) + throw new Error(`${what}: expected ${expected}, got ${actual}`); +} + +function shouldThrow(fn, constructor, what) { + try { + fn(); + } catch (error) { + if (error instanceof constructor) + return; + throw new Error(`${what}: threw ${error}`); + } + throw new Error(`${what}: did not throw`); +} + +function timesOut(fn, ms) { + try { + $vm.callWithTimeLimit(fn, ms); + } catch (error) { + if (!(error instanceof RangeError) || error.message != "timed out") + throw error; + return true; + } + return false; +} + +function nothingLeftBehind(what) { + shouldBe($vm.hasPendingTermination(), false, `${what}: something is still pending`); + let sum = 0; + for (let i = 0; i < 100; ++i) + sum += i; + shouldBe(sum, 4950, `${what}: the code after the call`); +} + +// The first trap check is before the function starts. +{ + let started = false; + shouldBe($vm.callWithTerminationAtTrapCheck(() => { started = true; }, 1), true, "n = 1"); + shouldBe(started, false, "n = 1: the function started"); + nothingLeftBehind("n = 1"); +} + +// A function has more trap checks while its inline caches and profiles are empty, so a walk runs it a few times first. +// After that the n-th check of a call is the same place, until a garbage collection empties a cache again. +function walk(reset, fn, check) { + for (let i = 0; i < 10; ++i) { + reset(); + fn(); + } + for (let n = 1; ; ++n) { + if (n > 1000) + throw new Error("the walk has no end"); + reset(); + const wasTaken = $vm.callWithTerminationAtTrapCheck(fn, n); + nothingLeftBehind(`n = ${n}`); + check(n, wasTaken); + if (!wasTaken) + return; + } +} + +// Each larger n lets the function get at least as far, and the function returns for the first n that it does not reach. +{ + let steps = 0; + let done = false; + function step() { + ++steps; + } + function work() { + step(); + step(); + step(); + done = true; + } + + const stepsOfCutCalls = new Set; + let furthest = 0; + walk(() => { + steps = 0; + done = false; + }, work, (n, wasTaken) => { + if (steps < furthest) + throw new Error(`n = ${n}: ${steps} steps, and ${furthest} steps with n - 1`); + furthest = steps; + shouldBe(done, !wasTaken, `n = ${n}: the function went to its end`); + if (wasTaken) + stepsOfCutCalls.add(steps); + else + shouldBe(steps, 3, `n = ${n}: the steps of a call that was not cut`); + }); + // Each call of step() has a trap check of its own, so the walk stopped the function before each of them. + for (let count = 0; count < 3; ++count) + shouldBe(stepsOfCutCalls.has(count), true, `a call that was cut after ${count} steps`); +} + +// Each trap check counts once. One turn of a loop is one trap check in the interpreter, so n + 1 stops a loop one turn +// later than n. +{ + let turns; + const turnsOfCalls = []; + for (let n = 20; n < 40; ++n) { + turns = 0; + shouldBe($vm.callWithTerminationAtTrapCheck(() => { for (;;) ++turns; }, n), true, `an endless loop, n = ${n}`); + turnsOfCalls.push(turns); + } + for (let i = 1; i < turnsOfCalls.length; ++i) + shouldBe(turnsOfCalls[i], turnsOfCalls[i - 1] + 1, `the turns of an endless loop for n = ${20 + i}, with ${turnsOfCalls[i - 1]} for n - 1`); +} + +// Native code has trap checks too: some n stops the call inside Object.defineProperties(). +{ + let before; + let after; + let insideNativeCode = 0; + walk(() => { + before = after = false; + }, () => { + before = true; + Object.defineProperties({ }, { a: { value: 1 }, b: { value: 2 }, c: { value: 3 } }); + after = true; + }, (n, wasTaken) => { + if (wasTaken && before && !after) + ++insideNativeCode; + }); + if (!insideNativeCode) + throw new Error("no trap check stopped the call inside native code"); +} + +// Before the n-th check no termination is pending, as when a timer thread requests one at that moment. The function +// sees none, and Atomics.wait() goes to its timeout. +{ + const cell = new Int32Array(new SharedArrayBuffer(4)); + let pending; + let result; + let cutBehindTheWait = 0; + function step() { + } + walk(() => { + pending = undefined; + result = undefined; + }, () => { + pending = $vm.hasPendingTermination(); + result = Atomics.wait(cell, 0, 0, 1); + step(); + }, (n, wasTaken) => { + if (pending !== undefined) + shouldBe(pending, false, `n = ${n}: a termination is pending inside the function`); + // A termination that is pending ends the wait at once, and then the wait has no result. + if (wasTaken && result !== undefined) + ++cutBehindTheWait; + }); + if (!cutBehindTheWait) + throw new Error("no trap check stopped the call behind its wait"); +} + +// Script cannot catch the termination. +{ + let caught = false; + let finalized = false; + const wasTaken = $vm.callWithTerminationAtTrapCheck(() => { + try { + for (;;) { } + } catch { + caught = true; + } finally { + finalized = true; + } + }, 10); + shouldBe(wasTaken, true, "an endless loop"); + shouldBe(caught, false, "an endless loop: catch"); + shouldBe(finalized, false, "an endless loop: finally"); + nothingLeftBehind("an endless loop"); +} + +// A count that the call does not reach ends with the call. The next call has a count of its own. +{ + shouldBe($vm.callWithTerminationAtTrapCheck(() => 1, 0x7fffffff), false, "a count that is not reached"); + nothingLeftBehind("a count that is not reached"); + shouldBe($vm.callWithTerminationAtTrapCheck(() => { }, 1), true, "a call after a count that was not reached"); + nothingLeftBehind("a call after a count that was not reached"); +} + +// The call withdraws its own termination only. A time limit around the call ends it, in a loop and in a wait. +{ + const cell = new Int32Array(new SharedArrayBuffer(4)); + for (const fn of [() => { for (;;) { } }, () => { Atomics.wait(cell, 0, 0); }]) { + let returned = false; + shouldBe(timesOut(() => { + $vm.callWithTerminationAtTrapCheck(fn, 0x7fffffff); + returned = true; + }, 5), true, "a time limit around a call that does not reach its count"); + shouldBe(returned, false, "a call that a time limit cut returned"); + nothingLeftBehind("a time limit around a call"); + } +} + +// A time limit around a loop of calls ends the loop, wherever the limit passes. Where it passes while a call withdraws +// its own termination, the call requests the termination of the limit again. A loop whose limit was lost ends by the +// clock, and is reported. +for (let round = 0; round < 10; ++round) { + const end = preciseTime() + 2; + shouldBe(timesOut(() => { + while (preciseTime() < end) + $vm.callWithTerminationAtTrapCheck(() => { }, 1); + }, 1), true, `a time limit around a loop of calls, round ${round}`); + nothingLeftBehind("a time limit around a loop of calls"); +} + +// The other way round: the count ends inside a time limit in the function, and that limit passes before the +// termination has left it. The time limit requests the termination of the count again, so the function does not go on. +{ + function spin(turns) { + for (let i = 0; i < turns; ++i) { } + } + for (let n = 1; n <= 120; ++n) { + let wentOn = false; + const end = preciseTime() + 2; + shouldBe($vm.callWithTerminationAtTrapCheck(() => { + while (preciseTime() < end) + timesOut(() => { spin(40); }, 0); + wentOn = true; + }, n), true, `a call around a loop of time limits, n = ${n}`); + shouldBe(wentOn, false, `n = ${n}: the function went on behind the n-th trap check`); + } + nothingLeftBehind("a call around a loop of time limits"); +} + +// A time limit inside the function ends what it limits, and the count goes on behind it. +{ + const cell = new Int32Array(new SharedArrayBuffer(4)); + let limited = false; + let turns = 0; + const wasTaken = $vm.callWithTerminationAtTrapCheck(() => { + limited = timesOut(() => { Atomics.wait(cell, 0, 0); }, 5); + // Only the count ends this loop. + for (;;) + ++turns; + }, 1000); + shouldBe(limited, true, "a time limit inside the function"); + shouldBe(wasTaken, true, "a call whose function used a time limit"); + if (!turns || turns >= 1000) + throw new Error(`the count went on for ${turns} turns of the loop behind the time limit`); + nothingLeftBehind("a time limit inside the function"); +} + +// A call inside the function of another call is refused: the VM has one count. +{ + let error; + const wasTaken = $vm.callWithTerminationAtTrapCheck(() => { + try { + $vm.callWithTerminationAtTrapCheck(() => { }, 1); + } catch (e) { + error = e; + } + }, 1000); + shouldBe(wasTaken, false, "the outer call of two"); + shouldBe(error instanceof TypeError, true, "the inner call of two threw a TypeError"); + nothingLeftBehind("a call inside a call"); +} + +// No trap check counts while terminations are deferred. The setter of an index of a sparse array is script that runs +// so: it never sees a termination, and it runs to its end once it has started. The call is refused there. +{ + const array = []; + array[1000000] = 1; + let pendingInside; + let started; + let finished; + let refusal; + function step() { + } + Object.defineProperty(array, 5, { + set() { + started = true; + pendingInside ||= $vm.hasPendingTermination(); + step(); + step(); + step(); + pendingInside ||= $vm.hasPendingTermination(); + if (refusal === undefined) { + try { + refusal = $vm.callWithTerminationAtTrapCheck(() => { }, 1000); + } catch (error) { + refusal = error; + } + } + finished = true; + }, + configurable: true, + }); + walk(() => { + pendingInside = started = finished = false; + }, () => { + array[5] = 2; + }, (n, wasTaken) => { + shouldBe(pendingInside, false, `n = ${n}: a termination is pending inside script that runs with terminations deferred`); + shouldBe(finished, started, `n = ${n}: script that runs with terminations deferred went to its end`); + }); + if (!(refusal instanceof TypeError)) + throw new Error(`a call while terminations are deferred: expected a TypeError, got ${refusal}. If this setter no longer runs with terminations deferred, use another place that does.`); + shouldBe(refusal.message, "$vm.callWithTerminationAtTrapCheck() cannot be called while terminations are deferred", "a call while terminations are deferred"); + nothingLeftBehind("script that runs with terminations deferred"); +} + +// What the function throws by itself stays thrown. +shouldThrow(() => $vm.callWithTerminationAtTrapCheck(() => { throw new SyntaxError("mine"); }, 1000), SyntaxError, "a function that throws"); +nothingLeftBehind("a function that throws"); + +shouldThrow(() => $vm.callWithTerminationAtTrapCheck(1, 1), TypeError, "not a function"); +shouldThrow(() => $vm.callWithTerminationAtTrapCheck(() => { }, 0), RangeError, "n = 0"); +shouldThrow(() => $vm.callWithTerminationAtTrapCheck(() => { }, 1.5), RangeError, "n = 1.5"); +shouldThrow(() => $vm.callWithTerminationAtTrapCheck(() => { }, 2 ** 31), RangeError, "n = 2 ** 31"); +shouldThrow(() => $vm.callWithTerminationAtTrapCheck(() => { }, NaN), RangeError, "n = NaN"); +shouldThrow(() => $vm.callWithTerminationAtTrapCheck(() => { }), RangeError, "no n"); +nothingLeftBehind("calls that throw"); diff --git a/Source/JavaScriptCore/runtime/VMTraps.cpp b/Source/JavaScriptCore/runtime/VMTraps.cpp index 3606b095c67a8..9caa7ec0d0ec9 100644 --- a/Source/JavaScriptCore/runtime/VMTraps.cpp +++ b/Source/JavaScriptCore/runtime/VMTraps.cpp @@ -36,6 +36,7 @@ #include "LLIntPCRanges.h" #include "MachineContext.h" #include "MacroAssemblerCodeRef.h" +#include "SideDataRepository.h" #include "VMEntryScopeInlines.h" #include "VMInlines.h" #include "VMManager.h" @@ -443,6 +444,59 @@ CONCURRENT_SAFE void VMTraps::updateThreadStopRequestIfNeeded() cancelThreadStopIfNeeded(); } +namespace { +// $vm.callWithTerminationAtTrapCheck(): see VMTraps::beginTerminationAtTrapCheck(). +class TerminationAtTrapCheckData final : public SideDataRepository::SideData { + WTF_DEPRECATED_MAKE_FAST_ALLOCATED(TerminationAtTrapCheckData); +public: + unsigned trapChecksLeft { 0 }; + VMTraps::TerminationAtTrapCheck state { VMTraps::TerminationAtTrapCheck::None }; +}; +char terminationAtTrapCheckKey; +// The number of counts that run, in all VMs. handleTraps() looks for the side data of its VM only when there is one. +std::atomic terminationsAtTrapCheckInProgress; +} + +static TerminationAtTrapCheckData& terminationAtTrapCheckData(VM& vm) +{ + return vm.ensureSideData(&terminationAtTrapCheckKey, [] { + return makeUnique(); + }); +} + +void VMTraps::beginTerminationAtTrapCheck(unsigned count) +{ + VM& vm = this->vm(); + auto& data = terminationAtTrapCheckData(vm); + ASSERT(count); + ASSERT(!vm.watchdog()); + ASSERT(data.state == TerminationAtTrapCheck::None); + data.trapChecksLeft = count; + data.state = TerminationAtTrapCheck::Counting; + ++terminationsAtTrapCheckInProgress; + // The bit of the watchdog brings each trap check to handleTraps(), which counts it and does not pass it on. + fireTrap(NeedWatchdogCheck); +} + +bool VMTraps::endTerminationAtTrapCheck() +{ + auto& data = terminationAtTrapCheckData(vm()); + ASSERT(data.state != TerminationAtTrapCheck::None); + bool wasRequested = data.state == TerminationAtTrapCheck::Requested; + data.trapChecksLeft = 0; + data.state = TerminationAtTrapCheck::None; + --terminationsAtTrapCheckInProgress; + clearTrap(NeedWatchdogCheck); + return wasRequested; +} + +auto VMTraps::terminationAtTrapCheck() -> TerminationAtTrapCheck +{ + if (!terminationsAtTrapCheckInProgress.load(std::memory_order_relaxed)) + return TerminationAtTrapCheck::None; + return terminationAtTrapCheckData(vm()).state; +} + bool VMTraps::handleTraps(VMTraps::BitField mask) { VM& vm = this->vm(); @@ -453,6 +507,20 @@ bool VMTraps::handleTraps(VMTraps::BitField mask) if (m_trapsDeferred) RELEASE_AND_RETURN(scope, false); // We'll service them on the next opportunity after deferring has stopped. + if (terminationsAtTrapCheckInProgress.load(std::memory_order_relaxed) && hasTrapBit(NeedWatchdogCheck, mask)) [[unlikely]] { + // $vm.callWithTerminationAtTrapCheck(): see beginTerminationAtTrapCheck(). The trap check that ends the count + // requests the termination, as another thread does. Before that, no termination is pending. + if (auto& data = terminationAtTrapCheckData(vm); data.state == TerminationAtTrapCheck::Counting) { + ASSERT(!vm.watchdog()); + mask &= ~NeedWatchdogCheck; + bool canBegin = !isDeferringTermination() && !hasTrapBit(NeedTermination) && !vm.hasPendingTerminationException(); + if (canBegin && !--data.trapChecksLeft) { + data.state = TerminationAtTrapCheck::Requested; + fireTrap(NeedTermination); + clearTrap(NeedWatchdogCheck); + } + } + } if (isDeferringTermination()) mask &= ~NeedTermination; diff --git a/Source/JavaScriptCore/runtime/VMTraps.h b/Source/JavaScriptCore/runtime/VMTraps.h index 3e6c7aa55d6d4..8dca9c36a0d58 100644 --- a/Source/JavaScriptCore/runtime/VMTraps.h +++ b/Source/JavaScriptCore/runtime/VMTraps.h @@ -216,6 +216,23 @@ class VMTraps { }; bool isDeferringTermination() const { return m_deferTerminationCount; } + // For tests ($vm.callWithTerminationAtTrapCheck()). While a count runs, each handling of traps at which a + // termination can begin counts down, and the one that reaches zero requests a termination and takes it. So a test + // can begin a termination at every exception check of an operation in turn, where a time limit reaches the few + // that are slow. A handling does not count while traps or terminations are deferred, while another termination + // is requested and not taken yet, or while one unwinds: the termination of the count is then not taken for that + // one. + // The count is side data of the VM (VM::ensureSideData()). While it runs it keeps the trap bit of the watchdog + // set, so the VM must have no watchdog. + enum class TerminationAtTrapCheck : uint8_t { + None, // No count runs. + Counting, + Requested, // The count has ended and has requested the termination. endTerminationAtTrapCheck() is still to come. + }; + JS_EXPORT_PRIVATE void beginTerminationAtTrapCheck(unsigned count); + // Returns whether the count has ended. + JS_EXPORT_PRIVATE bool endTerminationAtTrapCheck(); + JS_EXPORT_PRIVATE TerminationAtTrapCheck terminationAtTrapCheck(); inline void deferTermination(DeferAction); inline void undoDeferTermination(DeferAction); diff --git a/Source/JavaScriptCore/tools/JSDollarVM.cpp b/Source/JavaScriptCore/tools/JSDollarVM.cpp index a5a466f56bda6..12a8b03604ff9 100644 --- a/Source/JavaScriptCore/tools/JSDollarVM.cpp +++ b/Source/JavaScriptCore/tools/JSDollarVM.cpp @@ -2195,6 +2195,7 @@ static JSC_DECLARE_HOST_FUNCTION(functionTriggerMemoryPressure); static JSC_DECLARE_HOST_FUNCTION(functionGC); static JSC_DECLARE_HOST_FUNCTION(functionCallWithTimeLimit); static JSC_DECLARE_HOST_FUNCTION(functionCancelTermination); +static JSC_DECLARE_HOST_FUNCTION(functionCallWithTerminationAtTrapCheck); static JSC_DECLARE_HOST_FUNCTION(functionHasPendingTermination); static JSC_DECLARE_HOST_FUNCTION(functionEdenGC); static JSC_DECLARE_HOST_FUNCTION(functionGCSweepAsynchronously); @@ -2824,11 +2825,86 @@ JSC_DEFINE_HOST_FUNCTION(functionCallWithTimeLimit, (JSGlobalObject* globalObjec break; } } + // So did an enclosing $vm.callWithTerminationAtTrapCheck() whose count has ended. + if (vm.traps().terminationAtTrapCheck() == VMTraps::TerminationAtTrapCheck::Requested) + vm.notifyNeedTermination(); if (!wasCutShort) RELEASE_AND_RETURN(scope, JSValue::encode(result)); return throwVMRangeError(globalObject, scope, "timed out"_s); } +// Calls `fn`, requests a termination at the n-th trap check from here (n >= 1), and withdraws that termination when +// the call ends, as an embedder does with VM::cancelTermination(). Returns true if the n-th check was reached, and +// false if `fn` returned first. An error that leaves `fn` is thrown, also behind the n-th check (the error of a time +// limit inside `fn`), so a throw does not tell whether the count ended. A loop over n walks a termination through +// every exception check of what `fn` does: native code polls traps at each RETURN_IF_EXCEPTION, and the interpreter +// polls them where a function is called and where a loop goes around. +// The n-th check is a fixed place only for native code and the interpreter, and only while `fn` does the same work. +// Code from the optimizing JITs polls less or not at all, so such a loop runs with --useJIT=0. A function polls more +// while its inline caches are empty, so the loop calls `fn` a few times first. The VM keeps what it compiled by the +// text of the source, and a garbage collection drops that: the next calls that load the same text have more checks, +// and the loop goes back some places. A loop that needs each place loads text that no call before it has loaded. +// Compiled (AOT) code counts one poll up to three times. +// Before the n-th check no termination is pending, as when a timer thread requests one at that moment: +// VM::hasPendingTermination() is false, and Atomics.wait() is not ended (a wait without a timeout does not return). +// A check does not count where a termination cannot begin (DeferTermination), where another termination is requested +// and not taken yet, or where one unwinds. One that native code has cleared does not stop the count. +// The call withdraws its own termination only. A time limit ($vm.callWithTimeLimit()) around the call ends it, and +// one inside `fn` ends what it limits: the count goes on behind it. Another kind of limit inside `fn` (node:vm's +// `timeout`) can withdraw the termination of the call where both unwind together. +// The call is refused inside another call of this function, inside a DeferTermination scope, and in a VM that has a +// watchdog: the count uses the trap bit of the watchdog. +// Usage: $vm.callWithTerminationAtTrapCheck(fn, n) +JSC_DEFINE_HOST_FUNCTION(functionCallWithTerminationAtTrapCheck, (JSGlobalObject* globalObject, CallFrame* callFrame)) +{ + DollarVMAssertScope assertScope; + VM& vm = globalObject->vm(); + auto scope = DECLARE_THROW_SCOPE(vm); + JSValue function = callFrame->argument(0); + double count = callFrame->argument(1).toNumber(globalObject); + RETURN_IF_EXCEPTION(scope, { }); + auto callData = JSC::getCallData(function); + if (callData.type == CallData::Type::None) + return throwVMTypeError(globalObject, scope, "expected a function"_s); + if (!(count >= 1) || count > std::numeric_limits::max() || count != std::trunc(count)) + return throwVMRangeError(globalObject, scope, "expected an integer count from 1 to 2147483647"_s); + // The VM has one count. + if (vm.traps().terminationAtTrapCheck() != VMTraps::TerminationAtTrapCheck::None) + return throwVMTypeError(globalObject, scope, "a call of $vm.callWithTerminationAtTrapCheck() is in progress"_s); + // While the count runs, the watchdog would get none of its checks. + if (vm.watchdog()) + return throwVMTypeError(globalObject, scope, "$vm.callWithTerminationAtTrapCheck() needs a VM without a watchdog"_s); + // No trap check counts there, and VM::cancelTermination() cannot be called there. + if (vm.traps().isDeferringTermination()) + return throwVMTypeError(globalObject, scope, "$vm.callWithTerminationAtTrapCheck() cannot be called while terminations are deferred"_s); + // A termination that is pending already, in whatever form, is taken here, before the count starts. + vm.throwTerminationExceptionIfNeeded(); + RETURN_IF_EXCEPTION(scope, { }); + + vm.ensureTerminationException(); + vm.traps().beginTerminationAtTrapCheck(static_cast(count)); + call(globalObject, function, callData, jsUndefined(), ArgList()); + if (!vm.traps().endTerminationAtTrapCheck()) { + // The termination of this call was not requested. What `fn` threw stays thrown, and so does a termination + // that something else requested. + RETURN_IF_EXCEPTION(scope, { }); + return JSValue::encode(jsBoolean(false)); + } + // Withdraw the termination of this call, and whatever it has merged with since it began. + vm.cancelTermination(); + // An enclosing time-limited call whose limit has passed made a request of its own; make it again. + auto& calls = vm.ensureSideData(&timeLimitedCallsKey, [] { return makeUnique(); }); + for (auto& enclosing : calls.active) { + if (enclosing->didFire()) { + vm.notifyNeedTermination(); + break; + } + } + // This takes the request that was made again. It also returns with an error that left `fn` behind the n-th check. + RETURN_IF_EXCEPTION(scope, { }); + return JSValue::encode(jsBoolean(true)); +} + // VM::cancelTermination() / VM::hasPendingTermination(). // Usage: $vm.cancelTermination(); $vm.hasPendingTermination() JSC_DEFINE_HOST_FUNCTION(functionCancelTermination, (JSGlobalObject* globalObject, CallFrame*)) @@ -6108,6 +6184,7 @@ void JSDollarVM::finishCreation(VM& vm) addFunction(vm, allowIfNotFuzz, "crash"_s, functionCrash, 0); addFunction(vm, allowIfNotFuzz, "callWithTimeLimit"_s, functionCallWithTimeLimit, 2); addFunction(vm, allowIfNotFuzz, "cancelTermination"_s, functionCancelTermination, 0); + addFunction(vm, allowIfNotFuzz, "callWithTerminationAtTrapCheck"_s, functionCallWithTerminationAtTrapCheck, 2); addFunction(vm, allowIfNotFuzz, "hasPendingTermination"_s, functionHasPendingTermination, 0); addFunction(vm, allowIfNotFuzz, "breakpoint"_s, functionBreakpoint, 0); addFunction(vm, allowIfNotFuzz, "exit"_s, functionExit, 0); From d9fe4226e281458e615be670f7c18449b0ceae3d Mon Sep 17 00:00:00 2001 From: robobun <117481402+robobun@users.noreply.github.com> Date: Sun, 1 Nov 2026 11:29:25 +0000 Subject: [PATCH 2/3] [JSC] $vm.moduleLoaderLoadSync({ loader }, specifier): the synchronous module load, from the shell JSModuleLoader::loadModuleSync() loads, links and evaluates a module graph before it returns. Bun calls it for require() of an ES module. The jsc shell had no caller, so no JSTests test ran the synchronous queue of the loader (VM::SynchronousModuleQueueScope, drainSynchronousModuleQueue()). $vm.moduleLoaderLoadSync() calls it for a loader from $vm.createModuleLoader(), next to $vm.moduleLoaderImport(). It returns the namespace object, throws what the load failed with, and throws a TypeError for a load that did not finish (a top-level await). Test: JSTests/stress/module-loader-load-sync.js --- JSTests/BUN-TEST-DIFFERENCES.md | 1 + JSTests/stress/module-loader-load-sync.js | 59 +++++++++++++++++++ .../resources/module-loader-load-sync/dep.js | 2 + .../resources/module-loader-load-sync/root.js | 3 + .../module-loader-load-sync/throws.js | 2 + .../module-loader-load-sync/waits.js | 2 + Source/JavaScriptCore/tools/JSDollarVM.cpp | 45 ++++++++++++++ 7 files changed, 114 insertions(+) create mode 100644 JSTests/stress/module-loader-load-sync.js create mode 100644 JSTests/stress/resources/module-loader-load-sync/dep.js create mode 100644 JSTests/stress/resources/module-loader-load-sync/root.js create mode 100644 JSTests/stress/resources/module-loader-load-sync/throws.js create mode 100644 JSTests/stress/resources/module-loader-load-sync/waits.js diff --git a/JSTests/BUN-TEST-DIFFERENCES.md b/JSTests/BUN-TEST-DIFFERENCES.md index 094c21619ee79..8a4d8f36d1d69 100644 --- a/JSTests/BUN-TEST-DIFFERENCES.md +++ b/JSTests/BUN-TEST-DIFFERENCES.md @@ -237,6 +237,7 @@ Not upstream's: a sync neither brings nor changes them. Most are named `stress/b | the fr-CA block of `stress/intl-date-pattern-includes-literal-text.js` (the file is upstream's) | keeps the file's coverage on CLDR 49, where fr's pattern no longer has literal text: an `h` inside literal text is not mistaken for the hour field | | the last block of `stress/regexp-test-sticky-first-char-filter.js` (the file is upstream's; nothing of upstream's text changes) | the first-character filter of a sticky `test()` comes from the first sets of the alternatives (`yarr/YarrFirstFollowSets.*`): behind a leading lookbehind, forward reference or lookahead, and in groups nested more than eight deep, a byte that passes the filter still reaches the matcher | | `stress/vm-termination-at-trap-check.js` and `-with-watchdog.js` | `$vm.callWithTerminationAtTrapCheck(fn, n)` requests a termination at the n-th trap check of a call and withdraws it when the call ends: each larger n lets the call get at least as far, native code is stopped too, no termination is pending before that check (`Atomics.wait()` goes to its timeout), each trap check counts once, none counts while terminations are deferred, a time limit around the call or inside its function still ends what it limits, and nothing is left pending. A VM with a watchdog and script that runs with terminations deferred are refused. The test hook for what a termination leaves behind in the middle of an operation | +| `stress/module-loader-load-sync.js` | `$vm.moduleLoaderLoadSync({ loader }, specifier)` is `JSModuleLoader::loadModuleSync()`, the load of `require()` of an ES module in Bun: the graph has run when the call returns, a load that `import()` began is taken over, what the module throws is thrown, and a top-level await is a `TypeError` | `jsc` cannot compile a program ahead of time by itself (the compiler's input comes from the front end, in Bun's repository), so the programs that test ahead-of-time compilation are in Bun's `test/aot`. diff --git a/JSTests/stress/module-loader-load-sync.js b/JSTests/stress/module-loader-load-sync.js new file mode 100644 index 0000000000000..cd10b6def5352 --- /dev/null +++ b/JSTests/stress/module-loader-load-sync.js @@ -0,0 +1,59 @@ +//@ runDefault("--useDollarVM=1") + +// $vm.moduleLoaderLoadSync({ loader }, specifier): JSModuleLoader::loadModuleSync(), the load that `require()` of an +// ES module makes in Bun. The graph is loaded, linked and evaluated when the call returns. + +function shouldBe(actual, expected, what) { + if (actual !== expected) + throw new Error(`${what}: expected ${expected}, got ${actual}`); +} + +function shouldThrow(fn, constructor, message, what) { + try { + fn(); + } catch (error) { + if (error instanceof constructor && error.message == message) + return; + throw new Error(`${what}: threw ${error}`); + } + throw new Error(`${what}: did not throw`); +} + +const directory = "./resources/module-loader-load-sync/"; +globalThis.loadSyncLog = []; + +// The bodies have run, in the order of the graph, when the call returns. +const loader = $vm.createModuleLoader(); +const namespace = $vm.moduleLoaderLoadSync(loader, directory + "root.js"); +shouldBe(loadSyncLog.join(), "dep,root", "the bodies that ran before the call returned"); +shouldBe(namespace.value, 41, "the namespace object"); + +// A second load, synchronous or not, answers with the module that is there. +shouldBe($vm.moduleLoaderLoadSync(loader, directory + "root.js"), namespace, "the second synchronous load"); +shouldBe($vm.moduleLoaderLoadSync(loader, directory + "dep.js").dep, 40, "a synchronous load of the dependency"); +let imported; +$vm.moduleLoaderImport(loader, directory + "root.js").then(result => { imported = result; }); +drainMicrotasks(); +shouldBe(imported, namespace, "import() after the synchronous load"); +shouldBe(loadSyncLog.join(), "dep,root", "the bodies ran once"); + +// A module that import() has begun to load is taken over: the call does not wait for the jobs of that load. +{ + const loader = $vm.createModuleLoader(); + let imported; + $vm.moduleLoaderImport(loader, directory + "root.js").then(result => { imported = result; }); + const namespace = $vm.moduleLoaderLoadSync(loader, directory + "root.js"); + shouldBe(namespace.value, 41, "a synchronous load of a module that import() is loading"); + shouldBe(imported, undefined, "the import() has not been answered yet"); + drainMicrotasks(); + shouldBe(imported, namespace, "the import() that was first"); + shouldBe(loadSyncLog.join(), "dep,root,dep,root", "the bodies ran once for the second loader"); +} + +// What the module throws is thrown by the call. +shouldThrow(() => $vm.moduleLoaderLoadSync($vm.createModuleLoader(), directory + "throws.js"), RangeError, "from the module", "a module that throws"); + +// A module with a top-level await does not finish inside the call. +shouldThrow(() => $vm.moduleLoaderLoadSync($vm.createModuleLoader(), directory + "waits.js"), TypeError, "the load of the module did not finish", "a module that waits"); + +shouldThrow(() => $vm.moduleLoaderLoadSync({ }, directory + "root.js"), TypeError, "expected the result of $vm.createModuleLoader()", "not a loader"); diff --git a/JSTests/stress/resources/module-loader-load-sync/dep.js b/JSTests/stress/resources/module-loader-load-sync/dep.js new file mode 100644 index 0000000000000..4a9ff7b51d4d0 --- /dev/null +++ b/JSTests/stress/resources/module-loader-load-sync/dep.js @@ -0,0 +1,2 @@ +globalThis.loadSyncLog.push("dep"); +export const dep = 40; diff --git a/JSTests/stress/resources/module-loader-load-sync/root.js b/JSTests/stress/resources/module-loader-load-sync/root.js new file mode 100644 index 0000000000000..e3033f7edfd11 --- /dev/null +++ b/JSTests/stress/resources/module-loader-load-sync/root.js @@ -0,0 +1,3 @@ +import { dep } from "./dep.js"; +globalThis.loadSyncLog.push("root"); +export const value = dep + 1; diff --git a/JSTests/stress/resources/module-loader-load-sync/throws.js b/JSTests/stress/resources/module-loader-load-sync/throws.js new file mode 100644 index 0000000000000..07dc09256a8e3 --- /dev/null +++ b/JSTests/stress/resources/module-loader-load-sync/throws.js @@ -0,0 +1,2 @@ +import "./dep.js"; +throw new RangeError("from the module"); diff --git a/JSTests/stress/resources/module-loader-load-sync/waits.js b/JSTests/stress/resources/module-loader-load-sync/waits.js new file mode 100644 index 0000000000000..1f12543496f90 --- /dev/null +++ b/JSTests/stress/resources/module-loader-load-sync/waits.js @@ -0,0 +1,2 @@ +await 1; +export const late = 1; diff --git a/Source/JavaScriptCore/tools/JSDollarVM.cpp b/Source/JavaScriptCore/tools/JSDollarVM.cpp index 12a8b03604ff9..fc8f753006fd7 100644 --- a/Source/JavaScriptCore/tools/JSDollarVM.cpp +++ b/Source/JavaScriptCore/tools/JSDollarVM.cpp @@ -67,6 +67,7 @@ #include "JSString.h" #include "LinkBuffer.h" #include "MarkedSpaceInlines.h" +#include "ModuleRegistryEntry.h" #include "NativeCallee.h" #include "ObjectConstructor.h" #include "ObjectPropertyCondition.h" @@ -2291,6 +2292,9 @@ static JSC_DECLARE_HOST_FUNCTION(functionEvacuateAuxiliaryBlocks); static JSC_DECLARE_HOST_FUNCTION(functionGlobalObjectCount); static JSC_DECLARE_HOST_FUNCTION(functionCreateModuleLoader); static JSC_DECLARE_HOST_FUNCTION(functionModuleLoaderImport); +#if USE(BUN_JSC_ADDITIONS) +static JSC_DECLARE_HOST_FUNCTION(functionModuleLoaderLoadSync); +#endif static JSC_DECLARE_HOST_FUNCTION(functionGlobalObjectForObject); static JSC_DECLARE_HOST_FUNCTION(functionGetGetterSetter); static JSC_DECLARE_HOST_FUNCTION(functionLoadGetterFromGetterSetter); @@ -4495,6 +4499,44 @@ JSC_DEFINE_HOST_FUNCTION(functionModuleLoaderImport, (JSGlobalObject* globalObje RELEASE_AND_RETURN(scope, JSValue::encode(loader->importModule(globalObject, specifier, jsUndefined(), callFrame->callerSourceOrigin(vm), false))); } +#if USE(BUN_JSC_ADDITIONS) +// $vm.moduleLoaderLoadSync({ loader }, specifier): loads, links and evaluates the module through that loader before it +// returns, relative to the caller if that is a file, as `require()` of an ES module does in Bun (JSModuleLoader::loadModuleSync()). +// Returns the namespace object. Throws what the load failed with, and a TypeError for a load that did not finish. +JSC_DEFINE_HOST_FUNCTION(functionModuleLoaderLoadSync, (JSGlobalObject* globalObject, CallFrame* callFrame)) +{ + DollarVMAssertScope assertScope; + VM& vm = globalObject->vm(); + auto scope = DECLARE_THROW_SCOPE(vm); + JSModuleLoader* loader = moduleLoaderFromHolder(vm, callFrame->argument(0)); + if (!loader) + return throwVMTypeError(globalObject, scope, "expected the result of $vm.createModuleLoader()"_s); + JSString* specifier = callFrame->argument(1).toString(globalObject); + RETURN_IF_EXCEPTION(scope, {}); + // A caller that is not a file has no directory: the host resolves from where it resolves a load with no referrer. + SourceOrigin origin = callFrame->callerSourceOrigin(vm); + JSValue referrer = origin.url().protocolIsFile() ? JSValue(jsString(vm, origin.url().string())) : jsUndefined(); + Identifier key = loader->resolve(globalObject, specifier, referrer, nullptr, /* useImportMap */ true); + RETURN_IF_EXCEPTION(scope, {}); + JSPromise* promise = loader->loadModuleSync(globalObject, key, nullptr, nullptr); + RETURN_IF_EXCEPTION(scope, {}); + promise->markAsHandled(); + switch (promise->status()) { + case JSPromise::Status::Pending: + return throwVMTypeError(globalObject, scope, "the load of the module did not finish"_s); + case JSPromise::Status::Rejected: + throwException(globalObject, scope, promise->result()); + return {}; + case JSPromise::Status::Fulfilled: + break; + } + ModuleRegistryEntry* entry = loader->registryEntry(key); + if (!entry || !entry->record()) + return throwVMTypeError(globalObject, scope, "the module has no record"_s); + RELEASE_AND_RETURN(scope, JSValue::encode(entry->record()->getModuleNamespace(globalObject))); +} +#endif + JSC_DEFINE_HOST_FUNCTION(functionGlobalObjectForObject, (JSGlobalObject*, CallFrame* callFrame)) { DollarVMAssertScope assertScope; @@ -6323,6 +6365,9 @@ void JSDollarVM::finishCreation(VM& vm) addFunction(vm, allowIfNotFuzz, "globalObjectCount"_s, functionGlobalObjectCount, 0); addFunction(vm, allowIfNotFuzz, "createModuleLoader"_s, functionCreateModuleLoader, 2); addFunction(vm, allowIfNotFuzz, "moduleLoaderImport"_s, functionModuleLoaderImport, 2); +#if USE(BUN_JSC_ADDITIONS) + addFunction(vm, allowIfNotFuzz, "moduleLoaderLoadSync"_s, functionModuleLoaderLoadSync, 2); +#endif addFunction(vm, allowIfNotFuzz, "globalObjectForObject"_s, functionGlobalObjectForObject, 1); addFunction(vm, allowIfNotFuzz, "getGetterSetter"_s, functionGetGetterSetter, 2); From 3e04c127b66b0940ca98c808e1ee0afc4c0214b5 Mon Sep 17 00:00:00 2001 From: robobun <117481402+robobun@users.noreply.github.com> Date: Mon, 2 Nov 2026 04:31:17 +0000 Subject: [PATCH 3/3] [JSC] jsc shell: setModuleLoaderFetchHook(fn), script inside the fetch step of a module load A host can run script in its fetch hook (Bun's plugins do), and that script can start a load of its own. The shell read a file and nothing else, so no JSTests test had a load inside the fetch step of another. setModuleLoaderFetchHook(fn) makes the shell call fn(key) before it reads the file of a JavaScript module. A string that fn returns is the text of the module, and no file is read. What fn throws is thrown by the fetch hook of the loader. setModuleLoaderFetchHook() takes the function away. An object that fn returns makes a module without text: a deferred synthetic module (SyntheticSourceProvider::createDeferred()). Its exports are the own enumerable properties of the object, and the shell reads them when the graph that imports the module is about to be linked. No JSTests test had such a module. Test: JSTests/stress/shell-module-loader-fetch-hook.js --- JSTests/BUN-TEST-DIFFERENCES.md | 1 + .../stress/shell-module-loader-fetch-hook.js | 125 ++++++++++++++++++ Source/JavaScriptCore/jsc.cpp | 105 +++++++++++++++ 3 files changed, 231 insertions(+) create mode 100644 JSTests/stress/shell-module-loader-fetch-hook.js diff --git a/JSTests/BUN-TEST-DIFFERENCES.md b/JSTests/BUN-TEST-DIFFERENCES.md index 8a4d8f36d1d69..d6377d6c29064 100644 --- a/JSTests/BUN-TEST-DIFFERENCES.md +++ b/JSTests/BUN-TEST-DIFFERENCES.md @@ -238,6 +238,7 @@ Not upstream's: a sync neither brings nor changes them. Most are named `stress/b | the last block of `stress/regexp-test-sticky-first-char-filter.js` (the file is upstream's; nothing of upstream's text changes) | the first-character filter of a sticky `test()` comes from the first sets of the alternatives (`yarr/YarrFirstFollowSets.*`): behind a leading lookbehind, forward reference or lookahead, and in groups nested more than eight deep, a byte that passes the filter still reaches the matcher | | `stress/vm-termination-at-trap-check.js` and `-with-watchdog.js` | `$vm.callWithTerminationAtTrapCheck(fn, n)` requests a termination at the n-th trap check of a call and withdraws it when the call ends: each larger n lets the call get at least as far, native code is stopped too, no termination is pending before that check (`Atomics.wait()` goes to its timeout), each trap check counts once, none counts while terminations are deferred, a time limit around the call or inside its function still ends what it limits, and nothing is left pending. A VM with a watchdog and script that runs with terminations deferred are refused. The test hook for what a termination leaves behind in the middle of an operation | | `stress/module-loader-load-sync.js` | `$vm.moduleLoaderLoadSync({ loader }, specifier)` is `JSModuleLoader::loadModuleSync()`, the load of `require()` of an ES module in Bun: the graph has run when the call returns, a load that `import()` began is taken over, what the module throws is thrown, and a top-level await is a `TypeError` | +| `stress/shell-module-loader-fetch-hook.js` | `setModuleLoaderFetchHook(fn)` of the shell: `fn(key)` runs before the file of a module is read, a string that it returns is the text of the module, an object makes a module without text whose exports are read when the graph is about to be linked (a deferred synthetic module, `SyntheticSourceProvider::createDeferred()`), the hook is asked for each fetch of a JavaScript module, and what it throws is what the load fails with. Tests use it for modules without files and for script inside the fetch step of a load | `jsc` cannot compile a program ahead of time by itself (the compiler's input comes from the front end, in Bun's repository), so the programs that test ahead-of-time compilation are in Bun's `test/aot`. diff --git a/JSTests/stress/shell-module-loader-fetch-hook.js b/JSTests/stress/shell-module-loader-fetch-hook.js new file mode 100644 index 0000000000000..ea8aabc51359c --- /dev/null +++ b/JSTests/stress/shell-module-loader-fetch-hook.js @@ -0,0 +1,125 @@ +//@ runDefault("--useDollarVM=1") + +// setModuleLoaderFetchHook(fn): the shell calls fn(key) before it reads the file of a module. A string is the text of +// the module. An object makes a module without text, whose exports are the properties of the object. What fn throws is +// thrown by the fetch hook of the loader. So a test can make modules without files, and can run script inside the +// fetch step of a load. + +function shouldBe(actual, expected, what) { + if (actual !== expected) + throw new Error(`${what}: expected ${expected}, got ${actual}`); +} + +function importThrough(loader, path) { + const load = { state: "pending", value: undefined }; + $vm.moduleLoaderImport(loader, path).then(namespace => { + load.value = namespace; + load.state = "fulfilled"; + }, error => { + load.value = error; + load.state = "rejected"; + }); + drainMicrotasks(); + return load; +} + +const made = "./shell-module-loader-fetch-hook/"; +const files = "./resources/module-loader-load-sync/"; +globalThis.loadSyncLog = []; +const keys = []; +const failure = new RangeError("from the hook"); +let inner; +const syncLoader = $vm.createModuleLoader(); +setModuleLoaderFetchHook(key => { + keys.push(key.slice(key.lastIndexOf("/") + 1)); + if (key.endsWith("/made-root.js")) + return `import { value } from "./made-dep.js"; export default value + 1;`; + if (key.endsWith("/made-dep.js")) + return `export const value = 40;`; + if (key.endsWith("/throws.js")) + throw failure; + if (key.endsWith("/imports-object.js")) + return `import { answer, late } from "./object.js"; import { more } from "./more.js"; export default answer + late + more;`; + if (key.endsWith("/more.js")) + return `export const more = 0;`; + if (key.endsWith("/sync-root.js")) + return `import "./sync-begins-import.js"; import { value } from "./sync-dep.js"; export default value;`; + if (key.endsWith("/sync-begins-import.js")) { + // An import() of a module that the synchronous load has asked for already. + $vm.moduleLoaderImport(syncLoader, made + "sync-dep.js").then(() => { }, () => { }); + return `export { };`; + } + if (key.endsWith("/sync-dep.js")) + return `export const value = "made";`; + if (key.endsWith("/object.js")) { + return { + answer: 41, + get late() { + keys.push("late is read"); + return 1; + }, + }; + } + if (key.endsWith("/loads.js")) { + // A load of its own, inside the fetch step of this one. + inner = $vm.moduleLoaderLoadSync($vm.createModuleLoader(), made + "made-root.js").default; + return `export default ${inner} + 1;`; + } +}); + +// Modules that the hook makes. There are no such files. +let load = importThrough($vm.createModuleLoader(), made + "made-root.js"); +shouldBe(load.state, "fulfilled", "a module that the hook makes"); +shouldBe(load.value.default, 41, "the value of a module that the hook makes"); +shouldBe(keys.join(), "made-root.js,made-dep.js", "the keys that the hook was asked for"); + +// Where the hook returns neither a string nor an object, the file is read. +load = importThrough($vm.createModuleLoader(), files + "dep.js"); +shouldBe(load.state, "fulfilled", "a module of a file"); +shouldBe(load.value.dep, 40, "the value of a module of a file"); + +// What the hook throws is what the load fails with. +load = importThrough($vm.createModuleLoader(), made + "throws.js"); +shouldBe(load.state, "rejected", "a module whose fetch throws"); +shouldBe(load.value, failure, "what the load of that module fails with"); + +// An object makes a module without text. The shell reads the exports when the graph is about to be linked, which is +// behind the fetch of each module of the graph. +keys.length = 0; +load = importThrough($vm.createModuleLoader(), made + "imports-object.js"); +shouldBe(load.state, "fulfilled", "a module that imports a module without text"); +shouldBe(load.value.default, 42, "the value of a module that imports a module without text"); +shouldBe(keys.join(), "imports-object.js,object.js,more.js,late is read", "the exports of a module without text are read once, behind each fetch"); +load = importThrough($vm.createModuleLoader(), made + "object.js"); +shouldBe(load.state, "fulfilled", "a module without text"); +shouldBe(load.value.answer, 41, "an export of a module without text"); + +// The hook runs script: here a synchronous load through another loader, where the build has that function. +if ($vm.moduleLoaderLoadSync) { + load = importThrough($vm.createModuleLoader(), made + "loads.js"); + shouldBe(load.state, "fulfilled", "a module whose fetch loads another"); + shouldBe(inner, 41, "the load inside the hook"); + shouldBe(load.value.default, 42, "the value of a module whose fetch loads another"); +} + +// The hook is asked for each fetch of a JavaScript module, also where the loader passes the type of the module along: +// the import() inside this synchronous load is such a fetch. +if ($vm.moduleLoaderLoadSync) { + keys.length = 0; + shouldBe($vm.moduleLoaderLoadSync(syncLoader, made + "sync-root.js").default, "made", "a synchronous load with an import() inside its fetch step"); + shouldBe(keys.filter(key => key == "sync-dep.js").length >= 1, true, "the hook was asked for the module of the import()"); + drainMicrotasks(); +} + +// Without a function the shell reads files again. +setModuleLoaderFetchHook(); +load = importThrough($vm.createModuleLoader(), made + "made-root.js"); +shouldBe(load.state, "rejected", "a module that no hook makes"); + +let threw = false; +try { + setModuleLoaderFetchHook(1); +} catch (error) { + threw = error instanceof TypeError; +} +shouldBe(threw, true, "a hook that is not a function"); diff --git a/Source/JavaScriptCore/jsc.cpp b/Source/JavaScriptCore/jsc.cpp index 0485b802a78ab..fd22162726bcc 100644 --- a/Source/JavaScriptCore/jsc.cpp +++ b/Source/JavaScriptCore/jsc.cpp @@ -517,6 +517,9 @@ static JSC_DECLARE_HOST_FUNCTION(functionLoad); static JSC_DECLARE_HOST_FUNCTION(functionLoadString); static JSC_DECLARE_HOST_FUNCTION(functionReadFile); static JSC_DECLARE_HOST_FUNCTION(functionWriteFile); +#if USE(BUN_JSC_ADDITIONS) +static JSC_DECLARE_HOST_FUNCTION(functionSetModuleLoaderFetchHook); +#endif static JSC_DECLARE_HOST_FUNCTION(functionCheckSyntax); static JSC_DECLARE_HOST_FUNCTION(functionOpenFile); static JSC_DECLARE_HOST_FUNCTION(functionReadline); @@ -887,6 +890,9 @@ class GlobalObject final : public JSGlobalObject { addFunction(vm, "readFile"_s, functionReadFile, 2); addFunction(vm, "read"_s, functionReadFile, 2); addFunction(vm, "writeFile"_s, functionWriteFile, 2); +#if USE(BUN_JSC_ADDITIONS) + addFunction(vm, "setModuleLoaderFetchHook"_s, functionSetModuleLoaderFetchHook, 1); +#endif addFunction(vm, "write"_s, functionWriteFile, 2); addFunction(vm, "checkSyntax"_s, functionCheckSyntax, 1); addFunction(vm, "sleepSeconds"_s, functionSleepSeconds, 1); @@ -1677,6 +1683,28 @@ static inline SourceCode jscSource(const Vector& utf8, const SourceOrigin& sourc return jscSource(str, sourceOrigin, filename); } +#if USE(BUN_JSC_ADDITIONS) +// setModuleLoaderFetchHook(fn): see functionSetModuleLoaderFetchHook(). +class ModuleLoaderFetchHook final : public SideDataRepository::SideData { + WTF_DEPRECATED_MAKE_FAST_ALLOCATED(ModuleLoaderFetchHook); +public: + Strong function; + // The objects that the function returned for modules without text, each under the number of its fetch. + HashMap> exportsObjects; + uint64_t exportsObjectCount { 0 }; +}; +static char moduleLoaderFetchHookKey; +// Whether a VM of the process has been given a function. A fetch looks for the function of its VM only then. +static std::atomic moduleLoaderFetchHookWasSet; + +static ModuleLoaderFetchHook& moduleLoaderFetchHook(VM& vm) +{ + return vm.ensureSideData(&moduleLoaderFetchHookKey, [] { + return makeUnique(); + }); +} +#endif + template static bool fetchModuleFromLocalFileSystem(const URL& fileURL, Vector& buffer) { @@ -1744,6 +1772,56 @@ JSPromise* GlobalObject::moduleLoaderFetch(JSGlobalObject* globalObject, JSModul URL moduleURL({ }, moduleKey); ASSERT(moduleURL.protocolIsFile()); + +#if USE(BUN_JSC_ADDITIONS) + // Bun: setModuleLoaderFetchHook(). What the function throws is thrown by this hook, as a host's fetch hook may. + if (moduleLoaderFetchHookWasSet.load(std::memory_order_relaxed) && (!attributes || attributes->type() == ScriptFetchParameters::Type::JavaScript)) [[unlikely]] { + if (JSObject* hook = moduleLoaderFetchHook(vm).function.get()) { + MarkedArgumentBuffer arguments; + arguments.append(key); + JSValue text = call(globalObject, hook, arguments, "The fetch hook of the module loader is not a function"_s); + RETURN_IF_EXCEPTION(scope, nullptr); + if (text.isString()) { + String source = asString(text)->value(globalObject); + RETURN_IF_EXCEPTION(scope, nullptr); + auto sourceCode = JSSourceCode::create(vm, jscSource(source, SourceOrigin { moduleURL }, moduleURL.fileSystemPath(), TextPosition(), SourceProviderSourceType::Module)); + scope.release(); + promise->resolve(globalObject, vm, sourceCode); + return promise; + } + if (text.isObject()) { + // A module without text, whose exports are read when the graph that imports it is about to be linked + // (SyntheticSourceProvider::createDeferred(), AbstractModuleRecord::generateDeferredSyntheticModules()). + auto& fetchHook = moduleLoaderFetchHook(vm); + uint64_t exportsObject = ++fetchHook.exportsObjectCount; + fetchHook.exportsObjects.set(exportsObject, Strong { vm, asObject(text) }); + Ref provider = SyntheticSourceProvider::createDeferred([exportsObject](JSGlobalObject* globalObject, Identifier, Vector& exportNames, MarkedArgumentBuffer& exportValues) { + VM& vm = globalObject->vm(); + auto scope = DECLARE_THROW_SCOPE(vm); + JSObject* exports = moduleLoaderFetchHook(vm).exportsObjects.get(exportsObject).get(); + if (!exports) { + throwTypeError(globalObject, scope, "The fetch hook of the module loader was taken away"_s); + return; + } + PropertyNameArrayBuilder names(vm, PropertyNameMode::Strings, PrivateSymbolMode::Exclude); + exports->methodTable()->getOwnPropertyNames(exports, globalObject, names, DontEnumPropertiesMode::Exclude); + RETURN_IF_EXCEPTION(scope, void()); + for (auto& name : names) { + JSValue value = exports->get(globalObject, name); + RETURN_IF_EXCEPTION(scope, void()); + exportNames.append(name); + exportValues.append(value); + } + }, SourceOrigin { moduleURL }, moduleKey); + auto sourceCode = JSSourceCode::create(vm, SourceCode { WTF::move(provider) }); + scope.release(); + promise->resolve(globalObject, vm, sourceCode); + return promise; + } + } + } +#endif + // Strip the URI from our key so Errors print canonical system paths. moduleKey = moduleURL.fileSystemPath(); @@ -2754,6 +2832,33 @@ JSC_DEFINE_HOST_FUNCTION(functionReadFile, (JSGlobalObject* globalObject, CallFr WTF_ALLOW_UNSAFE_BUFFER_USAGE_BEGIN +#if USE(BUN_JSC_ADDITIONS) +// Bun: setModuleLoaderFetchHook(fn). Before the shell reads the file of a JavaScript module, it calls fn(key). If fn +// returns a string, that is the text of the module and no file is read. If fn returns an object, the module has no +// text: its exports are the own enumerable properties of the object, which the shell reads when the graph that imports +// the module is about to be linked (a deferred synthetic module: SyntheticSourceProvider::createDeferred()). +// Otherwise the file is read. What fn throws is thrown by the fetch hook of the loader. A test uses it to make modules +// without files, and to run script inside the fetch step of a load. Script that loads through the loader of that +// load, or that runs its jobs, can finish another request of the module that imports: JSModuleLoader::innerModuleLoading() +// has assertions against that. setModuleLoaderFetchHook() takes the function away, and the objects with it. +JSC_DEFINE_HOST_FUNCTION(functionSetModuleLoaderFetchHook, (JSGlobalObject* globalObject, CallFrame* callFrame)) +{ + VM& vm = globalObject->vm(); + auto scope = DECLARE_THROW_SCOPE(vm); + JSValue function = callFrame->argument(0); + if (function.isUndefined()) { + moduleLoaderFetchHook(vm).function.clear(); + moduleLoaderFetchHook(vm).exportsObjects.clear(); + return JSValue::encode(jsUndefined()); + } + if (!function.isCallable()) + return throwVMTypeError(globalObject, scope, "expected a function"_s); + moduleLoaderFetchHook(vm).function.set(vm, asObject(function)); + moduleLoaderFetchHookWasSet.store(true, std::memory_order_relaxed); + return JSValue::encode(jsUndefined()); +} +#endif + JSC_DEFINE_HOST_FUNCTION(functionWriteFile, (JSGlobalObject* globalObject, CallFrame* callFrame)) { VM& vm = globalObject->vm();