Repository navigation
Conversation
Collaborator
Author
|
Status: draft. It waits for the answer to the question in #754 (comment). How to reproduce
entry.mjs and a.mjs
export {};
import { readFileSync, writeFileSync } from "node:fs";
globalThis.runs = (globalThis.runs ?? 0) + 1;
if (globalThis.runs === 1) {
globalThis.fetches = 0;
Bun.plugin({
name: "hold the module's own load open until the reload",
setup(build) {
build.onLoad({ filter: /a\.mjs$/ }, () => {
const fetch = ++globalThis.fetches;
const loaded = { contents: "export const fetch = " + fetch + "; export const evaluation = (globalThis.evaluations = (globalThis.evaluations ?? 0) + 1);", loader: "js" };
if (globalThis.moduleMayLoad) return loaded;
const { promise, resolve } = Promise.withResolvers();
globalThis.moduleMayLoad = () => resolve(loaded);
writeFileSync(import.meta.path, readFileSync(import.meta.path));
return promise;
});
},
});
globalThis.inFlight = import("./a.mjs");
} else {
globalThis.moduleMayLoad();
try {
const show = m => "fetch " + m.fetch + ", evaluation " + m.evaluation;
console.log("in flight:", show(await globalThis.inFlight));
console.log("next:", show(await import("./a.mjs")));
process.exit(0);
} catch (error) {
console.log("rejected:", error);
process.exit(1);
}
} |
|
Preview build of e301a11: |
clearAll() takes every module out of the registry while loads are in flight. A load that has its entry goes on with that entry, and finishLoadingImportedModule() keeps its record out of the realm's [[LoadedModules]] (#748). A load by name (loadModule(name): import(), an entry point) has no entry until its fetch settles, so the clear cannot take it out. When the fetch settles, moduleLoadTopSettled calls provideFetch() by key and registers what was fetched before the clear in the registry of after it: const inFlight = import("./a.js"); // the fetch of a.js is pending loader.clearAll(); // the embedder reloads await inFlight; await import("./a.js"); // the module from before the clear In Bun that is a hot reload while an import() is fetching its module: the program keeps the old file until the next save. When the fetch fails instead, the error of the old file rejects the import() and is stored on the module that the key holds by then. The loader now has a registry epoch that clearAll() advances. A load by name keeps the epoch it started under (taken before fetch(), which can run script). When its fetch settles under another epoch, moduleLoadTopSettled does not use what the fetch settled with, a source or an error. It calls loadModule() by name again and the first load settles with the second. The second load is a load made after the clear in every respect: it joins the module the key holds or fetches, it has its own epoch, and a fetch of its own that fails is not kept on the key. What a load that has its entry at the clear does is not changed. A load that no clear overtook pays the copy of the epoch and one compare. ModuleLoadingContext keeps its size: the epoch is in the padding after m_step. Only loadModule(name) sets it, so the context of a static import costs what it did. hostLoadImportedModule() is not changed. registryEpoch() is public for an embedder that publishes a fetch by key in tables of its own (Bun's require cache). clearModuleRegistry() in the jsc shell calls clearAll(), for the new test. * JSTests/stress/module-load-by-name-in-flight-when-registry-cleared.js: Added. * JSTests/stress/resources/module-load-by-name-in-flight-when-registry-cleared/.gitignore: Added. * Source/JavaScriptCore/jsc.cpp: * Source/JavaScriptCore/runtime/JSMicrotask.cpp: * Source/JavaScriptCore/runtime/JSModuleLoader.cpp: * Source/JavaScriptCore/runtime/JSModuleLoader.h: * Source/JavaScriptCore/runtime/ModuleLoadingContext.cpp: * Source/JavaScriptCore/runtime/ModuleLoadingContext.h:
robobun
force-pushed
the
robobun/cfa20256/module-load-by-name-reload-after-clear
branch
from
October 21, 2026 19:39
add2afb to
e301a11
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft until the question in the notes (asked in #754) has an answer.
Problem
bun --hotreload during animport()can leave the old file registered until the next save.import()has no registry entry until its fetch settles, soclearAll()removes nothing of it.moduleLoadTopSettledthen registers the fetched source by key (JSMicrotask.cpp:1187). A failed fetch leaves its error on the module the key then holds.Fix
clearAll()advances a registry epoch. Animport()keeps the epoch it started under.moduleLoadTopSettleddrops the result and callsloadModule()by name again. Theimport()settles with that load, which is made after the clear.JSTests/stress/module-load-by-name-in-flight-when-registry-cleared.js(exit 3 on main's runtime, 0 here and with [JSC] A failed loadModule() stores its error in the registry entry it loaded #754 applied).Background
import(). Built (+233 lines): it breaks an import cycle that main loads.Downsides
import()fetches twice. If the second fetch fails, it rejects where main resolved with the old module.Notes
Question (also asked in #754)
Both candidate fixes stop the stale registration. They differ in what the overtaken
import()settles with. Rows from a Bun build (bun --hot, a pluginonLoadholds the fetch ofa.mjsacross the reload, the module text carries the number of its fetch):in flight: fetch 1,next: fetch 1in flight: fetch 1,next: fetch 2, 2 evaluationsin flight: fetch 2,next: fetch 2, 1 evaluationin flight: rejected,next: fetch 2in flight: fetch 2,next: fetch 2new: fetch 2,in flight: rejected,next: rejectednew: fetch 2,in flight: rejected,next: fetch 2fetch 2three timesimport()that joined the fetch of a static importfetch 1,import()fetch 1(evaluated again),next: fetch 1fetch 1,next: fetch 2fetch 1,import()fetch 2,next: fetch 2import()get: its module from before the clear (what [JSC] removeEntry() and clearAll() keep the realm's [[LoadedModules]] in step with the registry #748 does for a load that has its entry), or the result of a load started after the clear? And the same when its fetch failed?import()whose own fetch was in flight gets the new module. One whose dependency was in flight keeps its old graph ([JSC] removeEntry() and clearAll() keep the realm's [[LoadedModules]] in step with the registry #748,test/cli/hot/hot.test.ts:827in Bun). Is that split acceptable?"Keep its module" is built on the branch
robobun/cfa20256/module-load-by-name-detach-after-clear(+233 lines). Two cases that main gets right fail with it:in flight,nextand every laterimport()reject withReferenceError: Cannot access 'Base' before initialization.until the next save. The old root and the module that the cycle finds under the key are two instances.Bun.serve()in the module, the new module loaded first: the old body runs after the new one and its handler replaces the new handler.Both are as on main with this PR.
What this PR makes worse than main
import()then rejects. On main it resolved with the old module.import()still loads the old key.import()is a root for the new module. Its evaluation can start before the new entry point imports the module in its own order. For a fetch that ended with a module, main does the same with the old module. For a fetch that failed it is new.Other runs
import()gets the module of the third fetch, and the second is not evaluated. In the stress test, and in Bun (in flight: fetch 3,next: fetch 3, one evaluation).import()rejects, and nothing stays on the key. A later static import of it fetches and loads.Bun.ModuleGraph#dispose()with animport()in flight: bothimport()reject withERR_INVALID_STATEas on main. The graph fetches the module once more.import().Reach
bun --hot, animport()that the entry point does not await (a reload waits while the promise of the entry point is pending), and a save while the fetch of that module itself is in flight. The fetch is the read and the transpile of the file.Bun.ModuleGraph#dispose(), which callsclearAll().Not in this PR
m_resolutionFailures: a referrer from before the clear can still add a failure by key.removeEntry()(build.module(),mock.module()while animport()of the key is in flight).hmr-module.ts(bake: consolidate the open robobun dev-server, HMR runtime, production build and router fixes bun#39488).Cost
Release builds of Bun at ea6a2c7b7c, with WebKit 274ebb3 (main then) and with this PR's preview build
autobuild-preview-pr-853-e301a116. Sizes and instruction counts fromllvm-objdump -dofbun-profile:JSModuleLoader::loadModule()by namerunInternalMicrotask()(holdsmoduleLoadTopSettled)JSModuleLoader::loadModule()by referrerJSModuleLoader::hostLoadImportedModule()ModuleLoadingContext::create(), both overloadsbunloadModule()by name has 4: it loads the epoch beforefetch(), keeps it on the stack, and stores it into the context.moduleLoadTopSettledhas 4: two loads, a compare and a branch that is not taken. The other added instructions are the block that starts the load again.ModuleLoadingContextkeeps its size (thestatic_assertin its constructor). OnlyloadModule(name)writes the field, so the context of a static import is made by the same instructions as before.JSModuleLoaderkeeps its 136 bytes: the epoch is in padding.loadModule(name)hands to the embedder'sfetch()is not kept for the second load. Neither Bun's hook nor the shell's reads it.Bun half
The pin bump and the tests (
test/cli/hot/hot.test.ts) follow the answer. The rows above are from that branch. A Bun debug build with this PR's preview build passes them, and the hot, isolation, plugin, mock, resolve,node:moduleand module-graph suites.