Skip to content

[JSC] Run a CommonJS module's generator where its importing graph evaluates it - #891

Draft
robobun wants to merge 2 commits into
mainfrom
robobun/21c0c737/commonjs-evaluates-at-slot
Draft

robobun wants to merge 2 commits into
mainfrom
robobun/21c0c737/commonjs-evaluates-at-slot

Conversation

@robobun

@robobun robobun commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Draft, not for merge as it is. It waits for a maintainer decision.

Problem

Fix

  • SyntheticModuleRecord::evaluate() runs the generator, where InnerModuleEvaluation reaches the record. The pre-link pass is removed.
  • Until then the record hasPendingExports() and ResolveExport answers Pending. Importers bind such imports in JSModuleRecord::execute, before the module body.
  • Verified: with the paired Bun branch, 9 of 9 import shapes run in Node's order (main: 0 of 9).

Background

Downsides

Notes

Known defects. All reproduced on a release build of Bun with this branch and the paired Bun branch robobun/21c0c737/cjs-evaluates-at-import-slot. "Bun" is a build of main.

input Node v26.3.0 and Bun this PR
a barrel with export * from "./consts.mjs" and export * from "./lib.cjs"; a module in an import cycle with the barrel reads VERSION of consts.mjs 1.0 undefined
the same cycle with class Child extends Base, Base from an ES module behind the barrel works TypeError: The superclass is not a constructor.
the namespace of that barrel, read in the cycle its names, not frozen no keys and frozen. Later its names, not frozen
a lazy builtin export (promises of node:fs) behind such a barrel an object ReferenceError: Cannot access 'promises' before initialization. on every call
import() that a CommonJS body reaches through require() of an ES module settles never settles. #871 is the fix for this route
a name that no module exports, imported through a barrel cycle SyntaxError, exit 1 no error, exit 0
a graph that is imported again after its CommonJS module was evicted and now throws, then a call of a function from the first import TypeError abort: shared code reads a scope offset that the failed record no longer reserves
a CommonJS module that requires back the ES module that imports it Node: ERR_REQUIRE_CYCLE_MODULE. Bun: no error ReferenceError when it reads a binding
the new JSTests with --collectContinuously abort: the shell hook holds exports weakly until the record owns it

Paired Bun branch, known gaps. Also reproduced on the release build.

  • A CommonJS module that deletes its own require.cache entry while it runs is run again for every later importer (3 runs where Node and Bun have 1).
  • An import of a CommonJS module that threw for an earlier require() rejects with Failed to evaluate module, not with the error of the module.
  • mock.module() of an ES module that re-exports a CommonJS module that has not run yet does nothing. After a failed import of a CommonJS module, mock.module() of it does nothing.
  • A plugin that a CommonJS body registers no longer applies to the static imports of its own graph (Bun applies it in 4 of 5 runs, a race). Node has the same order.
  • module.id "." and the order of require.cache keys still follow the order in which the fetches settle.

Rework that these point to (not built).

  • Star rule: a resolved binding wins over a pending one for now, and the importer checks for ambiguity when it binds its pending imports.
  • Namespace rule: the local exports of a module are on its namespace from the start. Only names that come through export * of a pending record arrive later.
  • One binder: the slot fill and the environment lookup bind through CyclicModuleRecord::linkImportEntry, so a lazy export is not skipped.
  • A failed generator keeps its offset reservations.
  • The record owns its owner cell from the JSSourceCode that makeModule holds, so nothing can collect it between the fetch and the record.
  • Pending imports are bound in innerModuleEvaluation after the requested modules, and again when a cycle completes.

Measurements. Two release builds of Bun (no LTO, identical flags) on this branch and on its parent commit. Instruction counts are from PTRACE_SINGLESTEP, or from the hardware counter read at breakpoints, on the main thread.

what parent this PR
import edge to a builtin synthetic record at evaluate (innerModuleEvaluation for node:fs) 89 instructions 91
AbstractModuleRecord::link of a builtin synthetic record, per import edge 120 53
link() prologue on a 4-module ES graph (entry of AbstractModuleRecord::link to entry of CyclicModuleRecord::link) 1,130 instructions, 1 allocation 22, 0
pre-link walk on bench/module-loader import.mjs (501 ES modules) 5,716,227 instructions removed
link() of that graph 185,952,403 180,127,267
top-level innerModuleEvaluation of that graph 77,842,821 77,867,397
resolveExportImpl, 100 named imports through two levels of export * over 50 modules 100 calls, 37,848 to 45,694 instructions each 100 calls, 38,107 to 45,969
JSModuleRecord::execute of an empty ES module 2,875 2,875
ES module with 1 import from a pending record: link(), then its execute 152,4xx and 9,980 156,3xx and 11,198
ES module with 10 named imports from a pending record 213,2xx and 27,262 218,3xx and 32,281
allocations in link() of such a module 156 165: 2 new (PendingImports and its vector), 7 first uses of a size class that the generator paid for earlier on the parent
8 more loaders import a 301-function ES module whose synthetic dependency has another export layout in each loader +8 ModuleProgramExecutable, +2,408 FunctionExecutable, +2,408 FunctionCodeBlock +0, +0, +0
sizeof: JSModuleRecord, CyclicModuleRecord, SyntheticModuleRecord, AbstractModuleRecord, JSModuleNamespaceObject 384, 304, 312, 288, 72 384, 312, 312, 288, 72
text of the Bun binary 101,412,495 101,418,799

CyclicModuleRecord gains one pointer. JSModuleRecord keeps its size because m_features moves into the tail padding of CyclicModuleRecord. WebAssemblyModuleRecord keeps its cell size (336). SyntheticSourceProvider grows from 208 to 216 bytes (the owner callback).

Tiers. In an import cycle, 2,000,000 reads before the record has evaluated give undefined and 2,000,000 reads after give the value, with the default tiers, useJIT=0, useDFGJIT=0, useFTLJIT=0, and with thresholds that force the FTL.

What the new tests cover. The shell makes a deferred synthetic record for a module named *.deferred.js: the file is the body of function (exports) { ... }, and the record exports the properties that exports has after the body ran.

scenario result
import "./setup.js"; import "./lib.deferred.js" the body runs after setup.js, once
default, named, namespace, export *, export { x } from, export * as ns bind to what the body exported
a name that exists only because of what an earlier module did binds
a hoisted function of the importer, called before the record is evaluated (import cycle), 10,000 times reads undefined, then the values
a barrel with export * of the record, in a cycle with a module that imports from the barrel undefined, then the value
a name the record does not export the importer rejects with a SyntaxError after the body ran. Other importers are not affected
the body throws every import of it rejects with the same error. The body runs once
export * of the record and of an ES module that export the same name SyntaxError (ambiguous)
a graph that does not link the body does not run. The next graph that imports the record evaluates it
a module with a top-level await ahead of it the body runs after the synchronous part of that module
four loaders import the same ES module, and x is at another place among the exports of each loader's record the records share linked code and read their own x

Shared code. Records for one module in several loaders share the ModuleProgramExecutable when every import resolves to the same place (JSModuleRecord::resolvesImportsLike). For an exporter that is not a source text module that place is a scope offset, which did not exist before the exporter had run. A record that shares code now asks the pending exporter for the offset that the code embeds, and the exporter puts the name there when it builds its environment. A reservation that cannot be kept refuses the sharing, as a different offset did before.

Early code. JSScope::abstractResolve links an import that still waits as ModuleVar with a reserved offset, so every CodeBlock of one UnlinkedCodeBlock keeps the same resolve type. JSModuleRecord::fillImportSlot gives such code an environment that holds undefined at that offset, and does not fill the import slot, until the exporter has its environment.

@robobun
robobun force-pushed the robobun/21c0c737/commonjs-evaluates-at-slot branch 2 times, most recently from a95e1e5 to 44aebcd Compare October 23, 2026 20:13
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Preview build of ad787db: autobuild-preview-pr-891-ad787dba

… strand or start its importers early

ECMA-262 has no job that runs while a module graph is evaluated. A host can
run one from a module body: a microtask checkpoint, or a synchronous
require() of an ES module. That job can finish a module with a top-level
await while the modules that wait for it are in the middle of
InnerModuleEvaluation or of AsyncModuleExecutionFulfilled.

Inside InnerModuleEvaluation, GatherAvailableAncestors skipped an importer
that was still EVALUATING and left its count of pending dependencies as it
was. The importer never ran, and the promise of its Evaluate() never
settled. A rejection gave a module on the stack the status EVALUATED with
no cycle root, and the walk then read that root.

Inside AsyncModuleExecutionFulfilled, GatherAvailableAncestors counts a
module without a top-level await as done when it appends the module to
execList, before the body of that module runs. A job that the body ran
could finish the last other dependency of an ancestor. The ancestor then
ran before that body ended, also when the body threw afterwards.

A count of pending dependencies now goes down when a dependency has
finished. asyncExecutionFulfilled() gathers the modules that the finished
module was the last pending dependency of, runs them in the order of their
[[AsyncEvaluationOrder]], and gathers again for a module without a
top-level await when its body has run. A module gets its order after each
module that it waits for, so this is the order of the specification's
sorted execList.

An importer on the stack of the walk gets its count decremented. The walk
reads the count at step 12 of InnerModuleEvaluation. An importer that is
past that step, a member of a cycle that is not closed, runs in the job. A
rejected module on the stack keeps its status and remembers the error, and
the walk throws it when it returns to that module. A module that imports
the rejected module before that, in that walk or in a walk that a job
started, fails with the same error and does not run.

A job does not settle the promise of an Evaluate() that a job called on a
module on the stack: the cycle of that module has not run.
@robobun
robobun force-pushed the robobun/21c0c737/commonjs-evaluates-at-slot branch from 44aebcd to b8ebd27 Compare October 23, 2026 22:08
…luates it

A record made for a SyntheticSourceProvider::createDeferred() provider (Bun makes
one for a CommonJS module that an ES module imports) ran its generator in a pass
before its importing graph linked. So the CommonJS module ran before every ES
module of the graph, including the ones that are imported ahead of it.

SyntheticModuleRecord::evaluate() now runs the generator: the module runs where
InnerModuleEvaluation reaches its record. Until then the record hasPendingExports():

- ResolveExport answers Resolution::Type::Pending for a name that the record may
  export. Link leaves such an import entry or indirect export for later, and
  JSModuleRecord::execute binds them right before the module's body
  (CyclicModuleRecord::linkPendingImports), with the errors of link.
- A module namespace object that takes names from such a record is created
  without names and takes them at its first use after the record has evaluated.
- Code that is linked before the record has evaluated (an import cycle) reads
  undefined until it has. The record reserves the scope offset that such code
  embeds (SyntheticModuleRecord::reserveExportOffset), and also the offsets of
  code that records of other loaders share (JSModuleRecord::resolvesImportsLike).

AbstractModuleRecord::generateDeferredSyntheticModules() is removed.

CyclicModuleRecord gains one pointer (m_pendingImports). JSModuleRecord keeps
its size of 384 bytes: m_features moves into the tail padding of
CyclicModuleRecord.

The shell makes a deferred synthetic record for a *.deferred.js module, for
JSTests/modules/deferred-synthetic-module.js and
JSTests/modules/deferred-synthetic-module-loaders.js.
@robobun
robobun force-pushed the robobun/21c0c737/commonjs-evaluates-at-slot branch from b8ebd27 to ad787db Compare October 24, 2026 01:04
@robobun

robobun commented Oct 24, 2026

Copy link
Copy Markdown
Collaborator Author

@Jarred-Sumner @dylan-conway this draft needs a direction from you before more work goes into it.

What it does. A CommonJS module that an ES module imports runs where the evaluation of the graph reaches it, not before the graph links. With the paired Bun branch robobun/21c0c737/cjs-evaluates-at-import-slot, 9 of 9 import shapes run in Node's order. Main has 0 of 9: import "./setup.mjs"; import "./lib.cjs" runs lib.cjs first. The diagnosis on oven-sh/bun#4677 was the order of two CommonJS siblings, and oven-sh/bun#18086 works around that for reflect-metadata only. I found no issue that reports the case with an ES module ahead.

What it costs. The export names stay the keys of the evaluated module.exports, so importers link before the names exist and bind late. That is about 1,000 added lines in the module record sources. It removes the pre-link pass of #662 and conflicts with #781 in 7 files. A CommonJS file also no longer runs while its siblings load: an ES entry that imports express, lodash and react starts about 4 ms later (97 ms to 101 ms). As built, it breaks programs that Node and Bun run today (the table in the PR body). So it does not yet meet the bar of the review on oven-sh/bun#35971: "Not happening if it doesn't at least align with node".

The choice.

  1. Evaluate at the slot, as here. Then I rework the rules that break the cycle cases, and restack this on [JSC] A job that finishes a module from inside a module body does not strand or start its importers early #832, [JSC] Settle an Evaluate() of an EVALUATING module when the walk pops its cycle #871 and [JSC] Add live module exports for module mocks, plus fifteen smaller changes for bun test #781.
  2. Keep the pre-link pass of [JSC] Run a CommonJS module's generator before its importing graph links, not when its fetch completes #662 and make every CommonJS record deferred in Bun (decision 3 of Evaluate entry points in an event-loop task, outside the module loader's checkpoint bun#45551). That orders CommonJS modules among themselves, and the reflect-metadata special case should become unnecessary. It does not move a CommonJS module behind an ES module that is imported ahead of it.
  3. Leave the order as it is.

For choice 1, please also say whether #781 lands first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant