Repository navigation
Conversation
a95e1e5 to
44aebcd
Compare
|
Preview build of ad787db: |
… strand or start its importers early ECMA-262 has no job that runs while a module graph is evaluated. A host can run one from a module body: a microtask checkpoint, or a synchronous require() of an ES module. That job can finish a module with a top-level await while the modules that wait for it are in the middle of InnerModuleEvaluation or of AsyncModuleExecutionFulfilled. Inside InnerModuleEvaluation, GatherAvailableAncestors skipped an importer that was still EVALUATING and left its count of pending dependencies as it was. The importer never ran, and the promise of its Evaluate() never settled. A rejection gave a module on the stack the status EVALUATED with no cycle root, and the walk then read that root. Inside AsyncModuleExecutionFulfilled, GatherAvailableAncestors counts a module without a top-level await as done when it appends the module to execList, before the body of that module runs. A job that the body ran could finish the last other dependency of an ancestor. The ancestor then ran before that body ended, also when the body threw afterwards. A count of pending dependencies now goes down when a dependency has finished. asyncExecutionFulfilled() gathers the modules that the finished module was the last pending dependency of, runs them in the order of their [[AsyncEvaluationOrder]], and gathers again for a module without a top-level await when its body has run. A module gets its order after each module that it waits for, so this is the order of the specification's sorted execList. An importer on the stack of the walk gets its count decremented. The walk reads the count at step 12 of InnerModuleEvaluation. An importer that is past that step, a member of a cycle that is not closed, runs in the job. A rejected module on the stack keeps its status and remembers the error, and the walk throws it when it returns to that module. A module that imports the rejected module before that, in that walk or in a walk that a job started, fails with the same error and does not run. A job does not settle the promise of an Evaluate() that a job called on a module on the stack: the cycle of that module has not run.
44aebcd to
b8ebd27
Compare
…luates it A record made for a SyntheticSourceProvider::createDeferred() provider (Bun makes one for a CommonJS module that an ES module imports) ran its generator in a pass before its importing graph linked. So the CommonJS module ran before every ES module of the graph, including the ones that are imported ahead of it. SyntheticModuleRecord::evaluate() now runs the generator: the module runs where InnerModuleEvaluation reaches its record. Until then the record hasPendingExports(): - ResolveExport answers Resolution::Type::Pending for a name that the record may export. Link leaves such an import entry or indirect export for later, and JSModuleRecord::execute binds them right before the module's body (CyclicModuleRecord::linkPendingImports), with the errors of link. - A module namespace object that takes names from such a record is created without names and takes them at its first use after the record has evaluated. - Code that is linked before the record has evaluated (an import cycle) reads undefined until it has. The record reserves the scope offset that such code embeds (SyntheticModuleRecord::reserveExportOffset), and also the offsets of code that records of other loaders share (JSModuleRecord::resolvesImportsLike). AbstractModuleRecord::generateDeferredSyntheticModules() is removed. CyclicModuleRecord gains one pointer (m_pendingImports). JSModuleRecord keeps its size of 384 bytes: m_features moves into the tail padding of CyclicModuleRecord. The shell makes a deferred synthetic record for a *.deferred.js module, for JSTests/modules/deferred-synthetic-module.js and JSTests/modules/deferred-synthetic-module-loaders.js.
b8ebd27 to
ad787db
Compare
|
@Jarred-Sumner @dylan-conway this draft needs a direction from you before more work goes into it. What it does. A CommonJS module that an ES module imports runs where the evaluation of the graph reaches it, not before the graph links. With the paired Bun branch What it costs. The export names stay the keys of the evaluated The choice.
For choice 1, please also say whether #781 lands first. |
Draft, not for merge as it is. It waits for a maintainer decision.
Problem
import "./setup.mjs"; import "./lib.cjs"runslib.cjsfirst. Node runs them in import order.generateDeferredSyntheticModules()([JSC] Run a CommonJS module's generator before its importing graph links, not when its fetch completes #662) runs such a record's generator before its graph links, because link needs the export names.Fix
SyntheticModuleRecord::evaluate()runs the generator, whereInnerModuleEvaluationreaches the record. The pre-link pass is removed.hasPendingExports()andResolveExportanswersPending. Importers bind such imports inJSModuleRecord::execute, before the module body.Background
Downsides
export *of a CommonJS module readsundefined. Animport()reached through a CommonJS body never settles without [JSC] Settle an Evaluate() of an EVALUATING module when the walk pops its cycle #871.Notes
Known defects. All reproduced on a release build of Bun with this branch and the paired Bun branch
robobun/21c0c737/cjs-evaluates-at-import-slot. "Bun" is a build of main.export * from "./consts.mjs"andexport * from "./lib.cjs"; a module in an import cycle with the barrel readsVERSIONofconsts.mjs1.0undefinedclass Child extends Base,Basefrom an ES module behind the barrelTypeError: The superclass is not a constructor.promisesofnode:fs) behind such a barrelReferenceError: Cannot access 'promises' before initialization.on every callimport()that a CommonJS body reaches throughrequire()of an ES moduleSyntaxError, exit 1TypeErrorERR_REQUIRE_CYCLE_MODULE. Bun: no errorReferenceErrorwhen it reads a binding--collectContinuouslyexportsweakly until the record owns itPaired Bun branch, known gaps. Also reproduced on the release build.
require.cacheentry while it runs is run again for every later importer (3 runs where Node and Bun have 1).require()rejects withFailed to evaluate module, not with the error of the module.mock.module()of an ES module that re-exports a CommonJS module that has not run yet does nothing. After a failed import of a CommonJS module,mock.module()of it does nothing.module.id"."and the order ofrequire.cachekeys still follow the order in which the fetches settle.Rework that these point to (not built).
export *of a pending record arrive later.CyclicModuleRecord::linkImportEntry, so a lazy export is not skipped.JSSourceCodethatmakeModuleholds, so nothing can collect it between the fetch and the record.innerModuleEvaluationafter the requested modules, and again when a cycle completes.Measurements. Two release builds of Bun (no LTO, identical flags) on this branch and on its parent commit. Instruction counts are from
PTRACE_SINGLESTEP, or from the hardware counter read at breakpoints, on the main thread.innerModuleEvaluationfornode:fs)AbstractModuleRecord::linkof a builtin synthetic record, per import edgelink()prologue on a 4-module ES graph (entry ofAbstractModuleRecord::linkto entry ofCyclicModuleRecord::link)bench/module-loaderimport.mjs(501 ES modules)link()of that graphinnerModuleEvaluationof that graphresolveExportImpl, 100 named imports through two levels ofexport *over 50 modulesJSModuleRecord::executeof an empty ES modulelink(), then itsexecutelink()of such a modulePendingImportsand its vector), 7 first uses of a size class that the generator paid for earlier on the parentsizeof:JSModuleRecord,CyclicModuleRecord,SyntheticModuleRecord,AbstractModuleRecord,JSModuleNamespaceObjectCyclicModuleRecordgains one pointer.JSModuleRecordkeeps its size becausem_featuresmoves into the tail padding ofCyclicModuleRecord.WebAssemblyModuleRecordkeeps its cell size (336).SyntheticSourceProvidergrows from 208 to 216 bytes (the owner callback).Tiers. In an import cycle, 2,000,000 reads before the record has evaluated give
undefinedand 2,000,000 reads after give the value, with the default tiers,useJIT=0,useDFGJIT=0,useFTLJIT=0, and with thresholds that force the FTL.What the new tests cover. The shell makes a deferred synthetic record for a module named
*.deferred.js: the file is the body offunction (exports) { ... }, and the record exports the properties thatexportshas after the body ran.import "./setup.js"; import "./lib.deferred.js"setup.js, onceexport *,export { x } from,export * as nsundefined, then the valuesexport *of the record, in a cycle with a module that imports from the barrelundefined, then the valueSyntaxErrorafter the body ran. Other importers are not affectedexport *of the record and of an ES module that export the same nameSyntaxError(ambiguous)awaitahead of itxis at another place among the exports of each loader's recordxShared code. Records for one module in several loaders share the
ModuleProgramExecutablewhen every import resolves to the same place (JSModuleRecord::resolvesImportsLike). For an exporter that is not a source text module that place is a scope offset, which did not exist before the exporter had run. A record that shares code now asks the pending exporter for the offset that the code embeds, and the exporter puts the name there when it builds its environment. A reservation that cannot be kept refuses the sharing, as a different offset did before.Early code.
JSScope::abstractResolvelinks an import that still waits asModuleVarwith a reserved offset, so every CodeBlock of oneUnlinkedCodeBlockkeeps the same resolve type.JSModuleRecord::fillImportSlotgives such code an environment that holdsundefinedat that offset, and does not fill the import slot, until the exporter has its environment.