Repository navigation
AOT: files that a load has made, symbols behind loops, mocks in test runs, and smaller fixes - #1067
Conversation
…by then While a load goes on, the loader makes the records of the files that a module requests before it stores them in the registry. An ES module that is required meanwhile read what such a CommonJS file exported when its record was made.
…t is stored The lines stand under `ENABLE(AOT)`. A module that isn't compiled is asked for its class once when its record is made and once when it is stored. Instructions a call, once for each module made: - runModuleRegistryFetchSettled(): 50 before, 60 now on x64; 53 before, 62 now on arm64. - runModuleRegistryModuleSettled(): 48 before, 58 now on x64; 49 before, 66 now on arm64.
…made when it is bound A file that is loaded from disk and imports such a member saw a later store into the module's object.
…CompiledCode() A process without compiled code asks one byte where it asked the class of the module. Instructions of `initializeEnvironment()` in plain Bun: 319 on x64 and 307 on arm64, as before; a named import takes 62.8 before, 61.2 now on x64, and 61.0 before, 55.8 now on arm64.
…ounts The compiler's own check of that failed on some programs. No wrong answer is known. - A getter whose other objects aren't in sight is called by unseen code. - The message of the check says what its shadows are.
Whether a CommonJS file may be released at the end of a test file asked for the last import() of it, settled or not, so a mock of what the file requires was seen by the test file behind it. Only a pending import keeps it.
A program with direct evals was linked by two flows, one after the other: one without what its evals show, and one with it. The second takes up the first one where it stopped. What the link finds is the same.
…at the file exports by then For a file that is mocked by its original, and that is read while it changes its exports.
The mocks of a test file end in no order. The end of one brought a file back, and the end of another released it, so the next test file ran it again, depending on the paths.
…ts tests allow A table that is its own row, or that stands in itself twice, was moved as if its rows were numbers. An array of int32s holds no rows, so a store into it leaves them as they are.
… evaluated The promise of importActual() of a file that two mocks of an earlier test file lay over never settled.
…s when its text comes Of two or three import() calls at once, a CommonJS file of one graph ran in front of the body of another graph.
It answered `{ done: true }`. What a termination has left is not complete.
For the host, which tells the mocks that hooks lay from those of the preload file.
For what vi.doUnmock() does over a CommonJS file that a preload ran.
On arm64 the build of a loop that polls and whose numbers are handed over in turns could stop. Neither way dominates the other.
… what the front has proven
`while (x < w) { s += g[y * y + x] ?? 0; x++; if (x > 300) break; }` stopped a build.
…for size They only look at what the compiler has made. The compiler is smaller by about 20 KB.
… file's last instance A link to that file is written to the instance that the file had before its mock is run, and no instance is made for it there. Only a file that is in no cycle of imports, or whose factories can see it, is linked so. Test runs.
…iler knows
`const brand = Symbol("brand")` behind a loop, laid on a prototype with `Object.defineProperty()`:
the process ended with a check of the compiler. A call in a loop makes another symbol each time, and
which blocks are in loops is known after the scan.
…e test file The loader keeps the modules of the graph that it was loading.
…es leaves with its original What a run with resets reads by a statement. Who asks for one of its importers next is told.
The types of thousands of nodes may name the same layouts, and what lies in each is visited where it is named first. For an entry that exports what a file outside the build answers, the work of a link grew with the square of what hangs on it: 9.8, 11.4, and 17.1 billion instructions of a build beside 250, 500, and 1,000 modules, now 9.4, 9.8, and 10.7 (arm64).
…st for an array Builds of the edit distance whose rows are pushed stopped.
There was a problem hiding this comment.
I reviewed this PR and didn't find any bugs. Because it is 24 commits across the AOT link/flow analysis, the module loader's mock and release bookkeeping, and two shared runtime paths, a human look would still be worthwhile.
What was reviewed:
- Shared runtime touch points: the new code in
CyclicModuleRecord.cppandJSMicrotask.cppstays inside#if ENABLE(AOT)and keeps existingBUN_JSC_ADDITIONSnesting; themade-and-not-storedflag is set only when the make-module promise already fulfilled with the same record and is cleared again inrunModuleRegistryModuleSettled. - Link changes: each
#pragma clang attribute push(minsize)has a matchingpopin the same file (AOTLink, AOTLinkFlow, AOTBodyVerifierFlow, AOTMachineValidate, AOTWalkGuesses);forEachContainedNode's new signature has a single caller;FunctionsThatEvaluate/LinkBeforeEvaluationdo not collide with other names inaot/; the secondlinkBeforeEvaluationcall passes an empty split so itstablesis always present. AOTScan::mayLieInLoopsuffix-min and theMoveLoopkinds-allowed check were read for index bounds and consistency with the emitter; the generator change now leaves state untouched only for termination exceptions.- Not done by this review: running
Tools/Scripts/aot/check-unified-sources.pyand building withENABLE_AOT=0and=1.
Extended reasoning...
The change is 608 insertions across 28 C++ files, almost all in the experimental AOT compiler (link/flow analysis, codegen loop handling, module instance and mock bookkeeping for bun test) plus two guarded additions to the shared module loader and module record linking. It touches no auth, crypto, injection, or data-exposure surface. The diff is internally consistent (pragma pairs, signatures, guards) and the bug-hunting rounds found nothing, but the size, the number of interacting state flags in the module runtime, and the absence of tests in the diff mean confidence rests on the author's reported test runs rather than anything verifiable here, which decided defer over approve.
|
No actionable comments were generated in the recent review. WalkthroughThe PR changes AOT direct-eval analysis, module loading and mock handling, and loop and code-generation checks. It also updates object-construction paths, bytecode-generator exception state, and Clang function-size attributes. ChangesAOT direct-eval and link-flow analysis
AOT module loading and test mocks
AOT loop and code-generation checks
Possibly related PRs
Merge Risk:
|
|
Preview build of 12f515c: |
… the object that was made A constructor function that gives another object in front of its own stores, as with `return new Other()`: the process ended with a check of the compiler. The object that was made is dropped there, and nobody sees it. - The check stands in the arm that gives the object that was made: in the direct call, in the copy of the body, and in the entry from unknown code.
- `autobuild-preview-pr-1067-12f515c5` is built from 12f515c583d19f52512e626a1a54f54ea4b66ab4, the head of the pull request.
- `ae8978d0331bfce97259b446378265eb327c407a` is oven-sh/WebKit#1067 on WebKit's main. Its tree is that of the preview that the branch pinned.
- `ae8978d0331bfce97259b446378265eb327c407a` is oven-sh/WebKit#1067 on WebKit's main. Its tree is that of the preview that the branch pinned.
…s of the engine's fixes (#49108) Bun's side of fixes of the ahead-of-time compiler (`bun build --compile --aot` and `bun test --aot`, both experimental), with the programs and test runs of the engine's. The JavaScriptCore side is merged: oven-sh/WebKit#1067, which is `ae8978d0331bfce97259b446378265eb327c407a`. This follows #49081. 378 files, +2,351 -335: src 23 files, +414 -93; the pin; docs; test 353 files, +1,933 -239. Programs and types - `--aot-allow-code-from-disk` builds a file that names an evaluator only to test it. - `eval("import('./a.js')")` and `new Function("return import('./a.js')")`: the replacement of `eval` or `Function` is called where the build can see that the program puts one in place, also by `Object.assign()` and `Reflect.set()`. Elsewhere the call throws an `EvalError`. The docs say it. - A member of an enum of another file is read off the enum where the program stores into it through a load of one of several files. Test runs - A CommonJS file that a preload ran stays loaded under two mocks of a test file, and the original that the test asks for behind them is a run of its own. An ES file of a preload that a mock couldn't release is run again for its original. - A hook's uncalled mock doesn't come back for the next test file. - A test's `vi.doMock()` of a file that a preload mocks by its original loads the file again, also behind a first file that only resets. In a run with resets, such a mock is taken out with its original. - Where `bun test` runs a file again, `vi.doUnmock()` over a CommonJS file of a preload and a required mock with an async factory say so. Order files - `compile.aotOrder` given as a getter is asked once. Tests - Files that are made and not stored while a load goes on, files from disk that import what a built-in module makes when it is first read, files of graphs that are imported at once, generators that a time limit ends, tables that are their own rows, loops that hand over numbers in turns, loops whose keys are sums of squares, symbols made behind a loop, mocks that end together, `importActual()` of a file that two mocks lay over, modules of requires that failed, rows of an edit distance that are pushed, and rows that name each other in a list that the entry exports. - Constructor functions that give another object. - The test of a region that a pool's thread starts has a pool of its own, and waits with a deadline. - The durations of the tests under AddressSanitizer are the measured ones. Known: the compiler's own check that its types have reached a fixed point fails on a few large bundles, as on main.
main has #1067 (AOT). No file conflicts. Six files change on both sides: aot/runtime/AOTModuleEvaluation.cpp, AOTModuleInstances.cpp, AOTModules.cpp, AOTModules.h, runtime/CyclicModuleRecord.cpp and runtime/JSMicrotask.cpp. #1067 notes which CommonJS files a load has made and not yet stored (setIsMadeAndNotStored()): in runModuleRegistryFetchSettled(), in runModuleRegistryModuleSettled() and behind the steps that transferSynchronousLoad() runs. That is bookkeeping of the record and runs no code of the program. It also reads a deferred export of a built-in module where a binding is linked (readDeferredExportForBinding()), inside the job of the load. #1067 adds no job of the module loader, no call of import() that is made again and no place that hands the async context of a load on. The lines that this branch changes against main are the same as before the merge. ae8978d is #1067 as merged. Bun's main pins it (oven-sh/bun#49108).
main has #49108: its pin is ae8978d033, which is oven-sh/WebKit#1067 as merged. WEBKIT_VERSION is now the preview build of the head of oven-sh/WebKit#941 (0d50be2a58), which is ae8978d033 with the change. Conflicts: the pin, and test/parallel-allowlist.json, where main and this branch each add entries at the end of the list. All three entries stay.
…ong views, and smaller fixes (#1069) Fixes of Bun's ahead-of-time compiler (`bun build --compile --aot` and `bun test --aot`, both experimental). Bun's side is oven-sh/bun#49145, which pins this one. This follows #1067. 22 files, +213 -65: 14 in Source/JavaScriptCore/aot/ (+176 -30) and 8 outside it. Programs - A name that a program was found not to have on a short typed array (`4294967295`) was taken for one that a view of 4,294,967,296 elements doesn't have either. It is asked of the view. - Code that is made at run time and stores to a constant of a script throws, as in plain Bun. - `for..of` over an object whose `[Symbol.iterator]()` hands back an iterator of a collection that code has laid a `return()` or a `next()` on, kept or stepped, calls the iterator's own. Test runs - Code from source beside a preload's mock whose entry a reset has removed asks for the entry again. - An import under another file's patch and a file that threw in an earlier test file are told apart from what they were taken for. - A `require()` under a second patch that waits is told of. Builds - A build that stopped under `--aot-harden=proof-checks` is built: where the name that a check asks about is handed on at the head of a loop, at a join or at a call that was copied, and where something stands between a branch and what it narrowed. One that stopped under `--aot-check=VerifyEachStage` is built too. If you don't use `--aot` - The loader asks whether a process has compiled code in front of a module's class. A fresh module takes 17 instructions fewer of 145,683 on x64 and 55 of 142,515 on arm64, one that imports 44 and 194 fewer. innerModuleLinking() and innerModuleEvaluation() take three to eleven more. - A store of code made at run time into a global variable takes 66 instructions on x64, as before, and 63.5 on arm64 (it took 61.5). Known: the compiler's own check that its types have reached a fixed point fails on a few large bundles, as on main. No wrong answer is known.
Fixes of the ahead-of-time compiler (experimental) behind #1064. The programs and test runs are in oven-sh/bun#49108.
AOTModuleEvaluation.cpp,JSMicrotask.cpp).AOTModuleLinking.cpp,CyclicModuleRecord.cpp).import()calls at once, a CommonJS file of one graph ran in front of the body of another graph.{ done: true }.AOTScan.cpp).while (x < w) { s += g[y * y + x] ?? 0; x++; if (x > 300) break; }, and on arm64 a loop that polls and hands over numbers in turns.AOTWalkLoopsBuiltTwice.cpp,AOTCodeGenerator.cpp).AOTCodeGeneratorMoveLoops.cpp).bun test --aot: a mock of what a CommonJS file requires was seen by the next test file, where an import of the file had settled.bun test --aot: a file that is mocked with what it requires ran again in the next test file, depending on the paths.AOTTestRunMocksLetGo.cpp).bun test --aot: a file mocked by its original, read while a CommonJS file that has run changes its exports, saw what it exported before.bun test --aot:importActual()of a file that two mocks of an earlier test file lay over never settled.bun test --aot: the modules that a failedrequire()had fetched were left to the next test file.bun test --aot: mocks that a hook laid, anunmock()over a file of a preload, and a preload's mock by its original were seen by the next test file.AOTModuleInstances.cpp); the rows are in oven-sh/bun#49108.AOTWalkListReadsInLoops.cpp).AOTLinkFlow.cpp,AOTLinkReach.cpp).AOTLink.cpp,AOTLinkReach.cpp,AOTFacts.cpp).FlowFixpoint) failed on some programs at getters and at counts. No wrong answer is known.A commit for each row or two. Two commits have no row: checks of the link, the machine code and the types are compiled for size, and a link to a file whose mock is still waiting is to the file's last instance.
If you don't use
--aotJSMicrotask.cpp: the loader asks a module for its class when its record is made and when it is stored. For each module that a plain program makes,runModuleRegistryFetchSettled()takes 60 instructions on x64 and 62 on arm64 (it took 50 and 53), andrunModuleRegistryModuleSettled()58 and 66 (48 and 49).CyclicModuleRecord.cpp: what a binding of a compiled module is made of is recorded behindhasCompiledCode(). A named import takes 61.2 instructions on x64 and 55.8 on arm64 (it took 62.8 and 61.0); the rest ofinitializeEnvironment()is the same.Source/JavaScriptCore/aot/changes.What was read
The native tests pass on Linux x64 (2,180) and arm64 (2,162), and so do 2,865 tests of the IR on both. Of the engine's tests 574 pass on arm64 and 576 on x64, where five fail that fail without the changes too. The test runs and programs of the changes print what plain Bun prints: on Linux x64 40 of 40, in both orders, by default, in random order, and rerun, and on arm64 all 44 of a text that held two forms more.
Known issues
FlowFixpoint) fails on a few large bundles, as on main. No wrong answer is known.