Repository navigation
AOT: keys and steps of generators, built-ins, arguments passed on by apply() - #1055
Conversation
- `list[key] = key` stored the number 1 in place of the string "1" where the compiler knew `key`
to be the text of an array index, as in `for (const key of ["0", "1"]) list[key] = key`. Such a
store is compiled as written.
- A step of a generator that only calculates with numbers is taken without resuming the frame.
Where the step ended its turn with a test against the value that a variable had before the turn
(`do { yield s; u = s; s = s + 1; } while (s <= u)`), the test compared the new value with
itself, and the generator did not end. The operands are read where the test stands.
- Where a loop inside another yielded a value that the loop around makes, some ways to the yield
made the value and others left it in the frame, and the step read the register on both. Such a
step resumes the frame.
- A loop over a generator that is left early was walked in place although a built-in in it, such
as `String.raw`, reads a property of a primitive through a getter of the program, which can
store the generator's `return`. The program aborted.
…d methods of one name - The build stopped with `the program calls built-in functions that the load stage removed` where a built-in is read behind `?.` (`globalThis.Temporal?.PlainDate`), out of a closure, or out of a variable of the file that may be undefined, and a static of it is called. - A program stopped with `internal AOT error: type check failed` where a function's prototype is an instance of another function whose field has the name of one of its methods.
- The build stopped where a class inside a function that runs once fills a static field at its
first read (`static get max() { return this.limit ??= readLimit(); }`) and a method of another
class reads it in a loop that catches.
- A pattern that takes a pair apart throws where the step in front of it is done.
…of list methods - A function that passes its arguments on with `f.apply(this, arguments)` inside another function lost methods of the object: the program stopped with a TypeError. What the arguments object holds flows to every place where a call by binding hands the list over. - Where a loop adds `v[i]` and an element before it, such as `v[i - 1]` or `v[i >> 1]`, over a list with a hole, the sum could be a number where `bun run` gives NaN. A turn behind the first that leaves in the middle hands over what stood in front of the turns. - Where a turn of a loop stored in front of a chain of list methods and a guard in a callback of the chain failed, an executable could run for ever and some builds stopped. The undoing of the store is no operation of the chain.
- `Reflect.set()` of a property of an object that has lived a while answered false although the property was stored.
- `import(path, options)` of a file that only `require()` has loaded read `options.with` twice while a mock's factory was pending. It is read once. - A CommonJS file of a package required an ES package whose `vi.mock()` has an async factory that takes its original, and the test file did not load (`require() async module`). - A graph that a `require()` runs while an `import()` is in flight gave undefined for what it reads of a CommonJS file through a re-export, or a ReferenceError through `export *`. - A test file that requires a JSON, TOML or YAML file and stores into it gave its object to an import of it in another test file, where `vi.resetModules()` stands anywhere in the run. - Behind a `vi.doMock()` that nothing loads over a required CommonJS file, a later import had the mock's names; and the names of a barrel over a CommonJS file that an earlier test file loaded were gone.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the
Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. WalkthroughThe AOT changes update plan metadata, link-flow facts, getter-field tracking, code generation, and walker analysis. They also change module import and mock handling, adjust ReflectSet barrier handling, and add IR link tests. ChangesAOT analysis and link flow
Code generation and walking
Module runtime
ReflectSet barrier handling
Merge Risk:
|
There was a problem hiding this comment.
Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.
Beyond the inline finding, I also checked a few other paths and ruled them out: the ReflectSet barrier stub's new beforeReturn runs after the saved registers are restored and on the poll path too, so the 1 in the result register survives; the diff adds no new namespace-scope names (all additions are member functions), so the unified-sources constraint holds by reading, though check-unified-sources.py was not run; and the isImmutable callers in AOTOperatorEmitter exclude the maker function itself, while AOTLinkCalls::callName still notes BuiltinUsedViaFlow whenever the member lookup is not a found, unshared, non-light-form hit.
Extended reasoning...
The change spans 25 files under Source/JavaScriptCore/aot (compiler link/walk/codegen, write-barrier stubs, and the module run time for test runs) with one confirmed codegen regression already posted inline; it touches no auth, injection or data-exposure surface. Source/JavaScriptCore is covered by CODEOWNERS and the change is large and interdependent, so approval is not appropriate; the body records only what was additionally examined and found not to be affected.
| auto read = [&](ValueID id) { | ||
| if (!id || m_code.isConstant(id)) | ||
| return; | ||
| auto definition = m_placesOfBlocks.find(m_code.blockOf(id).index()); | ||
| if (definition == m_placesOfBlocks.end() || isBehindYield(id)) | ||
| return; | ||
| found |= !((dominators[place] | reachable[place]) & (1u << definition->value)); |
There was a problem hiding this comment.
🟡 (optional) Numeric generators whose loop has a planned block that leaves the loop without reaching the yield block again now lose the native leaf step they get on the base branch. In hasPartialDefinition a value defined in m_blockOfYield before the yield is accepted only when the yield block is in dominators[place] | reachable[place] at AOTSavedFrameEmitter.cpp:857. A region block whose only successor is unplanned (e.g. if (i >= n) { total += i; break; } ending in GiveWay) reaches neither, so reading the loop-carried value there sets found and order() refuses the step. Fix: treat definitions in m_blockOfYield that are not behind the yield as always available, instead of relying on reachability. This is a lost optimization only; results stay correct.
Why this was flagged
A generator such as function* g(n) { let i = 0; for (;;) { i += 1; yield i; if (i >= n) { i = i * 2; break; } } return i; } where the yield block is the loop header and the block E holding i * 2 ends in a Jump to the Return block, so E is a planned block whose only successor is unplanned. order() at AOTSavedFrameEmitter.cpp:817 now also requires !hasPartialDefinition(). For place = E, read(i) at AOTSavedFrameEmitter.cpp:851-857 finds the definition in m_blockOfYield (not behind the yield); reachable[E] is only E and dominators[E] holds only start and E, so the test at line 857 sets found and plan() returns nullptr at line 628-629. On the base branch order() returns m_entries[0] == 1 and the leaf step is emitted with a GiveWay at E, so these generators ran their numeric steps natively; after the change they always resume through the full generator. No safeguard restores the step: the value is fully defined on every path, so the refusal is a false positive of the new check, not a real partial definition.
Verification: In /home/claude/webkit/Source/JavaScriptCore/aot/codegen/AOTSavedFrameEmitter.cpp order() now returns m_entries[0] == 1 && !hasPartialDefinition() (line 817). For an exit block E whose successor is the unplanned Return block reachable[E] == {E} and dominators[E] == {start, E}, so read() sets found (line 857) and a leaf step is refused. A regression in optimization only.
|
Preview build of 2cbcf27: |
…mage - A build that refuses code from outside the image, as every hardened one does, left out a function of the library that only a variable exported by an entry names: a built-in in a variable is followed to the variable's readers, and nobody inside such a build reads what an entry exports. A variable that an entry exports is let go, as before; every other one is followed.
- In builds that refuse code from outside the image, a function of the library was left out where an entry hands it out under a namespace (`export * as lib from "./lib"`): the commit before covered only the variables that an entry exports by name. A built-in that is written in the library is let go wherever a file stores it in a variable, as on main. Every other built-in is followed to the readers of the variable. - A link test of each form that hands a built-in out.
- A read of a name that a getter answers with a field (`get mode() { return this._mode; }`) kept its
first answer across a store into that field (`r._mode++`), where the object has no layout that the
compiler knows, as `Object.create()` and descriptors make. The same held for `length` in the
condition of a loop. Such a read rests on the fields that getters of the name return too.
- The run time answers a read with the field of a getter under the getter's own name alone. A getter
without a name of its own, `get [k]() { .. }`, answers under any.
There was a problem hiding this comment.
Beyond the inline finding, I also checked two things on this push: the keyed-read path in AOTWalk.cpp (o[k] with a constant text key) can still keep a stale getter-field answer across a store into that field, but that gap predates this PR, so I did not flag it; and the new namespace-scope statics added for unified-source bundles (findFieldsReadByGetters, answersUnder, readsOfNamedLookup, nameAnsweredByGetter) have no name collisions elsewhere under aot/.
Extended reasoning...
Nine commits touching only Source/JavaScriptCore/aot (compiler plan, walker, link facts, image tables, and the module/test-run runtime); no security-sensitive surface. The ruled-out items above are the only things examined beyond the posted inline finding; the inline finding on the hardened-build library-function path is what keeps this from an approval.
The latest push brings back an issue flagged in an earlier review; replied in that comment's thread.
Still open from earlier reviews (2):
- Unresolved: 1 minor or pre-existing, 1 blocking on lines changed since (possibly already fixed).
If you have decided not to act on one of these findings, resolve its thread (a reply alone leaves it open) and the next review stops counting it. To review this commit again now, use Re-run on its "Claude Code Review" check.
- In builds that refuse code from outside the image, functions of the library that nobody names were left out where an entry hands out a built-in that holds them: `export const p = Array.prototype`, or the same in a namespace, under another name or as the default. The commit before looked at the built-in that a file stores, and a holder has no function of its own. A variable that an entry hands out is let go, as on main, and every other one is followed to its readers. What an entry hands out is found by the walk that Link takes through the exports of a module that the host asks for whole. - A link test of each form.
There was a problem hiding this comment.
Beyond the inline findings, I also checked two things the unified build and image format make sensitive: the new namespace-scope statics (findVariablesHandedOutInProgramFacts, findFieldsReadByGetters, nameAnsweredByGetter, answersUnder, readsOfNamedLookup, readsOfLength) have no other definition under aot/, and the 12-byte FieldGetterData keeps function as the first ordering key so the run time's lower_bound on function alone still agrees with the encoder's AOT::sort.
Extended reasoning...
The ten commits touch only Source/JavaScriptCore/aot: link analysis, plan, walker, codegen, image tables, property caches and the test-run module runtime, with no security-sensitive surface. Two confirmed findings in AOTFactsOfProgram.cpp are posted inline; this note only records the unified-source name check and the image-record ordering check that were ruled out.
Still open from earlier reviews (2):
- Unresolved: 1 minor or pre-existing, 1 blocking on lines changed since (possibly already fixed).
If you have decided not to act on one of these findings, resolve its thread (a reply alone leaves it open) and the next review stops counting it. To review this commit again now, use Re-run on its "Claude Code Review" check.
- Takes back the rule of the commits before that Link follows a built-in into a variable, because in builds that refuse code from outside the image it left out functions of the library that an entry hands out: such a built-in is let go, as on main. - The link tests of the forms in which an entry hands a built-in out stay, without the one that held only by that rule.
There was a problem hiding this comment.
I reviewed the latest push and found no new bugs; because this is a +733/-81 change across the AOT compiler (plan, walk, codegen), link and runtime, a human look is still worthwhile.
What was reviewed:
- The final commit takes
Source/JavaScriptCore/aot/facts/AOTFactsOfProgram.cppback to the base revision, so the hardened-build (exclude=CodeOutsideImage) hand-out paths my earlier inline comments pointed at are no longer part of this diff. - The
FieldGetterData8-to-12-byte record: the encoder in AOTLayoutTables.cpp, the only runtime reader (findFieldReadByGetter, lower_bound onfunction) and the table check (checkAscendingonfunction) agree; a getter whose name is not in the identifier table is dropped from the table rather than mis-answered. - The new namespace-scope statics (
nameAnsweredByGetter,answersUnder,readsOfNamedLookup,findFieldsReadByGetters) are unique across aot/, as the unified-sources rule requires; thehasPartialDefinitionbitsets fit (maxBlocks= 12 under the 32-bit bound) and the ReflectSet barrier path sets its result before returning. - No build, .ir test run or check-unified-sources.py run was done in this review; only reading.
Extended reasoning...
The diff touches 40 files under Source/JavaScriptCore/aot only: plan (new isImmutable and FieldsReadByGetters), walk (named-lookup effects, Map-pair patterns, generator-in-loop effects), codegen (numeric generator steps, unrolled-loop turns), link (apply() arguments, appended prototype fields), the image record FieldGetterData (8 to 12 bytes) with its runtime readers, module/mock handling in test runs, and a ReflectSet write-barrier stub; it touches no auth, injection or data-exposure surface. The bug hunt ran dry with no findings, and the file my three earlier rounds of inline findings targeted (AOTFactsOfProgram.cpp) has been reverted to base by the last commit, which is the substantive change since my last review. The change is large and complex compiler/runtime work and Source/JavaScriptCore is listed in .github/CODEOWNERS, so approval without a human look is not warranted.
Still open from earlier reviews (1):
- Unresolved: 1 minor or pre-existing.
main has #1055 (AOT). No conflicts: four files of aot/runtime merge by themselves, and the lines of this change are the same against the new main as against the old one. #1055 changes how mocks and requires of a test run are kept, and hands the parsed options of an import() to importOriginalUnderPatch(). That call still goes through the loader. #1055 adds no job of the module loader, no call that is made again and no synchronous load. 260715a is #1055 as merged.
Bun's side of the next round of fixes of the ahead-of-time compiler (`bun build --compile --aot` and `bun test --aot`, both experimental). The JavaScriptCore side is oven-sh/WebKit#1055, whose merged commit `scripts/build/deps/webkit.ts` pins. This follows #48815. 241 files, +2,324 -53: 12 files of src/aot (+335 -32), the pin, and 228 files of test, most of them stored programs and test runs. Executables - The load stage keeps the functions of a built-in that a program reads behind `?.`, out of a variable or a closure, changes or copies (`globalThis.Temporal ??= { .. }`, `Object.freeze(Temporal)`, `Symbol?.iterator`, `{ ...Iterator }`). The build stopped with `the program calls built-in functions that the load stage removed`. - A store under a name on `globalThis`, such as `globalThis.__DEV__ = true`, changes that name alone. Test runs - A test that stores into or spies on an exported helper that a compiler printed (`__publicField`, `__name`) is seen by the helper's own file, as in `bun test`. - A star barrel beside a CommonJS file under mocks: a mock of the barrel was not seen after an earlier test file had loaded it, and `spyOn()` of a name of the CommonJS file was refused. - The build of a test run stopped with `nothing defines ImplicitConstructor` where a set-up file mocks a file and a file above it holds a class expression that only an unused export uses. Tests - Stored programs and test runs of these fixes and of the engine's. A TLS server in `socket.test.ts` sends its whole reply where a `write()` takes a part of it, and a worker in `node-http2.test.js` that exits before it has connected fails the test at once. Known: one form may still stop the build, as before: a static of a built-in that a variable of a module holds (`const Instant = globalThis.Temporal?.Instant`, then `Instant.from(v)`), in a file that also has a class that extends a typed array.
…f checks (#1059) Eight fixes of Bun's ahead-of-time compiler (`bun build --compile --aot` and `bun test --aot`, both experimental). Bun's side is a pull request that pins this one. This follows #1055. 14 files, +204 -30, all in Source/JavaScriptCore/aot/. - A CommonJS file that throws while import() calls of it are in flight runs again behind them, as Bun runs it: a compiled program threw the first run's error. The same where the file calls import() of itself and throws, and where several retries at once come behind a required ES module that failed. - A name that two `export *` give, one of them through a CommonJS file that an earlier test file has loaded, is ambiguous in a test run. A compiled test run handed out one of the two. - With --aot-harden=proof-checks a build stopped where a list has a getter that stores into a list and a read that the compiler remembers is made again to be checked, and where a pattern takes every name of an object away. A program stopped with a failed type check where closures hold a variable that a loop stores ahead of its reads. - `for (const x of list) switch (x) { case "a": n += x.name; }` ended the build by a signal. - What a mock of a file that runs from source replaces in compiled code can be read by the host that puts it back, such a mock reaches the names of a CommonJS file, and the mock is read, stored and put back on the load that the source file's names are bound to. Known: where a CommonJS file behind such a mock is loaded a second time, the mock can reach the readers of its first load, and a namespace that was imported from a compiled barrel reads the second load. #1050 is a draft of the same by the place that the mock wrote.
The JavaScriptCore side of the next round of fixes of Bun's ahead-of-time compiler (
bun build --compile --aotandbun test --aot, both experimental). This follows #1047.Merge order
Its own CI is what keeps it from merging; it waits for nothing in oven-sh/bun. The schema of the intermediate representation is as on main. The image's record of a getter that answers with a field holds the getter's name too; a build refuses an image of another build, as before. Bun's part of this round (its front end, the stored programs and the test runs) is separate and pins the merged commit of this.
What it is
The eleven commits are separate:
list[key] = keystored the number 1 wherekeyis the text "1" of an array index. A generator that only calculates with numbers could run for ever, where a step ends its turn with a test against the value that a variable had before the turn, and read a register that some ways had not made, where a loop inside another yields. A loop over a generator that is left early was walked in place although a built-in in it reads a property of a primitive through a getter of the program.?., out of a closure or out of a variable that may be undefined, and a static of it is called. A program stopped where a function's prototype is an instance of another function whose field has the name of one of its methods.apply(), sums over holes, chains of list methods. A forwarderf.apply(this, arguments)inside a function lost methods. A sum of an element and of one before it over a list with a hole gave a number for NaN. A loop that stored in front of a chain of list methods could run for ever.Reflect.set()of a property of an object that has lived a while answered false although the property was stored.import(path, options)read its options twice under a pending mock; a package could not require a package whose mock awaits its original; graphs that are required during an import, data files that are required before they are imported, and barrels of mocked CommonJS files answer as inbun test.If you don't use
--aotNothing changes: all commits touch
Source/JavaScriptCore/aot/only.What's in it
40 files, +733 −81 against #1047's merge, all in
Source/JavaScriptCore/aot/: 3, 4, 8, 6, 1, 5, 2, 2, 12, 8 and 10 files per commit (a file that two commits change counts twice).Tests
New link tests for the arguments that
apply()passes on and for functions of the library behind built-ins that entries export and hand out. The stored programs and test runs of these fixes belong to Bun's repository.Known issues
As on main: what oven-sh/bun#48771 and oven-sh/bun#48815 list. This pull request adds none that is known.
CI
The lanes here build
jscand run JavaScriptCore's tests. The tests of the twohardened-aotlanes were red on the first six commits, six tests of functions of the library; the seventh mends them. A build of Bun with the first six commits passes the 2,832 tests of the intermediate representation on x64 and on arm64, and the native tests of the compiler's runtime (2,178 on x64, 2,160 on arm64); with the seventh, the same on both and the six under hardening on both; with the eighth, 2,833 of the first kind on both. The commits were built together, not each alone.