Repository navigation
chore(sync): merge upstream Bun through 34cbc70fa1 - #218
Merged
Merged
Conversation
### What does this PR do? Moves `WEBKIT_VERSION` from `5e3554face13` to `680ef47cec2c`, the head of oven-sh/WebKit `main`. The `autobuild-680ef47c…` release is published. What the fork's `main` gained in that range: - oven-sh/WebKit#889: merge of upstream [`f59345e04a8d`](WebKit/WebKit@f59345e) (2026-10-22). 193 upstream commits since `33ce3bf11a71`, 37 of them in JavaScriptCore, WTF or bmalloc. - oven-sh/WebKit#892: a completely full block that the allocator stumbles on is recorded as allocated. - oven-sh/WebKit#880: the control flow profiler counts the parts of a loop header. - oven-sh/WebKit#807: `BytecodeOptimizer` aborted when jump offsets settled one pass before the forced widening. - oven-sh/WebKit#881: CI only (sha256 for toolchain downloads, FreeBSD 14.4, which is what Bun already targets). The Android NDK is unchanged. `JSType.h`, the WebCore bindings code generator and the release tarball names did not change. #### Changes in Bun besides the version **BIGINT32 is gone upstream** ([`f59345e04a`](WebKit/WebKit@f59345e)). It was never enabled (`USE_BIGINT32` was `0`), so none of this changes behaviour, but Bun does not compile without it: - `JSBigInt::makeHeapBigIntOrBigInt32(g, v)` → `JSBigInt::createFrom(g, v)`, which is what it expanded to (`NodeSqlite.cpp`, `JSBuffer.cpp`, `JSCTestingHelpers.cpp`). Every call site passes an `int64_t` or a `uint64_t`, so the overload is exact. - `JSValue::isBigInt32()` always returned `false`. Removed: `JSC__JSValue__isBigInt32` and `JSValue::is_big_int32`, their uses in `toBeEven` / `toBeOdd`, an `ASSERT` in `JSC__JSValue__asBigIntCompare`, and the test bindgen emitted for a `bigint` that is not a cell. - `tryConvertToBigInt32(bigInt)` → `bigInt` in `SerializedScriptValue.cpp`. - The `#if USE(BIGINT32)` blocks in `bindings.cpp`, `SerializedScriptValue.cpp` and `napi.cpp`, which call functions that no longer exist. - The `JSGlobalObject*` of `JSC__JSValue__asBigIntCompare`, which only one of those blocks read, and with it the `global` of `JSValue::as_big_int_compare` and of the test runner's wrapper. **Loop headers have basic blocks of their own** (oven-sh/WebKit#880): - `JSInspectorProfiler.cpp` leaves out the blocks with `m_isInLoopHeader`. V8 has no range for them, and what reads `Profiler.takePreciseCoverage` is written for V8's ranges. - `source-positions-corpus.ts`: the count of nine loop header lines. `for (let i = 0; i < 3; i++) {` is 4, how often the condition ran, and was 1. Both are the same hunks as in oven-sh#46534, so they merge cleanly with it. `WEBKIT_VERSION` will conflict there: oven-sh#46534 pins `e450c494a84f`, which this contains. **`test/bundler/bundler_bytecode_portable.test.ts`**: the snapshot. Removing `op_is_big_int` renumbers the opcodes after it, so the serialized bytecode changed on every platform. 24 hashes and 13 sizes of `.jsc` payloads change, and no hash of the printed JavaScript. **`test/js/bun/jsc/webkit-upgrade-f59345e04a.test.ts`** (new): pins five of the engine changes below, like the files for the earlier syncs. #### Other upstream changes that reach Bun Checked against Bun's sources, no change needed: - `numberToShortestStringBufferSize` is 39, was 34. Bun passes the 124-byte `NumberToStringBuffer`. - `JSObject::forEachOwnIndexedProperty` lost its `JSGlobalObject*`, `immutableButterflyToStringCache` moved from `Heap` to `VM`, the inline parts of `HasOwnPropertyCache` moved to `HasOwnPropertyCacheInlines.h`, the `swapCodeRefForDebugger` family moved to `NativeExecutable`. Bun uses none of them. - `JSBoundFunction::create` gained an overload that takes a `SourceTaintedOrigin`. The one that takes a `SourceCode`, which Bun calls, is still there. - `SpecDataViewObject` is bit 49, was 50. Bun has no copy of `SpeculatedType`. - The bytecode cache key is a hash of the WebKit commit, so a cache written before the opcodes were renumbered is not read. Behaviour that changes with the engine: - In the head of a `for`: a destructuring declaration without an initializer is a SyntaxError, `let` is an identifier in sloppy code, and `for await (async of` is allowed ([`926bafc4e`](WebKit/WebKit@926bafc4e)). - An overwritten `length` of a sloppy arguments object is converted with `ToLength` ([`7b1cfc54c`](WebKit/WebKit@7b1cfc54c)). `-1` was 4294967295, so `Array.prototype.slice.call(arguments)` looped 2^32 times, and 2^32 and up were truncated. - `Stopwatch::reset()` clears past run intervals ([`49902180d`](WebKit/WebKit@49902180d)), and moving a `SamplingProfiler::StackTrace` keeps `stopwatchTimestamp` ([`b6c984205`](WebKit/WebKit@b6c984205)). - DFG fixes: a promise resolved with itself was fulfilled with itself in optimized code instead of rejected with a TypeError ([`b45f2446d`](WebKit/WebKit@b45f2446d)), `op_has_private_name` is not compiled as `InById` ([`dd24421c8`](WebKit/WebKit@dd24421c8)). - GC: leftover marks after the marking version wraps ([`e1a28d3cf`](WebKit/WebKit@e1a28d3cf)), a heap's Collector threads stop when it shuts down ([`eaf0efa57`](WebKit/WebKit@eaf0efa57)). ### How did you verify your code works? No local build. The first commit was built by CI ([#131689](https://buildkite.com/bun/bun/builds/131689)): all 13 `build-bun` jobs and the three `verify-baseline` jobs passed, so the aarch64 allowlist stays as it is, and the only test that failed was the bytecode snapshot, on all ten platforms that ran it. With the `bun-linux-x64` binary of that build, and the one of `main` at `a8fcd319` ([#131688](https://buildkite.com/bun/bun/builds/131688), WebKit `5e3554face13`) as the baseline: | | `main` | this PR | | --- | --- | --- | | `webkit-upgrade-f59345e04a.test.ts` | 0 of 5 pass | 5 of 5, also with `BUN_JSC_useJIT=0` | | `bundler_bytecode_portable.test.ts` | | 22 of 22 with the new snapshot, 21 of 22 before | | `source-positions.test.ts` | | 325 of 325 | | `inspector-profiler.test.ts` | | 52 of 52 | The snapshot was written by `--update-snapshots` on Linux x64. All 24 hashes are the ones in the CI logs of Windows x64, Linux aarch64 and Linux x64 ASAN, and the 11 that the annotation shows are the same on all ten platforms. The second commit's Rust was checked with `cargo check -p bun_jsc -p bun_runtime`; its C++ (one signature) is first compiled by CI. Before the first build, by reading: the diff of every JavaScriptCore, WTF and bmalloc header in the range, with each removed or changed name searched for in Bun, and every `JSBigInt::` member Bun uses looked up in the new `JSBigInt.h`. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…k.ts` (oven-sh#46641) ### What does this PR do? Moves the stream sinks to bindgen and deletes `src/codegen/generate-jssink.ts`. `src/runtime/webcore/Sink.bind.ts` declares four classes: | Class | What it is | | --- | --- | | `ArrayBufferSink` | `Bun.ArrayBufferSink` | | `FileSink` | `Bun.file(..).writer()`, `Bun.stdout.writer()`, `proc.stdin` | | `NetworkSink` | `s3file.writer()` | | `ReadableSinkController` | the `controller` of a `type: "direct"` stream that a native sink consumes, and what the pump of any other stream writes to | The old generator made a sink class, a constructor and a controller class for each of the seven sinks. Four of the sink classes were never instantiated, and the controllers differed only in which sink they forwarded to. There is now one controller class, as `ReadableStreamDirectController` in `bun-types` and the docs already describe it. It holds an enum of typed pointers to the six sinks, and each operation picks the sink with one `match` in `Sink.rs`. Bindgen parses the arguments: `write(chunk)` is a union of a string, a buffer source and anything else, `flush(wait)` a boolean, and `start(options)` a dictionary of the members that sink reads. `Bun.file(..).writer(options)` uses the same dictionary as `FileSink.start`. The logic that was generated C++ (attach, detach, `end`, `close(error)`, the closed and destination-closed states) is in `Sink.rs`, and `BunStreamSource.cpp` keeps the stream side. The two call each other through member functions of the generated class, which the schema declares, so neither side declares an `extern "C"` function for them. This is what `GeneratedSink.h` has for the controller: ```cpp // Defined in Rust. On exception, a JSValue result is empty. void close(JSC::JSGlobalObject*, JSC::JSValue error); JSC::JSValue end(JSC::JSGlobalObject*); JSC::JSValue write(JSC::JSGlobalObject*, JSC::JSValue data); bool isDetached(); void closeWithError(JSC::JSGlobalObject*, JSC::JSValue error); void startWithHighWaterMark(JSC::JSGlobalObject*, double highWaterMark); JSC::JSValue writeBytes(JSC::JSGlobalObject*, std::span<const uint8_t> bytes); // Defined in hand-written C++, called from Rust. JSC::JSValue assignToStream(JSC::JSGlobalObject*, JSC::JSValue stream); void onClose(JSC::JSGlobalObject*, JSC::JSValue reason, bool sinkClosed); void onReady(JSC::JSGlobalObject*); void setPipe(JSC::JSValue stream, JSC::JSValue done); JSC::JSValue pipeStream(); void clearPipeStream(); void setPipeDone(JSC::JSValue done); JSC::JSValue pipeDone(); JSC::JSValue takePipeDone(); void setFailReason(JSC::JSValue reason); JSC::WriteBarrier<JSC::JSCell> m_source; JSC::WriteBarrier<JSC::JSPromise> m_closePromise; JSC::WriteBarrier<JSC::Unknown> m_failReason; JSC::WriteBarrier<WebCore::JSReadableStream> m_readableStream; JSC::WriteBarrier<WebCore::JSReadableStream> m_pipeStream; JSC::WriteBarrier<JSC::JSPromise> m_pipeDone; ``` The pump now holds a `JSReadableSinkController*` and calls `write`, `end` and `close` directly. Before, it looked them up by name and built an options object for `start`. They are the same functions JS calls, so arguments convert the same way. Also gone: `Sink.h`, `generated_jssink.rs`, twelve lazy members of `ZigGlobalObject`, seven `DOMStructureSlot` entries, six iso subspace slots, the sink block of `headers.h`, `GlobalObject::assignToStream`, `host_fn_static` and `StartTag`. `RequestContext` holds its `HTTPServerWritable` directly, and `JSTransformStream` holds the controller in one `m_nativeSinkController` in place of a cell, a pointer and an id. The sinks were the last Rust code to read a `CallFrame`, so these have no users and are deleted: - `CallFrame::{arguments, argument, arguments_count, this}` and the copy of JSC's register layout that they decoded - the `CallFrame` parameter of `js_thread_of_caller` and `context_of_caller`, which leaves one function of each name - `JSGlobalObject::throw_illegal_constructor`, three `BuiltinName`s, three methods of `TaggedPtrUnion` and the tag bounds of `TypeList` - the `flush()` of four sinks, which only the old trait called #### Bindgen additions | Option | What it does | | --- | --- | | `writable: false` on a class | its native functions are read-only | | `{ constant: 3 }` in `proto` | a read-only integer, like a WebIDL `const` | | `enumerable` on a getter or a constant | overrides the class's, as `configurable` does | | `calledFromCpp: true` on a function of `proto` | also a member function of the generated C++ class: it takes a `JSValue` for each argument and converts it as a call from JS does | | `calledFromCpp: { .. }` on a class | member functions defined in Rust that are not properties; they take and return `JSValue`, `bool`, numbers and `b.byteSpan` as they are | | `calledFromRust: { .. }` on a class | member functions that hand-written C++ defines and Rust calls as `js_Name::x(this_value, ..)` | | `b.byteSpan` | bytes that C++ passes to Rust: `std::span<const uint8_t>`, `&[u8]` | | `internalFields: { done: b.cell("JSC::JSPromise", header) }` | a field for hand-written C++, a `WriteBarrier<JSC::JSPromise>`; fields declared as a list stay `WriteBarrier<Unknown>` | | `b.string.view` | the characters of a string where they are, as a `bun_core::StringView<'a>`: no reference is taken and a slice is not copied | | `.falsyAsNull` on `b.any` | a falsy value is `None` | ### Behaviour A script calls every method of an `ArrayBufferSink`, a `FileSink` and a server's controller with 33 kinds of argument, with no argument, with five kinds of wrong `this`, and after `close()` and `end()`. It also covers each member of the `start` and `writer` options, the constructors and every property descriptor. It ran on main and on this branch; these are the cases that differ. **Wrong on main, fixed here** | Case | main | This PR | | --- | --- | --- | | `Bun.ArrayBufferSink.prototype` | `undefined` | the prototype | | `sink instanceof Bun.ArrayBufferSink` | throws `instanceof called on an object with an invalid prototype property.` | `true` | | `class X extends Bun.ArrayBufferSink {}` | throws `The value of the superclass's prototype property is not an object or null.` | works, and `Reflect.construct` honours `newTarget` | **Different, neither wrong** | Case | main | This PR | | --- | --- | --- | | The controller's label in `console.log`, `Object.prototype.toString` and `heapStats()` | `ReadableHTTPResponseSinkController` and five others | `ReadableSinkController` | | `sinkId` of a controller | a data property of each prototype | an accessor, with the same value and attributes; on the prototype itself it is `undefined` | | A sink's method with a wrong `this` | `Expected FileSink`; no `code` from `close`, `_getFd` and a controller's `end` | `Expected this to be instanceof FileSink, but received ..`, always with `code: "ERR_INVALID_THIS"` | | `FileSink()` without `new` | `Illegal constructor` | `FileSink constructor cannot be invoked without 'new'` | | `start(options)` of an `ArrayBufferSink` | `asUint8Array` counts only if it is an own property | it counts inherited too, like `stream` and `highWaterMark` | | `start(options)` of a controller | reads `path` and `fd` for a `FileSink`, and `highWaterMark` for any other sink | reads all three, and uses the ones main uses; a getter that throws on one it does not use makes `start` throw | | `start({ path, fd })` and `writer({ path, fd })` | `fd` is read only if `path` is absent | both are read, and `path` still wins | | `write()` with no argument | `ERR_MISSING_ARGS`, `write() expects a string, ArrayBufferView, or ArrayBuffer`; a closed sink says that it is closed, and a controller whose client went away returns 0 | `ERR_MISSING_ARGS`, `The "chunk" argument must be specified` (`"data"` on a controller), whatever the state | | `estimateShallowMemoryUsageOf(new Bun.ArrayBufferSink())` | 96 | 80 | | `start(options)` with a wrong `this` | runs the getters of `options`, then throws `ERR_INVALID_THIS` | throws `ERR_INVALID_THIS` first | | `write(new String(..))` on a closed sink | throws before it converts the chunk | converts the chunk, then throws the same error | | A method borrowed across classes | a sink and its controller take each other's `write`, `flush` and `start`, and a controller of another sink does not | a sink's methods work on that sink only, and a controller's on every controller | **Right on main, wrong here**: none. `Sink.rs` discarded the result of detaching a controller. It goes to `dispatch::fold` now, like the same call in `FileSink.rs` and `streams.rs`, and the `jsresult-swallow` inventory loses the entry. ### Cost Release builds of main and of this branch, one pinned core, cycles for each operation. The third column is a second copy of main's binary. | Operation | main | main again | This PR | | --- | --- | --- | --- | | a direct stream to a file | 4594 | 4571 | 4028 | | `Bun.serve` with a stream body | 12487 | 12537 | 11547 | | `await Bun.write(path, new Response(stream))` | 8412 | 8439 | 7521 | | `arrayBufferSink.start({ highWaterMark })` | 243 | 246 | 102 | | `arrayBufferSink.flush()` | 369 | 369 | 321 | | `arrayBufferSink.write("hello world")` | 106 | 107 | 89 | These cost more: | Call | main | main again | This PR | | --- | --- | --- | --- | | `arrayBufferSink.write(uint8Array)` | 76 | 73 | 78 | | `arrayBufferSink.write(arrayBuffer)` | 90 | 88 | 94 | | `fileSink.write(uint8Array)` | 101 | 100 | 111 | | `fileSink.flush()` | 62 | 59 | 69 | | `controller.write(uint8Array)` | 80 | 82 | 102 | | `controller.flush()` | 39 | 40 | 51 | On main `write` and `flush` are each one Rust function that JSC calls and that reads its own argument. Now a generated C++ function checks `this` and converts the argument, and then calls the Rust function. A profile of `controller.write(uint8Array)` shows nothing else: three frames where main has one, and a call back into C++ for the pointer and length of the buffer, which `b.bufferSource` reads when the bytes are used and not when the argument is converted. The other calls of the three classes (`write` of a string to a `FileSink` or a controller, `start()`, `end()`, `ref()`, `unref()`, `_getFd()`, `sinkId`, `new Bun.ArrayBufferSink()`, `Bun.file(..).writer()`) measure within the difference between the two copies of main, or faster. The GC calls Rust when it sweeps a controller. On main the destructor of a detached controller made no call. `JsLocal<BufferSource>::with_bytes` gets the bytes from `Bun__Bindgen__bytesOfBlobOrBuffer`, as a `b.bytes` handle does, in place of `JSC__JSValue__asArrayBuffer`, which looked at the type again and filled a struct. The release binary is 86,016 bytes smaller than main's. ### How did you verify your code works? - `test/js/bun/util/arraybuffersink.test.ts` and `test/js/bun/http/serve-direct-readable-stream.test.ts` each have a new test that fails on main. - `test/internal/bindgen.test.ts` covers each bindgen addition, and each mistake in a schema that one of them reports. `TestMemberFunctions` goes from JS to Rust to C++ (`BindgenTest.cpp`, only in a build with assertions) and back to Rust, and keeps a `JSPromise` in a typed field across a GC. - `test/integration/bun-types/bun-types.test.ts` compares the schema with `Bun.ArrayBufferSink`, `Bun.FileSink`, `Bun.NetworkSink` and `ReadableStreamDirectController`. Its list of known differences gains the members that `bun-types` does not declare (`close`, `_getFd`, `sinkId`, and `ref` and `unref` of an `ArrayBufferSink`), `NetworkSink.stat()`, which `bun-types` declares and no sink has, and the `highWaterMark` of `FileSink.start(options)`, which `bun-types` declares and a `FileSink` does not read, on main as well. - These pass on a debug build: `bindgen`, `arraybuffersink`, `filesink`, `serve-direct-readable-stream`, `direct-readable-stream`, `async-iterator-stream`, `serve-stream-body-error`, `s3-upload-stream-gc`, `s3-stream-error-gc`, `spawn-stdin-destroy`, `spawn-stdin-readable-stream`, `text-encoder-stream`, `compression`, `body-stream`, `fetch-abort-stream-body`, `html-rewriter`, `html-rewriter-end-error`, `node-tls-upgrade`. - `bindgen`, `arraybuffersink`, `filesink`, `serve-direct-readable-stream`, `serve-stream-body-error`, `s3-stream-error-gc`, `compression` and `text-encoder-stream` also pass with `BUN_JSC_validateExceptionChecks=1`. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…ven-sh#46718) - `host.sh` installs the pinned Bun when `/usr/local/bin/bun` reports a different version. - `joinTailnet` returns when `tailscale status` already succeeds, before `install-system-daemon`. - `guest/bake.sh` runs `brew update` before the toolchain script. - The README says the scripts run on the Bun `host.sh` pinned when it last ran on the host.
…#46505) ### Problem - `console.groupCollapsed()` does not start a group. Later lines are not indented, and its `console.groupEnd()` removes the indent of an enclosing `console.group()`. A call with no label writes `undefined`. - Cause: `message_with_type_and_level_` (`src/jsc/ConsoleObject.rs:383`, `:394`) tests `MessageType::StartGroup` only. JSC sends `StartGroupCollapsed`. ### Fix - One `is_start_group` flag covers both types. The `undefined` fallback is removed: a collapsed group with no label was its only caller. The `bun-types` JSDoc is updated. - Correct: Node documents `groupCollapsed` as [an alias of `group`](https://nodejs.org/docs/latest-v26.x/api/console.html#consolegroupcollapsed), and the [WHATWG spec](https://console.spec.whatwg.org/#groupcollapsed) pushes both onto one group stack. - Verified: `test/js/web/console/console-log.test.ts`, `test/js/bun/test/dots.test.ts` (new tests fail on bun 1.4.3-canary). Also ran: console test directories, bun-types test. - Self-reviewed: 15 concerns raised, 15 addressed. Notes names two follow-ups that are not in this PR. ### Background - JSC passes each console call, with a `MessageType`, to `Bun::ConsoleObject::messageWithTypeAndLevel`. It forwards the message to the inspector, then to the Rust printer, which keeps the group depth in `default_indent`. - Considered a JS alias of `group` and a type mapping in the C++ client. The alias hides the collapsed type from the inspector, which can fold a group. The mapping splits one printer rule across two files. <details><summary>Notes</summary> Repro: ```js console.group("outer"); console.groupCollapsed("inner"); console.log("x"); console.groupEnd(); console.log("still in outer"); console.groupEnd(); console.log("after"); ``` | bun 1.4.3-canary (367d939) | this PR | Node v26.3.0 | |---|---|---| | `outer` | `outer` | `outer` | | `··inner` | `··inner` | `··inner` | | `··x` | `····x` | `····x` | | `still·in·outer` | `··still·in·outer` | `··still·in·outer` | | `after` | `after` | `after` | (`·` is a space.) **History** - There is no user report for this bug. A read of the code found it. It dates from oven-sh#15026, which added `console.group` for oven-sh#8099 and handled `StartGroup` only. - This is a refile of oven-sh#34237, which had the same two conditions. The stale action closed it after 90 days with no human review. - oven-sh#39592 moves `node:worker_threads` workers to the native console, and its branch has the `StartGroup`-only checks. Those workers print Node's output for `groupCollapsed` today. They keep it if this change lands first, or if oven-sh#39592 is rebased over it. The two diffs do not overlap. - oven-sh#42163 mirrors a page's `groupCollapsed` through `ConsoleClient::group()`. It does not need this change. **Scope** - The same code path serves `console.warn` and `console.error` inside a collapsed group. It also serves the console of a web `Worker`. Both are corrected. - Not affected: `new console.Console()` and `node:worker_threads` workers. They use the JS `Console` class, where `groupCollapsed` is already an alias of `group` (`src/js/builtins/ConsoleObject.ts:780`). Node does the same ([`constructor.js:681`](https://github.com/nodejs/node/blob/v26.3.0/lib/internal/console/constructor.js#L681)). - An outside caller: `Console.withGroup({ collapsed: true })` of `effect` 3.22.3 calls `console.groupCollapsed` on the global console (`src/internal/defaultServices/console.ts:57`). Nested in another group, it loses the outer indent on main and prints Node's output with this change. - Not changed: some output stays at column 0 inside a group of either kind. That is `console.count`, `console.timeLog`, `console.timeEnd`, `console.table`, and `console.assert` with no message. The later lines of a multi-line string stay there too. Related: oven-sh#30017, oven-sh#36470. The new JSDoc names only methods that do indent. **The removed fallback** - It was `else if message_type != MessageType::Trace { writer.write_all(b"undefined\n") }`, with no flush and no indent. On main, the text for a collapsed group with no label waits in the buffer. It shows up with the next `console.*` line on stdout, or not at all. - To reach it, a message needs no arguments, a type other than `Log` or `Trace`, and no earlier return. JSC drops `console.dir()`, `console.dirxml()` and `console.table()` with no arguments (`ArgumentRequirement::Yes` in `ConsoleClient.cpp`). `EndGroup`, `Clear`, `Assert` with no arguments, and now both group types return earlier. JSC sends no other type through this function. - Checked with a run of 19 console methods with no arguments. Main prints `undefined` for `groupCollapsed` only. This PR prints it for none. - Without the fallback, the early return for a group with no label is visible only under `bun test`. There it skips the before-print hook of the reporter. On main, a collapsed group with no label breaks the line of dots of `bun test --dots`. The new case in `dots.test.ts` pins the return. With the return put back to `StartGroup` only, that case fails and the case in `console-log.test.ts` passes. **Follow-ups that are not in this PR** - Node prints `undefined` for `console.dir()` and `console.table()` with no arguments, and an empty line for `console.dirxml()`. Bun prints nothing for the three on the global console. - `bun test --isolate` does not reset the group depth, the `console.count` totals or the `console.time` timers between test files. **Cost** - On a reduced copy of the two conditions (rustc 1.100.0-nightly, `opt-level=3`), `cmp; sete` becomes `add; cmp; setb`. No allocation, write or syscall is added. **Suites run with the debug build** - `test/js/web/console/`, `test/js/node/console/`, `test/js/bun/console/`, `test/js/bun/test/dots.test.ts`, `test/js/bun/test/only-failures.test.ts`: 118 pass, 0 fail. - `test-console-group.js`, `test-console-methods.js` and `test-console-with-frozen-intrinsics.js` from Node's suite: exit 0. - `test/integration/bun-types/bun-types.test.ts`: 24 pass. </details> ### Downsides - Output changes for callers of `console.groupCollapsed`: later lines gain two spaces for each open group. Snapshots of that output change. - Under `bun test`, a collapsed group that one file leaves open now indents the output of later files. An open `console.group()` does this today. - One more instruction per console message (a range check replaces an equality check).
…46439) ### What does this PR do? Four things for `bun lint`. #### `bun lint` is a command in release builds This reverts oven-sh#46438, which kept `bun lint` out of release builds. Bun v1.4.4 was released with it, so the next release is the first one that has `bun lint`. | in a release build | before | after | |---|---|---| | `bun lint` | the project's own `lint` script, file or executable, else `error: Script not found "lint"` | the linter, unless the project has its own `lint` | | `bun --help` | does not list `lint` | lists `lint` | | README from `bun init` | "Run `bun fmt` to format the code." | "Run `bun lint` to find problems in the code and `bun fmt` to format it." | | test runner | skips `test/cli/lint/` and the cases of `init.test.ts`, `bunfig-errors.test.ts` and `glob/match.test.ts` that run `bun lint` | runs them | | order file generator | does not trace `bun lint` | traces it | `docs/lint/index.mdx` loses the note that the command is in canary builds only. Nothing changes in a canary build. #### `require.resolve()` in an installed plugin adds only Node's extensions eslint-plugin-import-x looks for its resolver named "node" with `require.resolve(<directory of the nearest package.json> + "/node")` in a try / catch. In Node.js that throws, and the plugin goes on with its built-in resolver. Bun also adds `.ts`, so in a package that has a `node/index.ts` the plugin loaded that file as its resolver, which ran the project's code, and every file of the package got a false error: ``` 1:1 error Resolve error: node with invalid interface loaded as resolver import/order ``` | `require.resolve(path.resolve("node"))` in `node_modules/eslint-plugin-demo/index.js`, in a project with `node/index.ts` | | |---|---| | ESLint 10.12.0 on Node.js 26 | throws `MODULE_NOT_FOUND` | | `bun lint` before | `<project>/node/index.ts` | | `bun lint` now | throws `MODULE_NOT_FOUND` | ESLint runs plugins in Node.js. So `require()` and `require.resolve()` in a file under `node_modules` now add `.js`, `.json` and `.node` where they added Bun's default list, in the threads that run the rules and in the process that reads a configuration for `bun lint` and `bun fmt`. An extension that a package adds to `require.extensions` still counts. Nothing else changes: not `bun run`, not `import`, not the project's own files. In directus, 103 such errors are gone, and the output is ESLint's line by line. The output for Leaflet, jotai and vite is the same as before. #### `-A`, `-W` and `-D` reach a rule that's configured under two names With an `.oxlintrc.json`, `no-array-constructor`, `eslint/no-array-constructor` and `typescript/no-array-constructor` are one rule. If the file and the files it extends used two of these names, the flags missed the rule, so a run with `-D` passed that should fail. ```jsonc // .oxlintrc.json { "extends": ["./base.json"] } // base.json { "rules": { "typescript/no-array-constructor": "warn", "no-array-constructor": "warn" } } ``` | `export const a = new Array(1, 2, 3);` | oxlint 1.87.0 | `bun lint` before | `bun lint` now | |---|---|---|---| | `-D no-array-constructor` | error, exit 1 | warning, exit 0 | error, exit 1 | | `-A no-array-constructor` | nothing, exit 0 | warning, exit 0 | nothing, exit 0 | | both names set to `"error"`, `-W no-array-constructor` | warning, exit 0 | error, exit 1 | warning, exit 0 | | both names set to `"error"`, `-A no-array-constructor` | nothing, exit 0 | error, exit 1 | nothing, exit 0 | The same happened with one name in the file and the other in a file it extends, with the two names in two extended files, and with `-A all` or a category in place of the rule. With two extended files that give the two names different severities, a flag for another rule, like `-D eqeqeq`, changed which of the two applied. oxlint merges files by the names as they're written, and applies the flags to the rule afterwards. We copy the inherited rules into the file's own `rules` and change them there, and that copy skipped a rule that was already there under another name. Now each written name is copied. | compared with oxlint 1.87.0 | results | differed before | differ now | |---|---|---|---| | 194 configurations with two or three names of one rule, 13 sets of flags each | 2,522 | 443 | 0 | | 210 configurations where the two names have different options, 7 sets of flags each | 1,470 | 344 | 0 | Without flags, none differed before. #### Shorter hints Shortens the hints of `bun lint`: the `note:` line that `-f pretty` and `-f agent` print under a report whose message doesn't say why or what to do. They used more words than they needed, and often began with a vague "It is". ``` 1 | export function price(items: number[]) { 2 | let total = 0; ^ warn: The value assigned to 'total' is not used in subsequent statements. no-useless-assignment at src/price.ts:2:7 note: Reassigned on line 3 before it's read. ``` | rule | before | after | |---|---|---| | `no-useless-assignment` | It is overwritten on line 3 before it is read. | Reassigned on line 3 before it's read. | | `react-hooks/rules-of-hooks` | The return on line 4 can end the render before it is called. | The return on line 4 can skip this hook. | | `react-hooks/rules-of-hooks` | Whether it is called depends on the condition on line 9. | Depends on the condition on line 9. | | `no-useless-catch` | It only throws the error again. Without try and catch, the code does the same. | It only rethrows. Remove the try and catch. | | `no-dupe-keys` | Only the last one counts. The first is on line 2. | The last one wins. The first is on line 2. | | `no-compare-neg-zero` | -0 and 0 compare as equal. Use Object.is(x, -0) to tell them apart. | -0 === 0 is true. Use Object.is(x, -0). | | `no-compare-neg-zero`, `x < -0` | -0 and 0 compare as equal. Use Object.is(x, -0) to tell them apart. | -0 compares like 0. Write 0. | | `no-empty-character-class` | [] in a regular expression matches nothing. | [] matches nothing. | | `no-fallthrough` | Add break, or the comment // falls through if this is meant. | Add break, or a // falls through comment. | | `no-this-alias` | Use an arrow function instead. It has the this of the code around it. | Use an arrow function. It keeps the outer this. | | `no-explicit-any` | unknown is the safe choice where the type is not known. | Use unknown if the type isn't known. | | `no-unused-vars` | Remove it, or start its name with _ to keep it. | Remove it, or prefix it with _. | | `no-unused-vars` | A catch needs no parameter. Remove it and its parentheses. | The parameter is optional. Write catch { }. | | `no-const-assign` | It is declared with const on line 1. Use let if it has to change. | Declared with const on line 1. Use let if it changes. | | `no-octal` | The leading 0 makes it octal, so 071 is 57. Write 0o71 to make that clear. | The leading 0 makes it octal: 071 is 57. Write 0o71. | 51 hints of 36 rules change this way. Two lines are now listed as "lines 5 and 7", not "line 5 and line 7". The hint of `no-compare-neg-zero` now follows the operator: `Object.is` was the wrong advice for `<`, `<=`, `>` and `>=`, and `!=` and `!==` need `!Object.is`. Nothing else changes: the messages themselves are upstream's, and which reports get a hint stays the same. ### How did you verify your code works? The revert: it applies without conflicts, each of its 14 files is byte for byte what it was before oven-sh#46438, and nothing that `main` got since uses what it removes (`isCanary`, `IS_ENABLED`, the `STABLE` modifier). PR CI builds are canary, so CI does not cover the release column. No release build was made of it. The plugin fix: a new test in `test/cli/lint/js-plugins.test.ts` with a stand-in plugin, at the time the configuration is read and at the time a rule runs. It fails without the fix, and what it expects is what ESLint 10.12.0 prints. Not run on macOS or Windows. The hints: `test/cli/lint/hints.test.ts` has a case for each hint and expects the new text. It passes, as do `output.test.ts` and `git.test.ts`, which also quote a hint. Two tests that check that a format prints no hint looked for a word of the old text, which would have passed whatever was printed. They look for the new text now.
… files without an ESLint or oxlint config (oven-sh#46980) Fixes oven-sh#46971 ### Problem - With plugins loaded from `bunfig.toml` alone, a JS plugin rule sees `node.start` and `node.end` in `.js` files but not in `.ts` files. There, nodes, tokens, and comments have only `range` and `loc`. A rule that compares `comment.end <= node.start` evaluates `undefined <= undefined` and silently misfires. - `LintNodeState::kindFor` (`src/jsc/bindings/LintNodes.h`) picked typescript-estree's layout for every non-espree, non-oxlint file. With an ESLint config that is right. Without one there is no parser to match, and the run mixes two shapes. ### Fix - One function, `estree::has_start_end(file)`, says whether a file's nodes have `start` and `end`: espree's and oxlint's do, and typescript-estree's do when `LanguageOptions::has_no_foreign_config` is set. The selector evaluator (`selector/value.rs`) and the plugin host (`host.rs`, as a new flag bit) both ask it. - A fifth node layout in `src/lint/js_plugin/layouts.rs`, `TypeScriptWithStartEnd`: typescript-estree's properties with `start` and `end` right before `range`. `kindFor` picks it when `LintHost::hasStartEnd()` is set for a file that is neither espree's nor oxlint's. - Tokens and comments of those files get the same two properties through two new `LintTokenShape` kinds. The nodes that `worker/ast.js` builds for typescript-estree's deprecated properties follow the node they are built for. - The ESLint-config and oxlint-config shapes are unchanged. Their oracle tests still pass. - Verified: `test/cli/lint/js-plugins.test.ts`, two new tests (the tree oracle under `bunfig.toml`, and keys and offsets of a node, a type annotation, both comment kinds, a token, and a regex token in `.ts` and `.js`, plus a `[start=13][end>13]` selector). Also the whole file (148 pass), `js-parsers.test.ts`, `bunfig.test.ts`, `rule-throws.test.ts`. ### Background - `layouts.rs` is the one table that says which properties a node of each type has, in which order, per parser. Rust sends it to the plugin worker once. `LintNodes.cpp` builds a JSC Structure per node type from it, so every node of a file passes through it. `OxlintJavaScript` is an earlier variant of the same kind. - `has_no_foreign_config` is the linter's existing flag for "no ESLint or oxlint config, so no installed tool's behaviour has to be matched". The linter already uses it to clamp report locations and to remove whole lines in fixes. - Considered adding `start` and `end` to the TypeScript layout for everyone: plugins that spread nodes or compare keys would see a shape that typescript-estree does not produce, and the ESLint oracle test pins that shape. Considered oxlint's layout for every file under `bunfig.toml`: it changes `.js` files too (oxlint field order, `null` for absent fields, inherited `loc`, oxlint scopes and tokens), while the documented use case is ESLint plugins from npm. - `.oxlintrc.json` is not a substitute: it flips `has_no_foreign_config` and the other behaviours keyed on it. ### Downsides - Each plugin worker thread loads the layouts table once. It grows from 71,530 to 98,250 bytes (+26,720). Templates and Structures are made per node type on first use, so files that never hit the new layout make none. - A plugin that detects the parser by `"start" in node` sees typescript-estree's nodes with espree's offsets under `bunfig.toml` alone. No such check was found in the test suite or in the plugins that the docs name. - What stays typescript-estree's in this mode: an absent field is `undefined`, not `null` as in oxlint. A rule written for oxlint that checks `=== null` still differs. The docs bullet says so. <details><summary>Notes</summary> - Self-reviewed: 4 concerns raised, 4 addressed (docs wording, the synthesized nodes in `worker/ast.js`, a regex token shape without a test, the tree oracle run in the new mode). Review on the PR found two more, both addressed: selectors with `[start]` or `[end]` evaluated natively against the old rule, now through the same `has_start_end`; the synthesized nodes read a per-file global, now they look at the node they extend. - The tree oracle under `bunfig.toml` matches the ESLint oracle with `start: a, end: b,` inserted before every `range: [a, b]` of the non-`.js` files, including the nodes behind `TSMappedType.typeParameter` and `TSImportType.argument`. The only other difference is `sourceType`, which Bun derives from the file where ESLint's config sets it, so the test masks it. - Layout kinds in C++ now have `isTypeScript` and `isOxlint` predicates in place of `==` and `>=` comparisons, so a new kind at the end cannot fall into the wrong branch. - The `visitorKeys` numbering (0 typescript-estree, 1 espree, 2 oxlint) is unchanged. The new kind maps to typescript-estree's keys. </details>
…#46486) ### What does this PR do? Follow-up to oven-sh#46468. It fixes the three cases that the reviews of that pull request left open, and three more that the reviews of this one found. All are in `find_in_project`, on macOS and Linux. A native dependency is a package in `nativeDependencies` of the project's package.json, or in the built-in list when the project has no such field. For those, `bun install` links `node_modules/.bin/<name>` into the platform package: a dependency with an `os` and a `cpu`. | # | Case | Own bin file | Before | After | |---|---|---|---|---| | 1 | A folder above the project has the name of an optional dependency that comes before the platform package | `0644` | `is not executable`, exit code 126 | platform binary | | 1 | the same | `0755` | stub | platform binary | | 2 | The package is not a native dependency. `.bin/<name>` is the bin of one of its optional dependencies, which has an `os` and a `cpu` | `0755` | bin of the dependency | own file | | 3 | `.bin/<name>` is the bin of an optional dependency without `os` and `cpu` | missing or not executable | `is not executable`, exit code 126 | `.bin/<name>`, as before oven-sh#46468 | | 4 | The platform package of a native dependency is in `dependencies` | `0644` | `is not executable`, exit code 126 | platform binary | With case 3 alone, `.bin/<name>` of an unrelated package would also run in two layouts. Both keep the error of `main`: | # | Layout | Own bin file | Result | |---|---|---|---| | 5 | A folder above the project has the name of an optional dependency, and a package.json | not executable | `is not executable`, exit code 126 | | 6 | `.bin/<name>` is in `@scope/first`, and the package has the optional dependency `first` | not executable | `is not executable`, exit code 126 | | # | Change | |---|---| | 1 | `dependencies_with` returns every dependency whose name is in the real path of the link, not only the first | | 2 | `is_in_platform_package` also asks `List::is_native_dependency`. That reads the project's package.json with the parser of the installer. It looks up the `name` in the package's own package.json, so an alias works | | 3 | `find_without_executable_file` accepts a link into any optional dependency again | | 4 | `is_in_platform_package` looks in `dependencies` and `optionalDependencies`. The installer has no filter on the kind | | 5 | `dependencies_with` skips a folder that the package itself is in | | 6 | `dependencies_with` skips a folder whose parent begins with `@` | ### How did you verify your code works? - The tests in `bunx.test.ts` whose names begin with `after an install, a package whose optional dependency`, `a package that cannot run` and `the native dependency`. I ran each on a build without its fix first, and each failed there. All pass now. - `after an install, esbuild in a project with nativeDependencies <undefined | []>` covers the built-in list. It passed before too. - The rest of `bunx.test.ts` and `bun-install-native-binlink.test.ts` pass on macOS arm64, but for `should handle postinstall scripts correctly with symlinked bunx`. My machine does not let a copy of bun run from the temp folder. That test asks for `esbuild@latest`, and a request with a version does not reach this code. Linux is left to CI.
Fixes oven-sh#47006 ### Problem - `bun install` writes `[0.02ms] ".env"` to stderr whenever a `.env` file exists. `--silent` and `--quiet` do not remove it. Every install-family command (`add`, `remove`, `update`, `bunx`, ...) prints it. - `PackageManager::init` allocates its dotenv loader with `Loader::init()` (`src/install/PackageManager.rs:2384`) and never sets `quiet`. `Loader::load` (`src/dotenv/env_loader.rs:660`) prints the line unless `quiet` is set. ### Fix - The loader gets a `quiet_loaded_files` flag that hides only the `[Nms] ".env"` line. The `quiet` flag keeps its meaning: it also hides the `error loading .env file` messages (`env_loader.rs:816`, `:835`, `:879`), which `bun run` relies on. - Install sets `env.quiet_loaded_files = !cli.log_level.is_verbose()` right after the loader is allocated. This is the gate install already uses for its other `print_elapsed` timing traces. The line shows with `--verbose` (and under `RUNNER_DEBUG`, which promotes the level). - The `.env` files are still loaded, and an unreadable `.env` still prints `EACCES error loading .env file` at every level, as before. - Verified: `test/cli/install/bun-install-env-file.test.ts` (46 pass, 1 skip as root. The three new "not printed" cases fail on bun 1.4.3). Also `test/cli/run/env.test.ts` and `test/cli/run/log-test.test.ts`. ### Background - `bun run` and the transpiler already set `quiet` on their loaders: `run_command.rs:660` forces it, `transpiler.rs:1175` derives it from the log level (default `Warn`, so hidden). Install was the only `Loader::load` caller that left the line on. - Considered setting `quiet` on the install loader, as `bun run` does. That also hides the read error for a `.env` that carries the registry token, so a failed auth would have no explanation. The separate flag keeps that message. - Considered flipping the `Loader::init()` default to hidden. It changes nothing for any live loader today (every other `init()` user calls only `load_process`, and the transpiler overwrites the field) and still needs the install line. ### Downsides - A user who read the `[Nms] ".env"` line from the default `bun install` output must pass `--verbose` now. stderr lines for `bun install` with a `.env` in an empty project: 2 to 1 (default), 1 to 0 (`--silent`), 3 to 3 (`--verbose`). - Cost for other callers: one `bool` per `Loader` and one extra compare per `load`, no allocation or syscall. Binary size and instruction deltas were not measured: `bloaty`, `perf` and `strace` are not installed in this environment. <details><summary>Notes</summary> - Repro: `package.json` + `.env` (`FOO=bar`), then `bun install --silent`. Before: stderr is `[0.18ms] ".env"`. After: stderr is empty. - `cli.log_level` is the value `Options::load` later copies into `options.log_level` (`PackageManagerOptions.rs:1143`), so the bunfig and `.npmrc` paths cannot diverge from it. - A second `PackageManager::init` in one process (the `link` retry) reuses the loader and assigns the flag again. - The EACCES test is skipped as root. Checked by hand as `nobody` with a mode 000 `.env`: default and `--silent` both print `EACCES error loading .env file`, as on main. - The first revision set `quiet` on the install loader. Review pointed out that this also hid the read errors, so the loader got the separate flag. - The tests that assert the line is absent after `--no-env-file` or `env = false` pass `--verbose` now, so the assertion can still fail. - Debug binary `size` (PR build): text 643722302, data 65837697, bss 12169944. </details> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…r `Source`, `Path`, `ImportRecord` and resolve results (oven-sh#46834) ### What does this PR do? Fixes a heap-use-after-free in `BuildMessage.position.file`, by removing the false `'static` lifetimes that let it compile. `Source`, `Path`, `ImportRecord`, the bundler's graph types and the resolver's results now carry the lifetime of what they borrow, so the compiler checks this whole area instead of comments. #### The bug ```js const url = URL.createObjectURL(new File(["["], "bad.yaml")); const wide = ("Ā" + url).slice(1); // any specifier that is not 8-bit ASCII await import(wide).catch(e => console.log(e.position.file)); // reads freed memory ``` `Location::init_or_null_impl` stored `Cow::Borrowed(source.path.text)` in a `Cow<'static, [u8]>`. That compiled because `Source.path` was a `Path<'static>`. For a virtual source the path is really a view of `specifier.to_utf8()`, which `Bun__transpileFile` / `Bun__transpileVirtualModule` free when they return, while `process_fetch_log` moves the message into a `BuildMessage` that JS keeps. There are three ways in: a `blob:` URL of an in-memory `File`, `/bun-vfs$$/node_modules/<mod>/x.json5`, and a plugin's `onLoad` answer. No script has to read `.position`: printing an uncaught error reads `location.file` too. It reproduces on 1.4.4 release builds, where the first 8 bytes of the string come back as allocator metadata. #### The cause is a class, not a line `Source`, `ImportRecord`, `Chunk`, `ParseTask` and the resolver's `Result` all said `'static` for bytes they only borrow, and **safe** functions produced those `'static`s from any `&[u8]` (`IntoStr::into_str`, `path_as_static`, `LinkerContext::get_source`, `scoped_action`, the resolver's entry points). The borrow checker could see none of it, so correctness lived in comments, defensive copies and process-lifetime interners. #### What changes **Diagnostics own their bytes, by type.** `Location { file, namespace: Box<[u8]>, line_text: Option<Box<[u8]>> }`; `Clone` is derived on `Location` / `Data` / `Msg`. A `Log` may outlive every `Source` it reports on. This is what fixes the reported bug: `let file = Box::from(source.path.text)` in `init_or_null_impl`. With that, the machinery for deep-copying logs at every hand-over has no job left: `Log::append_to` moves the messages, and `clone_line_text`, `Log::dupe` / `owned_strings`, `clone_with_builder`, `count`, the `StringBuilder` stub and the three `*_with_recycled` copies are deleted. **Everything else borrows, with the real lifetime.** Lifetimes are erased at compile time: no reference counts, no new fields, no size change. | type | now | | --- | --- | | `bun_ast::Source` | `Source<'a>` (`path: Path<'a>`, `contents` / `identifier_name: Cow<'a, [u8]>`) | | `bun_ast::ImportRecord` | `ImportRecord<'a>` | | bundler | `InputFile<'a>`, `Chunk<'a>`, `CssImportOrder<'a>`, `ParseTask<'a>`, `parse_task::Result<'a>`, `ServerComponentParseTask<'a>`, `ResolveQueue<'a>` | | resolver | `Result<'r>`, `PathPair<'r>`, `ResultUnion<'r>`; `resolve<'r>(.., import_path: &'r [u8], ..) -> Result<'r>` | | module loader | `TranspileExtra<'a>`, `ResolveFunctionResult<'r>`, `ParseOptions<'a, 'b>`, `ParseResult<'a>` | | parser | `ScanPassResult<'a>` | | crash handler | `Action<'a>`, `ActionGuard<'a>` | - `Transpiler::parse<'p>` has its own lifetime instead of the `Transpiler`'s. The VM's is a `Transpiler<'static>`, which is why the module loader had to erase every path and source it parsed. - In the bundler, what lives in the graph is `'a`, the arena lifetime `Graph<'a>.heap` already had. A worker thread sees a `ParseTask<'static>`; that one erasure is in `task_callback` / `io_task_callback`, with its contract stated there. - `type Str = &'static [u8]`, `trait IntoStr`, `path_as_static` (16 callers), `unsafe fn Path::into_static` and both `import_path: &'static [u8] = unsafe { .. }` in the resolver (with their `TODO: thread an explicit lifetime through Result`) are deleted. `Path::dupe_alloc` is safe code now. The one holder that has to erase a whole `Source` uses `unsafe fn Source::into_static`. - `scoped_action!` binds its guard to a hygienic local (the `pin!` trick), so the guard cannot be leaked and the thread-local cannot outlive what the action borrows. Safe, same cost. - The scan pass takes the `ScanPassResult` containers by value and gives them back, instead of `&'a mut T<'a>`. #### What the compiler found once the types were true - `PackageInstall::get_installed_package_json_source` returned a `Source` whose path pointed into a local `spill` buffer (E0515). Latent: its log is dropped unread, so nothing dereferenced the path. Now the function returns the contents and the caller names the source. - The module loader could move a virtual source into the async module queue, which outlives it. (That branch cannot run today: nothing fills `pending_imports`. It now refuses a virtual source.) - `bun link` borrowed `manager.original_package_json_path` across a `&mut manager` call. - `bun run` and `Transpiler::resolve_entry_point` returned a resolve result made from a local `./`-prefixed copy of the name. Such a result does not borrow the name in practice, but nothing said so. `resolve_entry_point` now takes the buffer from its caller. - For a framework built-in module, `enqueue_file_from_dev_server_incremental_graph_invalidation` would put a path borrowing its `path_slice` argument into the graph: the case `enqueue_entry_item` had a second copy and a 10-line comment for. #### What existed only to back a fake `'static`, and is gone - `VirtualMachine.resolved_path_dups`: a heap copy of every `blob:`, `macro:`, node-fallback and `[eval]` specifier ever resolved, kept until the VM is destroyed. The one consumer copies the result into a JS string immediately. - `intern_transpile_path` on every synchronous transpile: each distinct path went into the process-lifetime `FilenameStore` and a thread-local map, for the sake of the async queue. It now runs in that branch only. - The second `dupe_alloc` of every entry point, the `*result.path() = path` write-backs, the copy of `tmp_source.path` in `run_resolver`, and the `free_list` parking of plugin `onResolve` results (that arm is now safe code). - A deep copy of every message each time a log was handed from a task to its owner. #### Cost Nothing is added to a hot path. Two places copy where they did not: - A located diagnostic allocates `file` and `namespace` when it is made. Before, it borrowed them and was deep-copied (text, file, namespace, line text, notes) when its log was handed over; now it is moved. - `path_with_pretty_initialized` copies a `node`-namespace path into the bundle's arena instead of returning its argument, and a plugin's external path is copied into the arena instead of keeping its two boxes alive. I have not benchmarked this and make no performance claim. #### Still erased, now visibly Each of these is an explicit `unsafe` with its contract next to it: the two task callbacks; `MapEntry.source`, `PackageJSON.source`, `ParseResult.source_contents_backing` and `eval_source` (self-referential or owner-beside-borrower); the async module branch; `BuiltInModule::Import` in `resolve_with_framework`; `ImportRule.url` at the CSS boundary. Out of scope and unchanged: the CSS crate (`arena: &'static Bump` by convention), AST strings (`StoreStr`, `E::String`), `LinkerOptions`, `Chunk.unique_key`. | in `src/**/*.rs` | main | this PR | | --- | --- | --- | | `.into_static()` calls | 7 | 1 | | uses of the safe erasers `path_as_static` / `into_str` | 26 | 0 | | `&'static [u8] = unsafe { .. }` | 37 | 30 | | `detach_lifetime(..)` | 92 | 84 | | `unsafe` blocks, fns and impls | 12,886 | 12,861 | #### Behaviour changes - `position.file` and `position.namespace` are correct in the cases above. No other message changes (see the probes below). - `Log::append_to` empties the source log; every caller dropped or reset it afterwards. ### How did you verify your code works? - New test in `test/js/bun/resolve/build-error.test.ts`, five cases over the three ways in (`import()` and `require()`). It fails on 1.4.4 (`USE_SYSTEM_BUN=1`, wrong bytes) and on a debug + ASAN build of main (heap-use-after-free), and passes here. - New unit test `location_tests::a_location_outlives_the_path_of_its_source` in `bun_ast`, which drops the source and its buffers before reading the locations. Passes under Miri (`bun run rust:miri -p bun_ast`). - Two differential probes, main (debug + ASAN) against this branch, byte-for-byte identical: - 3,399 `BuildMessage` / `ResolveMessage` objects (every field, `toJSON`, `String`, `Bun.inspect`, notes) from `import()`, `require()`, `blob:` URLs, `Bun.build` and `Bun.Transpiler`, over broken JS, TS, a macro import, a missing import, YAML, TOML, JSON, JSON5 and JSONC, leaving out the cases that are the bug; - 189 `Bun.build` runs: 3 targets × {no plugin, a declining `onResolve`, answering plugins, both} × entry points given as paths, bare names, packages, `data:` URLs and builtins, in-memory `files`, CSS, HTML, externals, dual packages; outputs, hashes, chunk names and logs. - On a debug + ASAN build these files pass with no failure and no ASAN report: - bundler: `bundler_{plugin,plugin_chain,files,loader,html,html_server,edgecase,browser,naming,splitting,barrel,defer,allow_unresolved,regressions,cjs2esm,bun}`, `metafile`, `html-import-manifest`, `plugin-error-nested-throw`, `plugin-sync-exception-fallback`, `css/css-modules` - runtime: `plugins`, `transpiler.test.js` (covers `scan` / `scanImports`), `macro-test`, `runtime-transpiler`, `inspect-error`, `worker_blob`, `run-eval`, `run-cjs`, `run-unicode`, `run-extensionless`, and in `js/bun/resolve`: `build-error`, `resolve`, `resolve-error`, `resolve-ts`, `import-meta`, `import-meta-resolve`, `import-query`, `import-custom-condition`, `non-english-import`, `require`, `bun-main-entry-point`, `load-same-js-file-a-lot`, `tsconfig-extends-leak`, both `resolve-autoinstall-*` - parsers: `json5`, `jsonc`, `toml`, `yaml`, `css-loader`, `invalid-utf8-column` - install: `bun-link`, `bun-pm`, `bun-pm-pkg`, `bun-pm-version`, `bun-workspaces`, `bun-workspaces-self-contained`, `bun-patch` - dev server: `bake/dev/{css,esm,html,plugins,hot,sourcemap,incremental-graph-edge-deletion}`, `dev-and-prod`, `serve-plugins-dev-server`, `framework-router`, `bun-serve-html`, `bun-serve-html-entry` About half of them (the bundler, plugin, transpiler, resolve and dev server ones, `bun-link`, `bun-pm-pkg`) were run again on the final commit; the rest ran one review round earlier. - These fail on my machine in the same way on a debug build of main, so they say nothing either way here and I am relying on CI for them: `bundler/cli.test.ts` (3, `--compile` of a debug binary takes 16 s), `native-plugin` (its build hook times out, so the native plugin's log path was **not** exercised locally), `worker-entry-point` (1), `worker.test.ts` (1 to 2), `run-crash-handler` (1, a symbol name), `bake/dev/bundle` (1, a proxy variable of my environment), and the memory tests `require-cache` (7) and `inspect-error-leak` (1), which time out on a debug build. `bake/dev/production` fails 3 to 9 tests at the 5 s default on both builds and passes 21 / 21 on both with `--timeout 90000` (64 s here, 67 s on main). - `cargo check --workspace --all-targets`, `cargo clippy --workspace --no-deps` and `cargo fmt --check` are clean. `bun run rust:check-all` passes for `x86_64-pc-windows-msvc`, `aarch64-apple-darwin`, `x86_64-unknown-linux-musl`, `x86_64-unknown-freebsd` and `aarch64-linux-android`; I did not run the other triples. Miri passes for the three crates of its CI set that this touches (`bun_ast`, `bun_paths`, `bun_bundler`). --- _Generated by [Claude Code](https://claude.ai/code/session_01GZEGTqHnmApKDDg6554YEM)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ter-free) (oven-sh#46825) ### What does this PR do? Printing an Error could read a source-map file name that had just been freed, and put the stale bytes in the printed stack. This fixes that bug and makes the mistake behind it a compile error. **The bug.** For a stack frame whose URL is not an absolute path, `Lookup::display_source_url_if_needed` returned a string that borrowed the map's own `sources` entry. The printer reads the Error's own `cause` between remapping the frames and printing them. A `cause` getter that loads the same file again replaces the map, which frees its names. `require(URL.createObjectURL(Bun.file("rel.js")))` gives such a URL. - AddressSanitizer build: `heap-use-after-free` under `print_stack_trace`. - Release 1.4.4: the frame prints as `(\^@\^VnO�\^B\^@:1:1)` instead of `(orig.ts:1:1)`. - `Bun.inspect`, `console.log`, `console.error` and an uncaught throw are affected. **The root cause.** `bun_core::String` is the owned type, but `String::borrow_utf8`, `from_bytes` and `ascii` were safe functions that took a `&[u8]` and dropped its lifetime. `StringView<'a>` already existed for borrowing. - Those three constructors are removed. Borrowing goes through `StringView<'a>`, which derefs to `&String`. - `String::clone` copies the bytes a view borrows. It was a bitwise copy, so `view.clone()` and `thread_isolated_copy()` returned a dangling string. - `create_static_external*` take `&'static`. `create_external` is `unsafe`. Both lose `is_latin1`: `false` read past the end of the slice, and every caller passed `true`. - New: `StringOrView<'a>`, a `String` field that may hold borrowed `'a` bytes instead of a ref. It is what a C++ `BunString` slot really is. It derefs to `&String`, drops the ref if it has one, and has no `DerefMut`. - New: `String::from_static_bytes`, `StringView::ascii`, `StringView::as_strings`. **What the compiler then flagged** (112 call sites; most became a `StringView` with no other change): - `ZigException`, `ZigStackTrace`, `ZigStackFrame` and `Holder` get a lifetime, and their strings are `StringOrView<'a>`. C++ fills them with refs, the dev server fills them with slices of the request body, and `remap_zig_exception` fills the source lines with slices of the source text. The caller owns that text and passes `&'a mut Utf8Bytes`, so the compiler checks that it outlives the holder. `ZigStackTrace::from_frames` keeps its frame buffer borrowed. - `Route`, `JSS3Error` and the npmrc test object get a lifetime. - `ResolvedSource` and `SystemError` own their strings. C++ wraps a borrowed ASCII string it receives by value with `createWithoutCopying`, so those borrows could outlive the Rust frame. - `bun bake` production moves each server chunk's bytes into the string that JavaScriptCore loads. The chunk was aliased by a string that JavaScriptCore keeps for the module's life, but `PerThread`, which owned the bytes, drops before the VM. **Cost, read from the code.** A value that ends up as a JS string is still copied once: C++ shares an owned impl and copies a borrowed one, so `clone_utf8` + share replaces borrow + copy (`process.argv`, `os.homedir()`, UDP `rinfo.address`, the `fs` `SystemError`, the inspector's bundle-failed payload). What costs more than on `main`: - The relative arm of `display_source_url_if_needed` copies the name. That is the fix. - `AsyncModule` copies `source_url`. The alias it replaces dangled once the `AsyncModule` was freed. - `from_static_bytes` (the `bun:wrap` source, a compiled module's bytecode origin path) makes a `WTF::ExternalStringImpl`: one more small allocation than `createWithoutCopying`, once per module load, freed with the string. - `PerThread::init` boxes one more key per server chunk. - Cloning a view copies its bytes. It was free, and dangling. What costs less: a builtin's resolved path is `String::static_`. In the module loader (`transferToWTFString`) a repeated import finds the atom and allocates nothing, where `main` made a `createWithoutCopying` header each time. The JS resolve APIs (`into_js`) copy it once, as on `main`. **Not changed here.** - `String::create_uninitialized_latin1` / `_utf16` return a `&'static mut` slice that lives only as long as the string. Same kind of lie, 24 callers in hot paths, so it needs its own PR. - A relative `Bun.file()` path behind a `blob:` URL stays the module's source URL: oven-sh#46842. **Also in the diff.** `test/js/web/websocket/proxy-test-utils.ts` carries the change from oven-sh#46170 unmodified. It fixes a flaky `ws-proxy.test.ts` case on Windows 2019 x64 that turned this PR's CI red and also fails on other branches. ### How did you verify your code works? - **The bug no longer compiles.** Putting `String::borrow_utf8(display)` back gives E0599. `Some(StringView::borrow_utf8(display))` gives E0308, `expected String, found StringView<'_>`. Changing the return type to `StringView<'_>` gives a lifetime error. - **Neither do these.** Declaring the source text after the holder that borrows its lines: E0597. Freeing the request body before the dev server prints its frames: E0505. Clearing the frame `Vec` while the trace points at it: E0499. - **No leak.** C++ stores an owned ref in source line 0 for an error with `lineText`. 1200 inspects of an error with a fresh 1 MB `lineText` grow RSS by 8 MB (debug build, ASAN quarantine at 1 MB). - `a cause getter that loads the throwing file again` in `test/js/bun/util/inspect-error.test.js` covers the four printers and `error.stack`. It fails with `USE_SYSTEM_BUN=1 bun test` and passes with `bun bd test` (46 pass). The repro reports the use-after-free on a debug build of `main`. - `stringViewCloneOutlivesBytes` in `test/internal/bindgen.test.ts` frees the bytes before it returns the clone. `stringViewEcho` now goes through `String::clone` for Latin-1 and UTF-16 views (226 pass). - An A/B probe of the touched behaviour gives the same output on release 1.4.4 and on this branch: `process.argv`, `os.homedir()`, brace expansion, `Transpiler.scan`, `dns.getServers()`, UDP `rinfo`, timer warnings, JSON IPC, builtin resolution (all 182 alias keys through five resolve APIs), `bun:wrap`, the valkey URL, error printing with non-ASCII source lines, system errors, redirects, S3 errors, CLI errors. - 36 related test files on a debug + ASAN build, with no ASAN report. The failures are environmental: no `USER`, no IPv6, no DNS, 5 s timeouts that pass with a longer limit, and a leak fixture that only knows an ASAN build by the name `bun-asan`. `native-plugin.test.ts` did not run locally (its setup hook timed out on a download). - `cargo check --workspace --all-targets`, `bun run rust:clippy`, `cargo fmt --check`, the source lints, and `rust:check-all` for `x86_64-pc-windows-msvc` and `aarch64-apple-darwin`. `mordant` was not run locally. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ams (oven-sh#46650) ### What does this PR do? Removes the HdrHistogram_c dependency. `createHistogram()` and `monitorEventLoopDelay()` now count in `bun_histogram`, a new crate with `#[forbid(unsafe_code)]` and no dependencies. It holds the 13 of the library's 39 functions that Bun needs. The six hand-written C++ files are replaced by three bindgen classes, so the classes stay native: every method and getter is a method of one Rust struct. **Now as in Node.js v26.3.0** - `min` and `max` are bucket edges, so `max === percentile(100)`. `add()` no longer takes a dropped value as the new `min`/`max`, and returns `undefined`. - `percentiles` holds numbers, not BigInts. - Validation and error text of `record`, `percentile*`, `add`, `createHistogram`: `record(1.5)` throws. - `Histogram`, `RecordableHistogram` and `ELDHistogram` classes, none constructible. `toJSON()` and `util.inspect()` output. - `monitorEventLoopDelay()`: a new histogram per call, without `record()`. It records the time between samples, not the lateness. `enable()` keeps earlier data. - `structuredClone()` and `postMessage()` share the counts across threads. - After `a.add(b)`, `a.recordDelta()` measures from the later `recordDelta()` of the two. It reads the clock of `uv_hrtime()`, with the counts locked, so threads that share a histogram record their times in order. - The shared `validateInteger` raised `max` to `min` when `min > max`, so `{ lowest: 10, highest: 15 }` named the wrong range. Fixed in `NodeValidator.cpp`. **Differs from Node.js, where Node or the C library has a bug** (measured on v26.3.0) | | Node.js | Here | |---|---|---| | `mean` of 2000 × `MAX_SAFE_INTEGER` | `-218371805369335.8` | `9005000231485440` | | `record(433722)`, then `percentiles.get(100)` | `433664`, below `percentile(100)` | `433919` | | `add()` that drops its only value | `count` 1, `exceeds` 0 | `count` 0, `exceeds` 1 | | `percentiles`, then `percentilesBigInt` | one `Map`, refilled | two | | Empty, `lowest: 1000`: `percentile(50)` | `511` | `0` | | `{ lowest: 1, highest: 10n }` | `TypeError` | works | | `{ lowest: 2n ** 61n, highest: 2n ** 62n }` | aborts | throws | | `add(new Proxy({}, { get: () => 1 }))` | aborts | throws | | `add()` | `TypeError` without a code | `ERR_INVALID_ARG_TYPE` | | `Reflect.ownKeys(histogram)` | `constructor` and three symbols | none | | `v8.serialize(monitorEventLoopDelay())` | does not throw, and the data is lost | `DataCloneError`, as in 1.4.4 | | `disable()`, 400 ms, `enable()` | records 420 ms | does not | Seven differences are not fixes. They come with a native class, and every native class in Bun has them: - `new Proxy(histogram, {}).count` and `Object.create(histogram).count` throw, because the counts live in the object itself. Both are printed as `RecordableHistogram {}`, where Node prints the numbers. - A getter with the wrong `this` throws a `TypeError` without a code: "The Histogram.count getter can only be used on instances of Histogram". Node throws `ERR_INVALID_THIS`. - A method with the wrong `this` throws `ERR_INVALID_THIS`, as in Node, with Bun's text: "Expected this to be instanceof RecordableHistogram, but received …". - The prototypes have `Symbol.toStringTag`, so `Object.prototype.toString.call(histogram)` is `[object RecordableHistogram]` and not `[object Object]`. - `toJSON()` reads the counts and not the public getters, which only shows if a getter is redefined on the instance. - `RecordableHistogram()` without `new` throws a `TypeError` with the code `ERR_ILLEGAL_CONSTRUCTOR`. In Node it has no code. - `console.log(Histogram.prototype)` lists the members, the getters as `undefined`. Node prints `{}`. **bindgen** `RecordableHistogram` and `ELDHistogram` inherit from `Histogram`, which bindgen could not say. It gets: - `extends`: a class whose C++ cell is a subclass of its base's and shares its Rust struct. The prototype and the constructor inherit from the base's, and an instance is an instance of the base wherever one is taken. It is what `JSS3File` writes by hand on top of the generated `JSBlob`, and how WebCore does inheritance. - `final: false` for a class whose struct is stored in the object. A subclass adds no fields, which `STATIC_ASSERT_ISO_SUBSPACE_SHARABLE` checks. - The key `@@inspectCustom`, which is `Symbol.for("nodejs.util.inspect.custom")`, and a JS builtin under an `@@` key. Two members are builtins written in JavaScript: `inspect.custom`, which calls `util.inspect`, and `[Symbol.dispose]`, which calls `this.disable()` as Node's does. A number in Node's `ERR_OUT_OF_RANGE` has its own format (`-0`, `9_007_199_254_740_992`). `bun_core::fmt::NodeNumber` prints it, for `record()`, `percentile()` and the `validate_integer` of `node:fs`, and in place of a loop that `crypto.randomInt` had. `createHistogram()` and `monitorEventLoopDelay()` validate their options in JavaScript, as before. **Memory** HdrHistogram_c allocates a slot for every value up to `highest` when the histogram is created: 360 KB by default, 38 MB with `figures: 5`. Here the counts are in chunks of 512 slots (4 KB), and a chunk is allocated when a value is first recorded in it. So the memory depends on how many different values there are, not on how large they are. | | Bytes | |---|---| | `createHistogram()` | 224 | | One value | 5 KB | | Six values from 100 to 1e9 | 25 KB | | A value in every chunk | 361 KB | | `figures: 5`, `record(1)` and `record(MAX_SAFE_INTEGER)` | 84 KB | `reset()` frees the chunks. Before, the size told to the GC was stored in a `uint16_t`. The GC is told what JavaScriptCore tells it of a `SharedArrayBuffer`, which also grows after it is created and which a structured clone also shares with other threads: | | Told to the GC | As in JavaScriptCore | |---|---|---| | An object is created, a clone too | the whole size, as an allocation | `JSArrayBuffer::finishCreation` | | An object is marked | the whole size, as held | `Heap::extraMemorySize`, `JSWebAssemblyMemory::visitChildrenImpl` | | `record`, `add`, `recordDelta` or a sample of the event loop allocates a chunk | the chunk, as an allocation in the thread that did it | `ArrayBuffer::grow`, `JSWebAssemblyMemory::growSuccessCallback` | The first two are what `estimatedSize` in bindgen already does, which is `[ReportExtraMemoryCost]` in WebCore. The third is `Heap::reportExtraMemoryAllocated`, which Rust could not call: it is now `VM::report_extra_memory_allocated`, and `JsHeap::report_extra_memory` in bindgen. So clones of one histogram each count all of it, and a thread is not told of what another thread adds, as with a `SharedArrayBuffer`. **Speed** (release builds, linux x64, the median of 3 runs) | | 1.4.4 (C) | Here | Node.js | |---|---|---|---| | `createHistogram()` | 18 µs | 0.17 µs | 14 µs | | Create, record 1e5 to 1e9, drop | 22 µs | 3.5 µs | 48 µs | | … peak RSS of 30,000 turns | 198 MB | 52 MB | 833 MB | | The same, keeping the last 2000 | 65 µs | 5.8 µs | 129 µs | | … peak RSS | 1397 MB | 165 MB | 3045 MB | | `mean`, 2 values far apart | 179 µs | 0.47 µs | 184 µs | | `mean`, dense | 161 µs | 35 µs | 171 µs | | `stddev`, dense | 468 µs | 77 µs | 467 µs | | `percentile(99.9)`, dense | 8.1 µs | 3.8 µs | 8.2 µs | | `percentiles`, dense | 124 µs | 27 µs | 129 µs | | `add`, dense | 166 µs | 66 µs | 173 µs | | `structuredClone()`, every chunk in use (360 KB) | throws | 25 µs | 6.6 µs | | … with `figures: 5` (37 MB) | throws | 144 µs | 6.6 µs | These were measured while the classes were written in JavaScript. `record` (13 ns, 7.3 ns in 1.4.4) and `count` (7.4 ns, 3.2 ns) each went through a JS method then, which they no longer do, and have not been measured since. The rest is time spent in Rust. With `figures: 5` the first `record` also allocates a 76 KB table of chunks. Slower than Node.js: a clone of a large histogram, because it counts as an allocation of the whole size and so makes the GC run. That is the cost of a clone of any shared buffer here: a 37 MB `SharedArrayBuffer` takes 136 µs in the same build. `packages/h3blast` linked the library's objects, so it gets a small histogram of its own. ### How did you verify your code works? - `test/js/bun/perf_hooks/histogram.test.ts` uses `node:test`: 98 pass here, 48 fail on Bun 1.4.4. Under Node v26.3.0, 85 pass and the 13 for the table above are skipped, so its expected values are Node's. It passed unchanged when the classes went native, but for the texts above. - `test/internal/bindgen.test.ts` covers `extends` with a class of its own: the two chains, a base member on an instance of the subclass, a member of the subclass on an instance of the base, `from_js` both ways, the internal fields of the base, `Drop`, and what a schema may not say. - Node's `test-perf-hooks-histogram.js` is added unmodified. It passes, and fails on 1.4.4. - `test/js/node/perf_hooks/perf_hooks.test.ts` checks the sizes above, and that histograms are collected without an explicit GC once they are dropped: those that grew, clones, and those that arrived from a worker. - The fuzzer below is not in the repo, so I planted 19 bugs in the chunks and the binding (a chunk skipped, a slot index off by one, `reset()` that keeps counts, a wrong size, the GC not told of growth, the disabled time recorded, deltas rounded to microseconds, the clock read before the lock, …) and ran only these two files: each bug fails at least one test. That was before the classes went native and has not been repeated since. - Real packages, under an idle and then a blocked event loop, on this branch, Node v26.3.0 and Bun 1.4.4: `prom-client`, `@opentelemetry/instrumentation-runtime-node`, `@fastify/under-pressure` and `piscina`. This branch reports what Node reports. 1.4.4 differs: an idle loop at `resolution: 10` has a p50 of 0.1 ms there, and 10.2 ms here and in Node. - A differential fuzzer outside the repo compares the crate with HdrHistogram_c 0.11.10, the same under UBSan, 0.12.1, Node's 0.11.9 and a naive model, and the JS API with Node v26.3.0. - Over the versions of the crate: about 1.3 million random programs, 2,064,400 configurations, 586 million slots, every reachable branch covered, and mutation testing. More than 110,000 random API programs against Node with no difference outside the table above. Also threads, GC stress, forged serialized bytes, `node:vm` realms, tampered builtins, the top JIT tier, and 54 kinds of object printed in 60 ways. - All of this was before the classes went native. The crate has not changed since. - The campaigns that go through the JS API were repeated on the native classes, at abf5d8c: no crash, no sanitizer report, no wrong number. - 3,000 random API programs against Node, with no difference outside the lists above. 60 rounds of up to 61,000 values with whole `percentiles` Maps and 400 `percentile` calls each are identical to Node. - 2,464 calls with hostile arguments and receivers, most of them throwing natively: no unchecked exception under `BUN_JSC_validateExceptionChecks=1`. - Every member called 200,000 times and then with a call that breaks the speculation, and 20 hot readers and writers beside a worker that fills and resets shared counts: also with `jitPolicyScale=0`, `useZombieMode`, `validateDoesGC`, `validateGraph` and `collectContinuously`. - The GC, clone and thread campaigns give the memory figures they gave before, equal to their `SharedArrayBuffer` controls. 110 exits racing workers under LeakSanitizer: none bad. - It found two bugs, both fixed in later commits and both now tested: printing an object that inherits from or proxies a histogram threw, and `record(-0)` said "Received 0". - The commits after abf5d8c (those two fixes, `NodeNumber`, and fixes from a review) are being fuzzed now. - It found six bugs of mine, all fixed here: `percentile(100)` below `max` at huge counts, four in how the counts were allocated, which is what led to the chunks, and a histogram that reached a thread empty and then full, of which its GC was told nothing (over 5 GB where it is now 0.7 GB; Node.js passes 5 GB too). That one came from a rule of my own for clones, which is gone. - All of it was repeated on the chunks, the last change to the crate, and found nothing: 370,000 random programs, the same sweeps, 3.85 million ranks, every reachable branch, 490,000 coverage-guided runs and 74,000 API programs. Each check also asserts that a chunk exists if and only if one of its slots is in use, and that the size told to the GC is exact. Of 317 mutants of the crate, 20 do not compile, 294 are killed, and the 3 that survive cannot change a result. - The histogram of `h3blast` gives what HdrHistogram_c gives for every value from 0 to 60,000,010 and in 3000 random rounds, bit for bit, under ASAN and UBSan. - macOS and Windows are untested locally. One thing is specific to macOS: `CLOCK_MONOTONIC` has whole microseconds there, which is why `recordDelta()` reads `uv_hrtime()`. The test for it fails on Linux when the deltas are rounded to microseconds, but only the macOS lanes can show that the real clock is fine. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
### What does this PR do? Moves `WEBKIT_VERSION` from `680ef47cec2c` to `982277c9dd3e`, the head of oven-sh/WebKit `main`, and updates the places in Bun that the new engine needs changed. The range is oven-sh/WebKit#757, oven-sh#903, oven-sh#905, oven-sh#906 and oven-sh#907. This PR does not add or enable JavaScriptCore's ahead-of-time compiler. That is oven-sh#45631. **Build scripts** - **Prebuilt names:** oven-sh/WebKit#757 renamed the release assets. `release`, `lto` and `lto-hardened` keep their names and are built with `ENABLE_AOT=0`. Every other variant now ends in `-aot`, so `prebuiltSuffix()` appends that. In CI this changes one lane, `linux-x64-asan`. Locally it changes `bun bd`. - **Debug warnings:** `AssemblyHelpers.h` captures `this` in generic lambdas, which `-Werror=unused` rejects in debug builds, so `-Wno-unused-lambda-capture` is added there. **Code** - **Removable features:** JavaScriptCore now calls `Bun__removableFeatureState` and `Bun__throwForRemoved*`, in every variant. `RemovedFeature.cpp` defines them the way WebKit's own `AOTNoFeatureRemoved.cpp` does: nothing is ever removed. - **Table of built-ins:** the `-aot` libraries leave 35 symbols (`JSC::AOT::builtinTable*` and `describedBuiltinTable`) for the program that links them to define. Only the compiler and the code it compiles read that table, and `Options::useAOT()` is off, so `RemovedFeature.cpp` defines an empty one under `#if ENABLE(AOT)`. - **Built-in functions:** `BuiltinGenerator` returns `JSC::BuiltinCode`, and `BuiltinsSourceProvider::create` takes the id of its first function. - **`JSType`:** five new types moved the numbers mirrored in `JSType.rs`. A `static_assert` now pins them. - **Static strings:** `StringImpl::ref()` and `deref()` no longer count static strings, which may be in read-only memory. The Rust port of the two does the same. Three `ASSERT(refCount() > 0)` in `BunString.cpp` accept a static string, whose count is now 0. `RemovedFeature.cpp` has the path oven-sh#45631 uses, so that PR gets a merge conflict instead of duplicate symbols at link time. `JSType.rs`, `bun_alloc/lib.rs` and `BunString.cpp` are the same hunks as in oven-sh#45631. ### How did you verify your code works? Bun was not built locally; CI is the build. - **CI build [#133439](https://buildkite.com/bun/bun/builds/133439)**, on an earlier commit of this PR with the same engine changes: all 13 `build-bun` jobs passed, ASAN included. That commit generated a full table of built-ins. The empty table that replaced it has not linked in CI yet. - **New test:** the three cases of `webkit-upgrade-982277c9dd.test.ts` fail on main's CI binary at the merge base (old engine) and pass on the linux-x64 binary of #133439 (new engine). - **Bytecode snapshot:** `BytecodeList.rb` changed, so six values in `bundler_bytecode_portable.test.ts` moved. They come from the linux-x64 binary of #133439, and linux-aarch64-musl reported the same six. Checks done without building: - **Embedder symbols:** `llvm-nm` over the old and new `linux-amd64` archives and the new `linux-amd64-asan-aot` archives gave the 40 symbols Bun has to define. `RemovedFeature.cpp`, compiled alone with Bun's flags, defines all 40. - **The table is not read:** in the engine outside `aot/`, the AOT hooks in VM, heap and realm setup that I read are behind `vm.hasCompiledCode()` or `Options::runsCompiledImage()`, and only `$vm` names the table. - **Assets:** all 30 asset names Bun can ask for exist at the new tag. - **Headers:** I diffed the 225 changed headers and searched `src/` for every removed or changed name. - **`JSType.rs`:** a script compared all 88 constants with `JSType.h` at the pinned commit. - **Syntax check:** `clang++ -fsyntax-only` with the flags from `compile_commands.json` passes on `RemovedFeature.cpp` (with and without `ENABLE_AOT`), the generated `WebCoreJSBuiltins.cpp` and 29 hand-written files that use the changed APIs. - **arm64 allowlist:** the outline-atomics helpers the arm64 archives use are the same set as before, so `allowlist-aarch64.txt` does not change. - **Lint:** `test/internal/source-lints` passes (184 tests), `tsc` passes for `scripts/build` and `src`, and Prettier, clang-format and `cargo fmt` are clean. Not covered: - `--webkit=local`, Windows and macOS debug builds, and `--hardened=on` are untested, and CI builds none of them. **Binary size:** the binaries grow by 0.75 to 1.64 MB (linux-x64 +1.39 MB, darwin-aarch64 +1.17 MB, windows-x64 +1.08 MB), over the 0.5 MB limit. It is JavaScriptCore's own growth: release builds already link the prebuilt without the ahead-of-time compiler, and Bun's diff adds a few hundred bytes. The head commit carries `[skip size check]`. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…#46638) ### What does this PR do? Removes the dependency on libarchive. The two things Bun used it for, tar and the gzip container, are now written in safe Rust. - **`src/tar` (`bun_tar`, new, `#[forbid(unsafe_code)]`)** - `Reader`: a push parser. The caller hands it whatever bytes it has and gets events back (`Entry`, `Data`, `EndOfEntry`, `EndOfArchive`, `NeedInput`), so running out of input is an ordinary state instead of the `ARCHIVE_RETRY` patches. Reads v7, ustar, pax, GNU long name/link and every GNU/SUN sparse format, in the two dialects Bun had: the npm one (node-tar rules) and the general one (concatenated archives). - `Writer`: restricted pax, regular files only. - `gzip`: header, trailer, multi-member and nested streams. DEFLATE itself still comes from zlib through `bun_zlib`. - `compose`: what Windows does to a UTF-8 pax path (composition only, not NFC, Unicode 6.0 tables). - What differs by platform (charset, AppleDouble `._` files) is a runtime option, so all of it compiles and can be exercised on any host. - **`src/libarchive` becomes `src/archive` (`bun_archive`)** and keeps only the extraction policy: path safety, symlinks, modes, sparse writes. - `Bun.Archive`, `bun install` (buffered and streaming), `bun pm pack`, `bun publish`, `bun pm diff` and `bun create` are ported. `TarballStream` loses its C read callback and phase machine, and its drain-only state moved into its own struct so no reference to the whole struct is held while the HTTP thread appends. - Deleted: the dependency, its six patches, the update workflow and `process.versions.libarchive`. The attribution stays in `LICENSE.md`, because parts of `src/tar` are derived from libarchive's tar reader and pax writer. The goal was no observable change, quirks included. These are the changes I know of: | Change | Why | | --- | --- | | ACL records (`SCHILY.acl.*`, Solaris `A` headers) no longer change an entry's permissions | Bug fix. An archive made with `tar --acls` could lose its exec bits or extract a directory as mode 000. GNU tar and node-tar take permissions from the header | | A malformed ACL record is no longer a warning | Follows from the row above. libarchive trusted an entry less after a warning: a regular entry whose name ends in `/` stayed a file, with its data. Now it becomes a directory, as it does after any well-formed header, and what was its data is read as headers. Only seen in generated archives | | An in-memory entry of more than 2 GiB − 1 bytes is written whole, by `new Bun.Archive({...})` and for the `package.json` that `bun pm pack` and `bun publish` print again | Bug fix. libarchive takes at most `INT_MAX` bytes per write call, and Bun made one call and did not look at the count. The header gave the full size, the rest of the entry was zeros, and there was no error. Measured with release binaries: for an entry of 2 GiB + 1,024 bytes, the last 1,025 bytes are zeros on main and right here. No test: it needs about 4 GiB of memory | | A pax record whose `<length> <key>=` is longer than 512 bytes, in a pax header of more than 1 MiB, inside gzip, is always read | libarchive looked for the `=` only in what was left of its 64 KiB block of unpacked bytes (or in 512 bytes if that was more), and otherwise passed over the rest of the header. So the same tar gave one name compressed and another uncompressed or shifted by a byte. Needs a key of about 506 bytes or more. The same on Windows, in nested gzip and in several members: only the position in the innermost unpacked stream counts. Found by measuring libarchive's own coverage | | A tarball extracted while it downloads no longer fails because of where a piece of the download ends | Bug fix. The patch that let libarchive wait for more input did not cover every read: the optional fields of a gzip header, an extension body over 1 MiB, and, outside npm's rules, more blocks of an old GNU sparse map, a sparse 1.0 map and a `._` file's contents. A piece that ended inside one of those failed the install with "Truncated tar archive". Reproduced with real `bun install` on main for tarball URLs and `github:` dependencies: six of six cases fail there and pass here, and the same tarballs install on main when they are not streamed. Only tarballs of 2 MiB and more are streamed by default, so it takes an unusual one. Every cut position of six small tarballs through real installs, 22,704 per binary: main fails at 1,064, exactly the ones a model of it in the harness predicts, and this branch at none. In that harness, 7.96 million pairs of libarchive with and without a read callback that runs dry: 61,080 fail only when it does, none fails only when it does not, and none succeeds both ways with different entries | | `Bun.inspect(archive)` counts files past a block with a bad checksum | It now agrees with `files()` | | Some error messages for damaged archives | Measured against libarchive on generated damaged archives: about 96% are word for word the same for plain tar. Most of the rest are cases where libarchive set no message (surfaced as `ReadError`) or left a stale warning in place. In `Truncated input file (needed N bytes, only M available)`, `M` is now the real count; libarchive printed 0 in nearly every case | | The verbose label `libarchive error:` is now `archive error:` | | | `process.versions.libarchive` is gone | | | Windows only: a name that cannot be composed (more than ten combining marks in a row) is written as raw bytes with `hdrcharset=BINARY` | `new Bun.Archive({...})` used to throw `ArchiveHeaderError` for it. Measured with CI's Windows binaries under Wine | ### How did you verify your code works? **New tests through public APIs.** The expected results were recorded with the last build that used libarchive (`record.ts` next to each fixture re-records them), so they compare Bun before with Bun after. They pass on a released 1.4.4 by design, except the ACL and `Bun.inspect` tests, which cover the intended changes. - `test/js/bun/archive-golden`: about 1,250 archives, well formed and damaged, plain and gzipped, through `Bun.Archive` `files()` and `extract()`; 21 writer snapshots through `bytes()`; entry names and written headers for Windows and macOS, with values measured on released Bun on those systems; gzip inside gzip (24 layers and no more, headers that span blocks, the 1 MiB name limit, how far an outer layer is read, a member that ends exactly on a 64 KiB block); header checksums over signed and unsigned bytes. - `test/cli/install/tarball-golden`: 191 tarballs through real `bun install`, extracted while they download and from memory. The server holds back the rest of each body until the extractor has started, so streaming does not depend on timing. - `bun-pack.test.ts`: the decompressed bytes and gzip header of a `bun pm pack` tarball. - `bun-install-streaming-extract.test.ts`: a piece of the download that ends in an optional field of the gzip header, or in a pax header of more than 1 MiB. Both fail on main. - `archive.test.ts`: the ACL fix, the `Bun.inspect` count, and the `File`s that `files()` returns: read in every way a `File` can be, printed, sliced, written to disk, and used after the archive is collected. The archives were picked from a structure-aware generator, keeping one only if it made the old reader report something no earlier one had. **New tests of the reader and writer themselves** (`test/js/bun/archive-golden/bun-tar.test.ts`). `Bun.Archive` shows only regular files, read by the rules of the system it runs on, and writes them with one mode and the current time. So `bun:internal-for-testing` gets two functions, which nothing else can reach: - `readTar(bytes, options)` returns everything the reader reports as text: type, mode, size, time, name and link target of every entry, where its data lies, and how the archive ends. The options are npm's rules or the general ones, the behaviour of macOS or Windows, reading from memory or like a download, the size of the pieces the input comes in, and which entries are skipped. - `writeTar(files, names, contents, options)` takes names as bytes, any mode, time and announced size, the naming of POSIX or Windows, and a gzip level. No Bun that used libarchive has them, so the expected results could not be recorded with one. Instead each was compared with libarchive itself, linked at Bun's pin and patches, when it was recorded: - 2,482 archives in 48 modes, 119,136 logs, all equal to what libarchive reports, wording of errors aside, and but for the permissions in 240 logs of seven archives with ACLs (first row of the table above). 1,463 archives are stored. 1,019 are made by the test: gzip cut, changed byte by byte, with every combination of header fields, in several members, nested up to 26 times, and ending on and around a 64 KiB block; values at and one over each limit of the reader; every sparse format; names that Windows composes or cannot; 19 shapes that a fuzzing corpus reached and the other archives did not; 77 written for lines and branches of libarchive's own tar reader that nothing else made it run. - The same archives with the input cut into pieces of 1, 7, 511, 512 and 513 bytes give the same logs. - 656 sets of files, each byte for byte what libarchive writes: 319 with the names of Windows, 231 compressed at levels 0 to 9, 35 with sizes around 8 GiB. - 18 long names for what Windows composes: all 931 pairs of the table among 70,000 pairs that do not compose, every character of the first two planes between a base and a mark, every mark of Unicode 6.0 against one mark of each combining class in both orders, each character of the blocks with marks where a class it should not have would show, and all Hangul syllables. libarchive read the same 18 names, and the writer has to record what the reader makes of each. A changed pair, block or class in the tables each failed a test when tried, also a class moved to a neighbouring one (27 to 29, 220 to 222, 230 to 232). - A dozen short tests that state the rules in the open, with answers libarchive also gave. Measured by replaying them through an instrumented `bun_tar`: the archives reach at least 96% of the lines of `reader.rs` and all of `header.rs`, and the file sets 96% of `writer.rs`. They run on every platform in CI, since the platform is an option. Before those names were added, the file took 1.4 s on CI's Windows machines, 5.7 s on macOS and 6.0 s under ASAN. Coverage of our own code cannot show a quirk of libarchive that no input ever sets off, so libarchive's tar reader (`archive_read_support_format_tar.c` at Bun's pin and patches) was measured too, in all 48 modes and six chunk sizes. The archives in this PR made it run 84.4% of its lines and 76.9% of its branches before the 77 were added, and 91.1% and 87.7% with them. Everything used while developing (80,102 files, the generators, a read callback that runs dry, and inputs written for this) reaches 92.3% and 88.6%, and all 55 functions. Each of the 198 lines left was read: an option Bun never set or a state it never creates (17 groups), another platform (6), a failed allocation (21), or dead code (23). This is a Linux build, so code for Windows and macOS alone is not compiled and not measured. 14 groups of lines and 4 branches could be reached and never had been. 1,108 inputs now reach all of them, and 0 of 52,944 logs differ. The pax row in the table above came from a branch that line coverage had counted as covered. **While developing**, with libarchive at Bun's pin and patches linked into the same process as `bun_tar` (not in this PR): - Reader: over 30 million comparisons of the full event log in twelve modes (both dialects, detection on and off, macOS and Windows behaviour), each also checked for not depending on how the input is cut into pieces. That includes skipping entries, gzip members and trailers of every shape, and up to 26 nested gzip layers, compared for exact equality, down to which entries are seen before a failure. - Writer: 33,000 comparisons, plain and at gzip levels 0 to 9, byte for byte. - Name composition: every code point pair that can compose, plus 9 million random strings. - A coverage-guided differential fuzzer (libFuzzer with a structure-aware mutator, every execution runs every mode): over 4 million executions, seeded with generated archives, archives made by other tar tools, libarchive's own test fixtures and npm tarballs. The harness reaches 96% of the lines of `bun_tar`; what is left is allocation failure and the unreachable side of checked indexing. **Old binary against new binary on Linux:** - About 11,300 archives through `Bun.Archive`: `files()` identical, `extract()` different only for ACL permissions. That includes what a rejected extraction leaves on disk. - About 9,000 generated tarballs through `bun install` as `file:`, buffered and streaming: same trees, same refusals. - `test/package.json` from its lockfile, 1,191 real packages served by a local registry: the two `node_modules` trees are identical (67,251 paths with mode, size and content hash). - 6,300 archives from a generator of hostile paths (symlink chains, `..`, absolute paths, directories swapped for files), half as root and half as an unprivileged user: no difference and nothing written outside the destination. - `bun pm pack` at every gzip level, and 4,000 random `new Bun.Archive({...})` inputs: identical bytes. - Five kinds of failing archive, 9,000 operations each in one process: no file descriptor left open, same memory growth. **Old binary against new binary on Windows (CI's x64 binaries under Wine):** 600,000 reads and 12,000 extractions with no difference; 150,000 writes that differ only in the row above. The existing archive, streaming-extract, tarball-integrity, symlink-traversal, pack, publish, pm-diff, cache-pack and create suites pass locally on Linux with main merged in, except `bun-create.test.ts` `should not crash > ["create"]`: in my container a bare `bun create` exits 0 with the build from before this PR too. CI has since run all of it on Windows, macOS, musl and ASAN. ### Performance CI's linux-x64 release binaries, main at the merge base against this branch, median of 11 runs on a memory-backed file system. `small` is 20,000 files in 100 MB (pax headers), `big` is one 200 MB file. | | main | this PR | | | --- | --- | --- | --- | | `files()` small.tar | 95.8 ms | 91.4 ms | 0.95 | | `files()` small.tar.gz | 61.2 ms | 52.7 ms | 0.86 | | `files()` big.tar | 469 ms | 316 ms | 0.67 | | `files()` big.tar.gz | 493 ms | 488 ms | 0.99 | | `extract()` small.tar | 188 ms | 175 ms | 0.93 | | `extract()` small.tar.gz | 137 ms | 134 ms | 0.98 | | `extract()` big.tar | 252 ms | 243 ms | 0.96 | | `extract()` big.tar.gz | 277 ms | 281 ms | 1.02 | | `new Bun.Archive({20,000 files}).bytes()` | 140 ms | 120 ms | 0.85 | | the same with `compress: "gzip"` | 210 ms | 170 ms | 0.81 | | `bun pm pack`, 20,000 files | 696 ms | 679 ms | 0.98 | | `bun install`, 1,191 packages, cold cache, local registry | 1006 ms | 1017 ms | 1.01 | Most of that time is zlib, system calls and JavaScript objects, which did not change. The readers alone, in one process at the same optimization level: walking an archive of plain headers takes 0.34 times as long as with libarchive, pax headers 0.53, a `.tgz` 0.61. Archives built to be expensive (millions of pax records or sparse regions, 256 MB of junk, a 1 GiB bomb under 24 gzip layers, a million empty gzip members) all take the same time as before or less. Anonymous memory is the same. The release binary is 64 KB to 130 KB smaller, depending on the target. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
### What does this PR do?
Removes `bytemuck` from the Rust dependencies. A small in-tree crate,
`bun_pod` (`src/pod/`, `#[no_std]`), replaces it, and `bun_core`
re-exports it, so call sites read `bun_core::cast_slice`,
`bun_core::bytes_of` and so on. The trait names are the same:
`NoUninit`, `AnyBitPattern`, `Pod`, `Zeroable`. `src/pod/README.md` is
the crate's documentation and holds its doctests.
#### What is better than bytemuck
- **Trait impls are proven by the compiler.** `impl_no_uninit!`,
`impl_any_bit_pattern!`, `impl_pod!` and `impl_zeroable!` are safe
macros. Each names a function whose only content is a
`core::mem::TransmuteFrom` bound, so rustc proves the claim from the
type's real layout on the target being compiled. Padding, a `bool`, an
enum or a pointer in the wrong place is a compile error at the macro
call. Every `unsafe impl bytemuck::*` is gone, and most `unsafe impl
Zeroable` for libc and Win32 structs are proven too.
- **An empty slice always casts.** bytemuck checked the alignment of the
dangling pointer of an empty `&[u8]` and panicked.
- **Panics say more.** They carry the caller's location
(`#[track_caller]`) and the address or length at fault. The panic
function is not generic, so there is one copy.
- **One `Zeroable` instead of three** (bytemuck's, `bun_core::ffi`'s,
and a private copy in the Windows shim).
- **Casts to bytes are `const fn`.**
- **Release code is the same.** In release assembly `slice_as_bytes`,
`bytes_of` and `cast_slice::<u16, u8>` fold into the same function as
bytemuck's versions, and the hot path of `cast_slice::<u8, u16>` is the
same eight instructions.
#### A crash this fixes
An `ArrayBuffer` can sit at any address (`bun:ffi` `toArrayBuffer`,
N-API external buffers). Printing a typed array wider than a byte over
one at an odd address panicked:
```js
import { ptr, toArrayBuffer } from "bun:ffi";
const bytes = new Uint8Array(64);
bytes.buffer;
console.log(new Uint16Array(toArrayBuffer(ptr(bytes) + 1, 0, 8)));
// 1.4.4: panic: cast_slice>TargetAlignmentGreaterAndInputNotAligned
```
`console.log`, `Bun.inspect`, the `expect()` diff printer and
`Bun.unsafe.arrayBufferToString` now read such memory through
`Unaligned<T>` (`Unaligned::slice_of`, `String::clone_utf16_bytes`).
`slice_of` has no alignment to check and leaves out a trailing partial
element, so it cannot fail. `ArrayBuffer::as_u32`, which `node:zlib`
uses for its write state and params, returns `&mut [Unaligned<u32>]`;
before, it formed a `&mut [u32]` behind a debug assertion only.
#### Shapes that no longer need a cast
- Sinks take UTF-16 text as `write_utf16(&[u16])`. One caller turned
`&[u16]` into bytes, and seven sinks cast it back with a check that
could panic. The fetch and S3 sinks share one `EncodedChunk` enum
(bytes, Latin-1 or UTF-16), now in `streams.rs`.
- The lexer keeps the raw string literal in one enum
(`StringLiteralRaw`) instead of bytes plus a format tag.
- Code generic over `u8`/`u16` dispatches through `PathByte` methods
(`units()`, `index_of_unit()`, `last_index_of_unit()`) instead of
`cast_slice::<T, u8>` behind an `IS_U16` check. This also replaces
`strings::Lanes` and `paths::string_paths::Ch`.
- `write_any_to_hasher` takes any `NoUninit` value. `AsBytes` and two
local copies of the function are gone.
- `BinLinkingShim::encode_into` writes into `&mut [u16]`, which removes
two `unsafe` blocks and the aligned wrapper struct in its caller.
- The zlib, brotli and zstd contexts return `write_result() ->
[avail_out, avail_in]`. The out-parameters existed to write straight
into the JS `Uint32Array`, which is no longer a `&mut [u32]`.
- `parse_hex_to_int::<T>` was only used with `u64` and is now
`parse_hex_to_u64`. `SignalCode::from::<T>` had one caller, which passed
a `u8`, and is gone.
#### Functions that were safe in name only
These took `T: Copy` (or nothing) and reinterpreted bytes. They now
carry the bound that makes them safe:
- lockfile `read_array<T: AnyBitPattern>` / `write_array<T: NoUninit>`.
The no-op `assert_no_uninitialized_padding` is deleted, and `read_array`
no longer depends on the alignment of the buffer.
- the manifest cache's `read_array` / `write_array`. `NpmPackage` is now
`Pod`, which replaces its hand-written `offset_of!` asserts. For that,
`has_extended_manifest` is a `u8` with a `has_extended_manifest()`
accessor. The on-disk layout is unchanged.
- `exe_format::read_struct` / `write_struct`,
`FixedBufferStream::read_struct`, Mach-O `LoadCommand::cast`.
- The Mach-O `LoadCommandIterator<'a>` borrows its buffer and implements
`Iterator`. It used to hold a raw pointer (`RawSlice`) so that two loops
in `macho.rs` could write to the same `Vec` while walking it, one of
them through a pointer derived from a shared borrow. Those loops now
walk owned ranges (`command_ranges()`) and edit through slices. The
walker has no `unsafe` left, and the compiler rejects the old pattern.
#### What to know before merging
- **One more nightly feature:** `#[feature(transmutability)]`, enabled
only in `bun_pod` and used by three functions in its `proof` module.
Crates that call the macros do not need it.
- **The proof covers soundness, not field order.** rustc accepts a
`repr(Rust)` struct when the field order it picked leaves no padding,
and the bytes then follow that order (`{ a: u8, b: u32, c: u8, d: u16 }`
comes out as b, d, a, c). A `macro_rules!` macro cannot see a `repr`, so
a new source lint, `test/internal/source-lints/pod-impl-repr.test.ts`,
requires a `#[repr(..)]` on every type named in the three byte-view
macros. All current types have one.
- **The proof does not know library invariants.**
`impl_any_bit_pattern!` accepts a wrapper around a type whose private
fields carry an invariant, even a type from another crate. bytemuck's
derive closes this with a bound on each field, which needs a proc macro.
No type in the tree is affected, and the README says not to do it ("What
the proof leaves to you").
- **What the proof cannot analyze is a compile error, never a wrong
answer.** It cannot analyze pointers, generic parameters, or a field
whose type is an associated type (this includes `bitflags!` storage).
- **Types over 4 KiB are refused.** The analysis recurses once per byte,
and rustc segfaults somewhere between 16 and 32 KiB. The macros fail
with a message before the analysis starts.
- **`bun_pod` has one feature, `alloc`,** for `vec_from_bytes`.
`bun_core` turns it on. The freestanding Windows shim, which is built
with `core` alone, does not.
- **Some `unsafe` stays, with a comment saying why.** `VersionType`,
`ExternalSlice<T>`, `EventListBytes`, `Dict` and the structs with
pointer fields keep an `unsafe impl`. `Offsets` and `PackageVersion`
keep their `unsafe` byte views (`PackageVersion` holds `bin::Value`, a
union whose arms differ in size).
### How did you verify your code works?
- **New tests.** The crashing inputs run in a spawned child, so an
unfixed build fails one named case each.
- `test/js/bun/util/inspect.test.js` "TypedArray over an ArrayBuffer at
an odd address prints" (all nine typed arrays wider than a byte) and
`test/js/bun/util/unsafe.test.js` "arrayBufferToString u16 at an odd
address" fail on Bun 1.4.4 with the panic above and pass on this branch.
- Two tests pass on 1.4.4 as well and guard the new code instead.
`test/js/node/zlib/zlib.test.js` "native streams take a write state and
params at an odd address" covers the `Unaligned<u32>` path in zlib,
brotli and zstd. A release build of 1.4.4 skipped the debug assertion
there and did unaligned stores. `unsafe.test.js` "arrayBufferToString
u16 of nothing" covers the empty-slice cast.
- **`bun_pod`.** 12 unit tests and 12 doctests, 7 of them `compile_fail`
(padding, an uneven union, a pointer, an enum, a `bool`, a non-zero
enum, an oversized type). The crate is added to `MIRI_CRATES`, so CI
runs all of them under Miri, and `rust-lints.yml` now also runs when
`src/pod/README.md` changes.
- **Mach-O output.** `bun build --compile --target=bun-darwin-arm64`
with the 1.4.4 darwin binary as the template and payloads of 20 B, 300
KB and 5 MB gives a signed file byte-identical to what Bun 1.4.4 writes.
Five malformed templates (zero, tiny and huge `cmdsize`, too many
`ncmds`, a short `LC_SYMTAB`) give the same result on both: four fail
with the same error and one gives identical output.
`compile-macho-codesign.test.ts` passes.
- **Type checks.** `cargo check --workspace` for Linux (through clippy,
`--all-targets`), Windows x64 and macOS arm64 on this commit, plus the
standalone Windows shim without `alloc`. FreeBSD, Android and musl
passed on an earlier state of the branch, and CI built all 12 targets on
the first commit.
- **The new lint.** It has a self-test with four synthetic sources, and
removing `#[repr(C, packed)]` from `SettingsPayloadUnit` makes it fail
with that type's name.
- **Lints.** `bun run rust:clippy`, `cargo fmt --check`, prettier and
`test/internal/source-lints/` pass.
- **Test suites on the debug build.**
- On this commit: `inspect`, `unsafe`, zlib (three files), `node:path`,
`test/js/bun/sourcemap`, `node:module` sourcemap, and the dev server's
sourcemap test.
- On the previous commit: `bun-lockb`, `migrate-bun-lockb-v2`,
`arraybuffersink`, `filesink`, `body-stream`, `fetch-abort-stream-body`,
`s3-networksink-leak`, `s3-connection-close`, `compile-corrupted-embed`,
`bun-install-cache-publish`.
- On earlier builds of the change: `blob`, `streams`, `s3`, `serve`,
`node-http2`, `buffer`, `transpiler`, `transpiler-cache`, `yaml`, `xml`,
`bundler_compile`, `bun-install-patch`, `html-rewriter`.
- **Failures seen locally.**
- These also fail on Bun 1.4.4 in the same container: the HTMLRewriter
"upstream body fails" group (no IPv6 there), some `node-http2` tests,
two S3 credential tests and the root-port test in `serve.test.ts`.
- Two tests ran past a 5 s limit on the debug build, and I did not
compare either with a baseline. `xml.test.ts` "deep values are a
catchable error" took 7 s. The child test in
`pretty-format-overflow.test.ts` took 5.3 s. Neither reaches code this
PR changes, and both files passed in CI on the first commit.
- `bun-build-compile.test.ts` did not finish within 280 s on the debug
build.
- **Compared with Node and Bun 1.4.4.** `node:path` `join`, `normalize`,
`resolve`, `relative` and `format`, posix and win32, on inputs from 1 to
70,000 segments (every pool tier and the heap spill) give the same 30
lines on all three. Streaming 200 KB through deflate, brotli and zstd
with a 64-byte `chunkSize` gives the same output and byte counts as
1.4.4. An already-transpiled UTF-16 source with two `sourceMappingURL`
comments remaps through the last one, as on 1.4.4.
- **By hand on the debug build.** `bun install` treats a manifest cache
file cut to 0, 1, 40, 60, 100, 150 or all but one of its 664 bytes as a
cache miss and installs. ASCII, Latin-1, UTF-16 and byte chunks through
a streaming `fetch` body and an S3 `writer()` arrive as the right UTF-8.
A failing `toEqual` prints an odd-address `Uint16Array` in its diff.
UTF-16 text through `ArrayBufferSink` and a direct `ReadableStream`, a
UTF-16 BOM `Blob` at even and odd offsets, `ucs2` writes to an odd
offset, and JSX entity strings all give the expected output.
- **Not run locally.** The crash handler's use of the load-command
iterator (macOS only) and the Windows-only code paths (the shim encoder,
`resolve_windows_t`'s drive-cwd lookup, the `PathBuffer` wide views)
were type-checked only. On the first two commits, CI passed every
Windows test job and one of the two macOS arm64 test jobs. The other
macOS arm64 job timed out both times downloading its build artifact,
before any test ran, and the two macOS x64 jobs never got an agent.
---
_Generated by [Claude Code](https://claude.ai/code)_
---------
Co-authored-by: Claude <noreply@anthropic.com>
### What does this PR do? Moves `WEBKIT_VERSION` from `982277c9dd3e` to `191c6a41380b`, the head of oven-sh/WebKit `main`. > [!NOTE] > The release `autobuild-191c6a41380bb1e2e54be4b6e50ecda83b189f96` was published at 10:39 UTC on 2026-10-27, after this went up. Builds #133617 and #133628 ran before that and failed where they download WebKit (HTTP 404). They say nothing about the change. The PR needs one new build. What the range brings: | Fork PR | What it changes | | --- | --- | | oven-sh/WebKit#912 | The ahead-of-time compiler (890 of the 942 files). Bun does not use it. Outside `aot/`: B3, the marked block cage and its reserve, `Interpreter::getStackTrace`, `ProxyObject`, `JSObject::putDirect`. | | oven-sh/WebKit#916 | The hardened Linux arm64 lanes are compiled without `-mtune`, so that glibc's `_FORTIFY_SOURCE` wrappers are inlined. | | oven-sh/WebKit#917 | The buffer of an old `WeakMap` or `WeakSet` that has shrunk counts until the next full collection. | The pin is the only change Bun needs: - `Heap::didChangeVisitedExtraMemory` is gone. Nothing in Bun calls it. - `Interpreter::getStackTrace` has one more parameter, at the end and with a default. Bun's five callers compile as they are. - The empty table of built-ins in `RemovedFeature.cpp`, which the debug and ASAN builds link: the range adds or removes no `builtinTable*` symbol, no entry of the two list macros and no `extern "C"` hook for the embedder. `BuiltinTable::typedArrayTypeOf`, which is new, is defined in the engine. - No asset is renamed in `lanes.mjs`, and no toolchain pin moves. - `BytecodeList.rb` is unchanged. The one change in the bytecompiler is for private names in object literals, which only built-in code can write, so the hashes in `bundler_bytecode_portable.test.ts` should stay. ### How did you verify your code works? Nothing was built. The list above is from reading the diff of the two commits against Bun's sources. The release has the same 77 asset names as the one of the old pin, all uploaded, and among them the 13 that the `build-bun` jobs of #133628 asked for. All 85 jobs of the fork's build of this commit passed, its JavaScriptCore tests on Linux x64 and arm64 included. `test/js/bun/jsc/webkit-upgrade-191c6a4138.test.ts` covers oven-sh/WebKit#917: | Binary | Result | | --- | --- | | `main`'s CI build (92e819d, linux x64, the old pin) | 0 pass, 2 fail: the heap is 262,032 bytes (`WeakMap`) and 129,498 bytes (`WeakSet`) smaller after the eden collection | | Bun 1.4.4 (an engine from before the bug) | 2 pass | Both the same with `BUN_GARBAGE_COLLECTOR_LEVEL` 0, 1 and 2. It has not run on the new engine yet: CI is the first run. `bun test test/internal/source-lints/` passes (186). --- _Generated by [Claude Code](https://claude.ai/code/session_01YF5TQxMfo4rBWHbttbVAhn)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
### What does this PR do? Consolidates the open pull requests on the CSS parser (`src/css`: parsing, minifying, lowering, printing, CSS modules) into one. 179 open pull requests were in scope. Each was judged by one rule: **the output must mean to a browser what the input meant.** Every claim was run through a build of `main`, a build with the change, lightningcss 1.33 and Chromium 156 (CSSOM, computed values, painted pixels for colors and gradients), and counted against 10,116 real stylesheets: the published CSS of about 190 popular npm packages, plus stylesheets from open source repositories. 107 land here, whole or in part, most of them cut down. The source diff is net negative. Of the 10,116, 9,509 are stylesheets that someone ships. The rest are test fixtures of other tools (521), Tailwind v4 source that needs its plugin (59) and SCSS, PostCSS or template source (27). | 9,509 shipped stylesheets | `main` | this branch | |---|---|---| | fail to build | 71 | 35 | | ... of those, valid CSS that Bun rejects | | 0 | | minified output reads differently in Chromium than the input | 552 | 2 | | CSS modules with a name that is referenced but not defined, or exported but not defined | 166 | 0 | | all 10,116 | `main` | this branch | |---|---|---| | fail to build | 192 | 141 | | minified output reads differently in Chromium than the input | 593 | 15 | | build on `main`, fail here | | 0 | | right on `main`, wrong here (minified, and at the default targets) | | 0 | | size of the minified output, no targets | 162.32 MB | 162.47 MB (+0.093%) | | size of the output at the default targets | 224.1 MB | 210.2 MB (-6.2%) | A syntax error is a build error, with one exception: the IE star hack `*zoom: 1` is skipped, because tachyons and handsontable ship it. The 35 shipped stylesheets that fail are invalid CSS: 28 have a selector that Chromium drops (semantic-ui-css, swiper, @ionic/core, Adobe Spectrum, xp.css), 2 an `@import` after other rules, and one each `@supports not X and Y`, a nested `@keyframes`, a stray comma, a selector list in `::slotted()`, `@media screen\0` (govuk-frontend). The 2 that read differently fold a `color-mix()` inside a gradient (see the known gaps). The growth of the minified output is rules and declarations that `main` deleted by mistake: 87 KB of rules, 45 KB of prefixed declarations that only browser targets can rule out. #### Popular CSS that `main` breaks today - normalize.css, modern-normalize, sanitize.css: `font-family: monospace, monospace` is collapsed to `monospace`, which changes the font size from 16px to 13px (oven-sh#46662). - CKEditor 5, Tabler, Carbon, VitePress, Ionic, PatternFly, Excalidraw: a rule of custom properties is deleted as a "duplicate" of a later rule with the same selector and other custom properties. 37 stylesheets lose 1,681 custom properties (oven-sh#33684). - Material Components, daisyUI, DocSearch, plyr: a `var()` size loses to its own fallback (oven-sh#42101). - nextra, daisyUI, CKEditor: declarations written after a nested rule are moved above it (oven-sh#45462). - antd 4, purecss, govuk-frontend: browser hacks in selector lists start to apply everywhere (oven-sh#42100). - foundation-sites, flatpickr, 98.css, blueprintjs, scalar, react-datepicker: the build fails on valid CSS (oven-sh#46756, oven-sh#38552, oven-sh#38534). tachyons, handsontable: it fails on the IE star hack (oven-sh#46751). - nodejs.org, Mantine: a CSS module exports classes that no rule defines, because the `@apply` or `@mixin` in them is moved out of the rule (oven-sh#46759). - hint.css and 42 more: `z-index: 1000001` and `1000000` become the same number (oven-sh#46715). - Adobe Spectrum: `--version: 5.0.1` prints as `5.1`. - A quarter of all stylesheets: minified output has a stray space in `align-items: center` (oven-sh#41993). #### Behavior changes to look at first 1. **`light-dark()` is kept as written at the default browser targets** (oven-sh#42009). The lowered form `var(--buncss-light, a) var(--buncss-dark, b)` is invalid unless a compiled `color-scheme` covers the element: 184 of the 215 real stylesheets that get it never define the variable, so every browser drops the declaration. The docs section on the polyfill is removed. 2. **`border` is not built from longhands** (oven-sh#42072): `border-width:0;border-style:solid;border-color:red` stays three declarations, because `border` also resets `border-image`. 305 real stylesheets grow by 30 KB in total; `bootstrap.min.css` by 113 bytes, 5 gzipped. No real stylesheet was found where the old merge shows, so this is the one to drop if size matters more. 14 existing expectations change. 3. **A CSS module no longer exports names that only occur inside `:global(...)`** (oven-sh#45487). 73 real modules lose 202 export keys, all of them names that no rule has. `import { g }` or `composes: g` of such a name becomes a build error. 4. **Container names and `@counter-style` names stay global in a CSS module** (oven-sh#45247 by @mikamikasuki). `main` hashes the `@container` name but not `container-name`, so a named query never matches. postcss-modules (Vite, css-loader, Next.js) keeps them global; lightningcss and esbuild hash both. Global keeps a container that is queried from another file or from JS working. 5. **Syntax errors are build errors, also two that `main` lets through**: a media query that stops short (`@media screen and {`, `@media not {`, `@import "a.css" screen and;`) and `@media (width <) foo {`. No stylesheet of the 10,116 has either. A condition that is valid by the grammar though no browser knows it, `( ... )` or `fn( ... )`, is kept as written (oven-sh#46756): `(min-width:0\0)`, `(not(hover))`. 6. **Declarations after a nested rule stay where they were written**, as a nested declarations rule (oven-sh#45462). Changes the output of 199 real stylesheets; in a fuzz of 2,850 nested programs compared by computed style, `main` was wrong in 1,056 and the pull request's code, before it was trimmed here, in none. 7. **`Bun.color`**: `color()` inputs are converted instead of returned as a string (`Bun.color("color(srgb 1 0 0)", "hex")` was `"color(srgb 1 0 0)"`); `currentColor`, system colors and `light-dark()` give `null` in the converting formats instead of the keyword as a string; a relative color or `color-mix()` outside sRGB is clipped, not gamut-mapped (`hsl(from oklch(100% .399 336.3) h s l)` is `#ff5bff`, was `#fff`). 8. The `:-moz-any()` and `:-webkit-any()` copies of rules with `:is()` are gone at the default targets (oven-sh#45945, oven-sh#42029). The `:-webkit-any()` copy outranked the original rule: `.title{color:blue} :is(h1,h2,h3){color:red}` painted red. 5.8% of all default-target output was the `:-moz-any()` copies. 9. `Property` grows from 216 to 224 bytes: `Url` stores its text as a slice so that two `url()` values can be compared (oven-sh#46895). The pull request packed a pointer and a `u32` behind `unsafe` to avoid that. #### Not in this PR, needs a decision - **Raise the default browser targets (oven-sh#40369) and add a `cssTarget` option (oven-sh#40368).** Today: Chrome 80, Edge 80, Firefox 78, Safari 14, Opera 67, and no public way to change them. At Vite 8's floor the output of the 9,924 stylesheets is 15.3% smaller (3.9% gzipped). About a dozen fixes here only matter at today's floor, and code for older targets is unreachable without the option. - Vendor prefixes for `user-select`, `appearance`, `hyphens` and five more (oven-sh#45814): 1,116 real stylesheets lack a prefix they need at the default targets. And for `backdrop-filter` (oven-sh#44648): 173. - Clip instead of gamut-map in the sRGB fallback of a plain wide-gamut color: deletes about 90 lines, changes every pinned fallback hex. If the mapping stays, oven-sh#44647 by @oddharsh makes it closer to what browsers paint (colors off by 2/255 or more: 168 to 110 of 3,170 real ones). #### Changes **Rules: duplicates, merging, nesting, fallbacks** - The `:-moz-any()` copy of every rule with `:is()` or `:not(a, b)` is gone: Firefox 78 has `:is()` (oven-sh#45945). 1,164 real stylesheets - Keep declarations written after a nested rule in place with a nested declarations rule (oven-sh#45462). 379 real stylesheets - With nesting compiled away, an unknown at-rule inside a style rule was moved out of it: `.wrapper{@apply flex}` printed a bare `@apply flex;` and the rule was gone, so a CSS module exported a class that no rule defines. It now stays in a block under the compiled selector. 72 real modules with Tailwind's `@apply`, 25 with Mantine's `@mixin` (oven-sh#46759). 316 real stylesheets - Keep !important in the rules generated for logical properties and color-scheme (oven-sh#38597). 311 real stylesheets - With browser targets, remove an earlier duplicate rule only when its declaration block is equal (oven-sh#46861). 171 real stylesheets - Compare custom property names in `PropertyId` equality and importance in duplicate-rule detection (oven-sh#33684). 40 real stylesheets - Keep same-condition @supports rules and empty named @layer blocks (oven-sh#46733). 31 real stylesheets - Keep target-added vendor prefix passes on cloned and re-minified rules, and never print a bare `:full-screen` (oven-sh#43339). 12 real stylesheets - Keep the selectors on rules added for color fallbacks and logical properties (oven-sh#43240). 3 real stylesheets - A later declaration of a property drops the `@supports` color copies of the earlier one, which printed after the rule and won (oven-sh#45560). 2 real stylesheets - Do not merge style rules in a stylesheet that holds `revert-rule` (oven-sh#46953). 1 real stylesheets - `@import ... supports(-webkit-mask: none)` kept its prefix (oven-sh#38589) **Properties and shorthands** - Print align-items:center without a stray space when no safe/unsafe keyword is given (oven-sh#41993). 3,317 real stylesheets - Emit the physical longhands instead of the inset shorthand for targets without it (oven-sh#38583). 1,452 real stylesheets - Keep the stops of an upward gradient as written, and drop a `50%` hint only at the midpoint (oven-sh#46790). 104 real stylesheets - Emit unparsed border-block-* declarations when compiling logical borders (oven-sh#38582). 89 real stylesheets - Keep repeated font families and dedupe only the inserted system-ui fallbacks (oven-sh#46662). 88 real stylesheets - A `width` / `height` / `min-*` / `max-*` with `var()` or `env()` was printed before the plain fallback written above it, so the fallback won (oven-sh#42101). 26 real stylesheets - Keep box-orient and box-direction when flex-direction or flex-flow follows (oven-sh#45672). 16 real stylesheets - Write held prefixed background declarations at every flush (oven-sh#46882). 6 real stylesheets - Derive equality for Size2D and BorderRadius, and keep every prefix in the border-radius handler (oven-sh#45665). 2 real stylesheets - Print `none` instead of an empty border-image, mask-border or mask-box-image value (oven-sh#45143). 2 real stylesheets - Keep font/transition sub-properties on the side of the shorthand they were written on (oven-sh#41990). 2 real stylesheets - CSS-wide keywords are not names: `font:12px Arial;font-family:inherit` printed `font:12px inherit`, `color-scheme:inherit` and `color-scheme:auto` printed `normal`, `transition-property:unset` was folded into `transition` (oven-sh#42022). 2 real stylesheets - Only fold font longhands into `font` when the rule declared a `font` (oven-sh#45459) - Keep a font family name quoted when its identifiers do not read back as the same name (oven-sh#46606) - Stop rewriting valid positions into ones browsers reject (oven-sh#45197) - `border` is no longer built from longhands unless the rule also sets what `border` resets (`border-image`), and an author's `border-image` reset is no longer dropped. `transition` is likewise only written where the rule wrote one, because it resets `transition-behavior` (oven-sh#42072) - `border; border-image: <gradient>; border-image-slice: 1` lost the gradient in every default build (oven-sh#42018) - A prefixed `linear-gradient` keeps its direction, written or generated: the prefixed syntax names the starting side and measures angles differently (oven-sh#46697) **At-rules** - Print @font-face format() keywords as strings (oven-sh#37979). 439 real stylesheets - Print @supports `not` conditions without a leading space (oven-sh#38559). 88 real stylesheets - Reset after the number token of a unicode-range (oven-sh#45835). 33 real stylesheets - Keep `<general-enclosed>` media and container conditions as written (oven-sh#46756). 24 real stylesheets - A strict range such as `(width < 768px)` lowered for old targets still matched at exactly 768px. The step is now `.02px`. The `not all and` form for other units is not taken (oven-sh#46806). 18 real stylesheets - Keep scanning an @supports condition after a nested block (oven-sh#38552). 9 real stylesheets - Every `@counter-style name` printed as `@counter-stylename`, which browsers drop (also oven-sh#46692). In a CSS module the name now stays global, like the `list-style` that refers to it (oven-sh#33679). 9 real stylesheets - Keep a keyframe whose block has an invalid item (oven-sh#46288). 2 real stylesheets - Read an @Property initial-value on the stylesheet's parser, keep an unmatched value as written (oven-sh#45806). 2 real stylesheets - `@media {}` is `all`. A media query that stops after `and`, `or` or `not` (`@media screen and {`) is a build error; `main` printed `not all` or dropped the condition, silently. The `@layer` and type selector parts are not taken (oven-sh#46869). 2 real stylesheets - Accept a name-only `@container` and a `style()` feature with no value (oven-sh#46689). 1 real stylesheets - Parse `<timeline-range-name> <percentage>` keyframe selectors (oven-sh#45652) - Allow `or` inside parentheses after a media type (oven-sh#45088) - Print the `=` comparison in media and container queries as `=` (oven-sh#45071) - Parse the @font-palette-values font-family descriptor as a list of family names (oven-sh#45484) **Selectors** - Never wrap a selector list in :is() (oven-sh#46787). 414 real stylesheets - Stop emitting :-webkit-any() copies of :is() that outrank the original rule (oven-sh#42029). 373 real stylesheets - With browser targets, a selector list with a vendor-prefixed member was split, which revives hacks that rely on other engines dropping the whole list: `.s *::-ms-backdrop,.s{flex:auto}` applied everywhere. Rewritten; the list is printed as written (oven-sh#42100). 42 real stylesheets - Print the vendor prefix on downleveled :any-link, :read-only, :read-write, :placeholder-shown and :autofill (oven-sh#42131). 12 real stylesheets - Fix nesting lowering of a `&` that does not lead its selector (oven-sh#43047). 8 real stylesheets - `@scope`: `&` and implicit selectors compile to `:where(:scope)`, relative selectors and bare declarations parse (oven-sh#43165). 7 real stylesheets - Only reject bare :global / :local pseudo-classes in CSS modules (oven-sh#38534). 5 real stylesheets - Stop truncating `@scope to (...)` and stop replacing `&` in the scope-end selector (oven-sh#43146). 2 real stylesheets - `[xl|href$=".pdf"]` lost its namespace prefix (oven-sh#45159). 2 real stylesheets - Match An+B keywords exactly and take only `+` as the leading sign (oven-sh#42953). 1 real stylesheets - Parse ::details-content and ::picker() like ::part() (oven-sh#45504). 1 real stylesheets - Parse :window-inactive as an ordinary pseudo-class, and accept it after ::selection (oven-sh#45815) - Print :nth-col(1) and :nth-last-col(1) in functional form (oven-sh#41575) - `li::before::marker` failed the build (oven-sh#45810) - `&` inside `:nth-child(... of S)` is resolved when nesting is compiled away. The second selector serializer (326 lines) is deleted (oven-sh#43024) **Colors and `Bun.color`** - Keep light-dark() as written under the default browser targets (oven-sh#42009). 227 real stylesheets - Number and percentage channels parse in `lab()`, `lch()`, `oklab()`, `oklch()`, `hsl()` and `hwb()`; the hue of `lch` / `oklch` is no longer scaled by alpha (oven-sh#45084). 130 real stylesheets - `Bun.color` returns `null` for `currentColor`, system colors and `light-dark()` in the converting formats, instead of the CSS keyword as a string (oven-sh#33047). 130 real stylesheets - Relative colors: channel keywords are on the scale of their function (`rgb(from oven-sh#369 calc(r + 10) g b)`), the origin's alpha is kept, a relative color kept as written gets no `@supports` copies, and a result outside sRGB is not folded (oven-sh#45362). 31 real stylesheets - Keep rotate: 0deg out of none, and keep the origin alpha in relative colors (oven-sh#38486). 13 real stylesheets - Compute the D50 reference white in f32 so greys convert to lab(l 0 0) like lightningcss (oven-sh#38535). 7 real stylesheets - Return a missing hue for a grey from conversions into lch, oklch, hsl and hwb (oven-sh#45201). 5 real stylesheets - Fix color(a98-rgb), srgb-linear relative colors and color-mix(in xyz-d50) (oven-sh#38487). 4 real stylesheets - Match named colors without regard to ASCII case (oven-sh#45092). 3 real stylesheets - Fold min(), max() and clamp() over plain numbers (oven-sh#39515) - `calc(l - .08)` in a percentage channel of a relative color became NaN and printed `#000` (also oven-sh#45970) (oven-sh#45971) - Bun.color: convert color() inputs instead of returning them as a CSS string (oven-sh#38488) - A `color-mix(in srgb, ...)` result outside sRGB prints `color(srgb ...)` instead of a gamut-mapped hex (oven-sh#39269) **`calc()` and numbers** - Rounding to 6 significant digits ate into the integer part: `z-index:2147483647` became `2147480000`, `max-width:9999999px` became `10000000px`, `--x:1234567.5` became `1234570`. It now only cuts a fraction short: a value kept as tokens prints its source text, a typed value prints every integer digit and at least one digit of the fraction (oven-sh#46715). 43 real stylesheets - A number without a unit is no longer repaired into `px` (`height:1.5715` is dropped by a standards-mode browser, so it prints as written), `calc(-1 * 0)` is not a length, and on a property of the range [0,∞] (sizes, border radii, padding, border and outline widths, gap, `flex-basis`, `font-size`) a `calc()` that folds below zero prints `0` instead of an invalid negative literal, and a negative literal no longer replaces the valid declaration before it (`width:10px;width:-5px` printed `width:-5px`) (oven-sh#42104). 24 real stylesheets - Print numbers from the f32's shortest digits, not its f64 expansion (oven-sh#38565). 17 real stylesheets - Print negative numbers below 1 with a single minus sign (oven-sh#38516) - Keep math functions over two different length units as written (oven-sh#38489) - Fold rem() with a truncated quotient, so it keeps the sign of the dividend (oven-sh#32286) - A one-argument `min()` / `max()` around a sum printed without any function (`width:100% - 20px`) (oven-sh#46145) - `round()` breaks ties upward and handles a negative step as browsers do (oven-sh#45249) **CSS modules** - A CSS module exports its `@keyframes` names (`styles.spin`). Hashing is unchanged (oven-sh#46720). 145 real stylesheets - A name inside `:global(...)` prints plain but was exported as `name_<hash>`, a class no rule has (also oven-sh#38573) (oven-sh#45487). 90 real stylesheets - Keep container names global in CSS modules (oven-sh#45247). 6 real stylesheets - Tag the keyframes name when an `animation` value falls back to raw tokens (oven-sh#46034). 6 real stylesheets - In a plain stylesheet `composes: a from "./b.css"` was treated as an import: the file was inlined and an absolute path printed (oven-sh#45961). 4 real stylesheets - Drop a rejected composes instead of failing the file at print time (oven-sh#38538). 1 real stylesheets - Keep every rule's custom properties for the composes conflict check (oven-sh#38523) - Keep composes working after a nested rule in the same block (oven-sh#38520) - `IdentOrRef::hash` read 2 of 16 bytes, so in release builds every class of a module hashed alike (oven-sh#30784) **Tokens, `url()`, diagnostics** - The block skip stack no longer allocates on every block exit (oven-sh#39533) - Report the diagnostic's line without its line breaks in position.lineText (oven-sh#38342). 192 real stylesheets - Compare `url()` values by their text (oven-sh#46895). 26 real stylesheets - Values kept as tokens keep the blanks they were written with instead of having them re-invented at print time: `attr(x type(<number>))` printed `type(< number> )`, which browsers drop. Minified output still drops the blanks that cannot matter (oven-sh#45577). 19 real stylesheets - The IE star hack (`*zoom:1`) failed the whole build. The declaration is skipped, as every browser does. It is the one syntax error that builds: tachyons, handsontable and jasmine ship it (oven-sh#46751). 8 real stylesheets - The path of a copied asset spliced into `url()` is always quoted, so a file name with a space or a parenthesis cannot end the `url()` (oven-sh#41557). 7 real stylesheets - Move the skip loop of parse_until_before out of the generic body (oven-sh#39520) - Remove dead code from bun_css: the unused `impl Default for Property` (oven-sh#45618) - Default the at-rule and qualified-rule parser traits, drop the url_to_css shim (oven-sh#31999) **Speed** Four inputs that took quadratic or exponential time, or produced output out of proportion, are now linear, with no new limit and no extra pass (oven-sh#31639, oven-sh#42243, oven-sh#38531, oven-sh#43421). **Tests only** - `css-fuzz.test.ts` never ran in CI and timed out on debug builds; it is a deterministic table now (oven-sh#38500) - `nth-anplusb-ident.test.ts` counts heaps in a child process (oven-sh#46331) - `color.test.ts` checks the ansi-16 sweep with one `expect()` (oven-sh#42824) - Not CSS, but it failed this PR's CI and fails on `main` too: `ws-proxy.test.ts` on Windows, "a message sent during a TLS renegotiation arrives after it". The helper server now renegotiates from the callback of the frame's write (oven-sh#46170) **No pull request covered these** - `--v: 5.0.1` printed as `5.1`, `--w: 1.0.0` as `10`, `--x: 1+2` as `12`, `2n+1` as `2n1`: a number token that touches another keeps its source text. - `color-scheme: auto` printed as `normal`, and `color-scheme: dark light` as `light dark`. - A bare `:global` or `:local` in a CSS module sets the scope of the names after it, up to the end of the selector, as in css-loader. It was a build error. - The scrollbar states (`a:horizontal`, `::-webkit-scrollbar{&:horizontal{}}`) are valid in a plain compound, as in WebKit and Blink. - Without browser targets, `-webkit-box-align`, `-ms-flex-align` and the like were dropped next to their standard form (flatpickr, foundation-sites, materialize-css: 28 stylesheets), and `-webkit-fill-available` before `-moz-available` (beercss). - `(-o-min-device-pixel-ratio: 2/1)` printed as `2`. - `sin()`, `cos()` and `tan()` are exact at multiples of 90deg: `calc(100px * cos(90deg))` printed `-.00000437114px`. - A `display-p3` color at targets that have P3 but not the other `color()` spaces (Safari 10.1 to 14) got itself as a fallback. - `width: rem(-18px, 5px)` folds to a negative literal once `rem()` is right; it prints `0`. - Consecutive nested declarations rules, which arise when same-query `@media` blocks in one rule merge, are folded into one and count against the existing selector expansion limit as the `&{}` rules they replace did. #### Known and not fixed here - Four longhands are still folded into a `transition` the author did not write where the targets add a prefixed copy of a property (`mask-image` at Chrome 80): only the shorthand gives the copy its own times. - A later `background`, `box-shadow` or `border-*` declaration still loses to the `@supports` copy of an earlier `var()` value of the same property with a wide-gamut color. Same on `main`; oven-sh#45560 is taken for the properties of the fallback handler and for unparsed values only. - A merged rule prints the unprefixed name in the passes of the rule it absorbed: `.d::-webkit-input-placeholder{}.d::-moz-placeholder{}` prints `::placeholder` for the second (oven-sh#42106, not taken: it breaks the merge of prefix-equivalent rules). - The `border`, `outline` and `font` shorthands, `line-height`, `scroll-padding`, `background-size` and a few more still print a `calc()` that folds below zero as a negative literal. No real stylesheet has one. - `all` is not a property Bun knows, so `a{margin-top:1px;all:unset}` prints `all` first. Same on `main`. - A selector kept as raw tokens loses the blank before `*` when minified: `--sel: .dark *` prints `.dark*`. Same on `main`. - `width:calc(0% + 12px)` folds to `12px`, which lays out differently in a table cell. Same on `main`. - The `:dir()` / `:lang()` rules of a lowered logical property print before the nested rules of their rule, so a logical declaration written after a nested rule of equal specificity still loses to it. Same on `main`. - A fraction is still rounded to 6 significant digits (`12345.678` prints `12345.7`, `33.3333333333%` prints `33.3333%`). Keeping every digit would add up to 282 KB over the corpus. - `@keyframes "quoted"` is hashed in a module but not exported. - In a module, a keyframes name inside a `var()` fallback is not hashed: `animation: var(--enter, fade 1s)`. Same on `main`. - A `color-mix()` or relative color folded to a hex inside a gradient changes the interpolation from Oklab to sRGB (VS Code, clawhub). Printing `color(srgb ..)` instead needs a fallback for `mask-image`, `list-style-image` and custom properties at the default targets, which have none. - A comma list in `@container` still fails the build. - After a bare `:global`, names that are hashed at print time (`::view-transition-old(foo)`) are still hashed; inside `:global(...)` they are not. #### Pull requests in scope that do not land here - Duplicates of a pull request that lands here: oven-sh#36961 (of oven-sh#37979), oven-sh#38478 (of oven-sh#46662), oven-sh#38480 (of oven-sh#46733), oven-sh#38482 (of oven-sh#46715), oven-sh#38491 (of oven-sh#38500), oven-sh#38508 (of oven-sh#45201), oven-sh#38513 (of oven-sh#45362), oven-sh#38573 (of oven-sh#45487), oven-sh#38640 (of oven-sh#38500), oven-sh#45525 (of oven-sh#45247), oven-sh#45589 (of oven-sh#46756), oven-sh#45970 (of oven-sh#45971) - Already fixed on `main`: oven-sh#38674 - Right, but for input nobody writes (0 of 10,116 stylesheets, and not new syntax): oven-sh#31916, oven-sh#38526, oven-sh#38592, oven-sh#39349, oven-sh#41139, oven-sh#41801, oven-sh#42042, oven-sh#42484, oven-sh#43078, oven-sh#45604, oven-sh#45908, oven-sh#46086, oven-sh#46221, oven-sh#46224, oven-sh#46273, oven-sh#46694, oven-sh#46707, oven-sh#46737, oven-sh#46755, oven-sh#46816, oven-sh#46870, oven-sh#46934 - Right or harmless, but not worth their code, their bigger output or the behavior they change: oven-sh#30905, oven-sh#31924, oven-sh#33332, oven-sh#34426, oven-sh#36225, oven-sh#38006, oven-sh#38512, oven-sh#38547, oven-sh#38551, oven-sh#38561, oven-sh#38572, oven-sh#38576, oven-sh#38607, oven-sh#38623, oven-sh#38667, oven-sh#38692, oven-sh#42106, oven-sh#42912, oven-sh#42918, oven-sh#43407, oven-sh#43434, oven-sh#44700, oven-sh#45442, oven-sh#45546, oven-sh#45747, oven-sh#45759, oven-sh#45825, oven-sh#45946, oven-sh#46061, oven-sh#46127, oven-sh#46206 - Need a decision (see above), stay open: oven-sh#40368, oven-sh#40369, oven-sh#44647, oven-sh#44648, oven-sh#45814 - Stays open: oven-sh#45787 Out of scope: CSS linking and chunking in the bundler, the dev server, docs-only pull requests, and repository-wide sweeps that touch `src/css` in passing. Thanks to @mikamikasuki (oven-sh#45247), @alii (oven-sh#39520, oven-sh#31999) and @oddharsh, whose measurements on oven-sh#40369 and oven-sh#41990 were used. Fixes oven-sh#16727. Fixes oven-sh#24770. Fixes oven-sh#30772. Fixes oven-sh#44704. Fixes oven-sh#45089. Fixes oven-sh#45237. Fixes oven-sh#46872. ### How did you verify your code works? - The 35 CSS test files pass on a debug build, also with `FORCE_COLOR=1`. 600+ of the tests in this branch fail on a build of `main`. An earlier head of this branch passed all 179 CI jobs that ran. - The reproduction of each issue listed above was run on `main` (wrong) and on this branch (right). - Differential run over the 10,116 stylesheets, `main` against this branch, minified without targets and through `Bun.build` at the default targets: no stylesheet that builds on `main` fails; Chromium 156 compares the output with the input rule by rule (CSSOM, then computed values on a sized element in a standards-mode document, colors as painted pixels in sRGB and P3). The comparison has 195 control pairs that must read equal or different. For stylesheets that read differently on both builds, mostly because of lowering, the sets of differences were compared; the three new ones are two minifier fixtures that are wrong on both and one comparison artifact. - CSS modules are checked for consistency instead, since hashed names defeat the comparison: every reference points at a name the output defines, every export exists. - `Bun.color`: 59 inputs in 13 formats on both builds; every changed cell was read.
…e --aot` (oven-sh#45631) ```sh bun build --compile --aot ./app.ts ./app ``` This adds `--aot` to `bun build --compile`. It compiles TypeScript and JavaScript to machine code at build time. The executable starts as machine code and has no JIT in it. `bun compile ./app.ts` is short for the same command. It's still JavaScript. npm packages, CommonJS, `eval`, proxies and `any` all work. There's no new syntax and nothing to annotate. The new part is that the compiler reads your TypeScript types and uses them. ```ts class Vec { constructor( public x: number, public y: number, ) {} } function dot(a: Vec, b: Vec): number { return a.x * b.x + a.y * b.y; } ``` A `Vec` is a 16-byte header and two raw doubles. Where the compiler can prove that only `Vec`s reach `a.x`, the read is one load. No hidden-class check, no inline cache, no boxed number. Where it can't prove that, it tests once and keeps a slow path. It's experimental, and it [loses in places](#where-it-loses). The engine half is oven-sh/WebKit#757. The docs start at `docs/bundler/aot.mdx`. ## The TypeScript compiler, compiled TypeScript 6 is written in TypeScript. TypeScript 7 is the TypeScript team's port of it to Go. We built the `v6.0.3` tag from its own source with `--aot` and type checked [effect](https://github.com/Effect-TS/effect) with it, 370,839 lines. Not a line of TypeScript's source was changed. | | wall, s | peak RSS, MB | CPU, s | | --- | ---: | ---: | ---: | | TypeScript 7 (Go) | 2.68 | 1,256 | 10.8 | | **TypeScript 6, `--aot`** | **4.25** | **963** | **5.3** | | TypeScript 7 (Go), `--singleThreaded` | 5.08 | 933 | 6.2 | | TypeScript 6 on Bun 1.4.3 | 14.00 | 2,581 | 29.2 | `tsc -p` on effect's `packages/effect`, 0 errors in every row. Linux x64, AMD EPYC 9R14, pinned to 16 cores that nothing else uses. LTO build, median of five runs, 2026-10-24. The two TypeScript 6 rows are the same compiler at the same tag. Compiled ahead of time it's 3.3x faster than on Bun 1.4.3, in 2.7x less memory and 5.5x less CPU time. Go checks in parallel, which TypeScript 6 doesn't, and that wins on the clock. Take the parallelism away with `--singleThreaded` and the original TypeScript finishes first, on less CPU time and about the same memory. Both still collect garbage on other threads in that comparison. <details> <summary>How to run it yourself</summary> Bun 1.4.3, TypeScript 7.0.2 (`@typescript/typescript-linux-x64`), TypeScript 6.0.3 (tag `v6.0.3`, sources and `lib/` unmodified), effect at `8b0eac6a`, `@types/node` 26.6.4. ```sh git clone --depth 1 -b v6.0.3 https://github.com/microsoft/TypeScript git clone https://github.com/Effect-TS/effect && git -C effect checkout 8b0eac6ac46a689a7208500e2ccf9e3b80fc3898 npm install typescript@7.0.2 @types/node@26.6.4 # --aot type checks what it compiles, and a bare checkout lacks two of TypeScript's # dev dependencies (@types/node, source-map-support). This supplies them. echo 'declare module "source-map-support";' > missing.d.ts cat > aot-tsconfig.json <<X { "extends": "$PWD/TypeScript/src/tsconfig-base.json", "compilerOptions": { "rootDir": "$PWD/TypeScript/src", "types": ["node"], "typeRoots": ["$PWD/node_modules/@types"], "composite": false, "declaration": false, "declarationMap": false, "emitDeclarationOnly": false, "isolatedDeclarations": false, "noEmit": true }, "files": ["$PWD/TypeScript/src/tsc/tsc.ts", "./missing.d.ts"] } X bun build --compile --aot --production --format=esm --tsconfig-override=aot-tsconfig.json \ --define "__filename=\"$PWD/TypeScript/lib/tsc.js\"" --define "__dirname=\"$PWD/TypeScript/lib\"" \ TypeScript/src/tsc/tsc.ts --outfile tsc-aot go=node_modules/@typescript/typescript-linux-x64/lib/tsc run="taskset -c 0-15 /usr/bin/time -v" $run $go -p tsconfig.json --pretty false $run ./tsc-aot -p tsconfig.json --pretty false $run $go -p tsconfig.json --pretty false --singleThreaded $run bun TypeScript/lib/tsc.js -p tsconfig.json --pretty false ``` `tsconfig.json` is effect's own options for `packages/effect`, with absolute paths: ```json { "include": ["<here>/effect/packages/effect/src"], "compilerOptions": { "noEmit": true, "incremental": false, "composite": false, "rootDir": "<here>/effect/packages/effect/src", "target": "ES2022", "module": "NodeNext", "moduleDetection": "force", "verbatimModuleSyntax": true, "allowJs": false, "rewriteRelativeImportExtensions": true, "erasableSyntaxOnly": true, "strict": true, "exactOptionalPropertyTypes": true, "noUnusedLocals": true, "noUnusedParameters": true, "noImplicitOverride": true, "noFallthroughCasesInSwitch": true, "stripInternal": false, "skipLibCheck": true, "noErrorTruncation": true, "jsx": "react-jsx", "types": ["node"], "typeRoots": ["<here>/node_modules/@types"], "preserveSymlinks": true } } ``` </details> ## A real CLI Claude Code's CLI ships as a `bun build --compile` executable. This is the released build against the same app built with `--aot`, in a scripted session: start, first prompt, five turns, with the API stubbed out locally. | | released | `--aot` | | --- | ---: | ---: | | time to first prompt, ms | 318 | **141** | | peak private memory, MB | 185.4 | **44.7** | | private memory at the first prompt, MB | 112.0 | **28.7** | | private memory at the end, MB | 150.1 | **37.5** | | peak RSS, MB | 308.5 | **180.8** | | CPU time, ms | 1,542 | **512** | | instructions, millions | 5,530 | **1,117** | Linux x64, AMD EPYC 9R14, 16 cores that nothing else uses. LTO build, default flags, median of 16 sessions per column, alternating, 2026-10-24. CPU time and instructions count all threads. The first prompt is up in under half the time, on a third of the CPU time and a quarter of the private memory. ## Servers One small API, written seven times (`test/aot/servers/app/api-*.ts`): by hand on `Bun.serve` and on `node:http`, and on Hono, Elysia, Express, Fastify and Koa. It has three middlewares (a request id, response headers, a bearer token), route parameters, a query string, JSON in and out, an error handler and a 404. Each server is built twice by the same `bun`: with `--aot`, and with plain `--compile`, which runs it on the JIT. Same runtime, same built-in modules, same native code. Only the compiler differs. Requests per second in the first second after the process starts: | | `--aot` | JIT | ratio | | --- | ---: | ---: | ---: | | Koa | **33,109** | 17,180 | 1.93x | | Express | **46,562** | 24,181 | 1.93x | | Fastify | **57,976** | 37,822 | 1.53x | | `node:http` | **69,085** | 46,693 | 1.48x | | Hono | **58,978** | 48,029 | 1.23x | | `Bun.serve` | **84,599** | 77,867 | 1.09x | | Elysia | **63,556** | 61,560 | 1.03x | Requests per second once the JIT is warm, over requests 100,001 to 400,000 of the same process: | | `--aot` | JIT | difference | | --- | ---: | ---: | ---: | | Hono | 59,737 | 58,151 | +2.7% | | `Bun.serve` | 85,553 | 83,490 | +2.5% | | `node:http` | 70,205 | 70,383 | -0.3% | | Express | 47,935 | **51,587** | -7.1% | | Elysia | 65,110 | **75,268** | -13.5% | | Fastify | 59,656 | **71,796** | -16.9% | | Koa | 34,194 | **44,362** | -22.9% | The same program built twice differs by up to 1.9%, so we call anything under 3% level. That's the top three. The other four lose. Counting every request since the process started, the JIT has caught up within 2 seconds on Elysia and within 5 on Fastify and Koa. Express is still ahead at 5 seconds. Startup and memory: | | first response, ms, `--aot` | first response, ms, JIT | peak RSS, MB, `--aot` | peak RSS, MB, JIT | | --- | ---: | ---: | ---: | ---: | | `Bun.serve` | 8.3 | **7.7** | **32.1** | 45.7 | | Hono | **9.3** | 11.5 | **34.4** | 51.8 | | Elysia | **11.5** | 27.5 | **40.8** | 56.6 | | `node:http` | **11.5** | 21.4 | **43.8** | 61.8 | | Fastify | **17.0** | 63.4 | **51.1** | 73.1 | | Express | **18.3** | 50.4 | **54.4** | 85.9 | | Koa | **20.1** | 66.7 | **57.2** | 85.3 | All three tables: Linux x64, AMD EPYC 9R14, each server pinned to one core and its load generator to another, both columns of a row at the same time. LTO build, 64 connections, six routes in turn, every response compared byte for byte, median of 7 starts, 2026-10-24. Memory is read at request 400,000. `test/aot/servers/measure/requests_a_second.py`. There's no source to parse, no bytecode, no JIT-compiled code and no profiling data to keep around. The machine code is file-backed pages of the executable, so the kernel shares it between processes. Private memory is 13 to 25 MB compiled and 15 to 47 MB on the JIT. The executables are 69 to 82 MB, against 141 to 144 MB. At a fixed 5,000 requests per second, the 99th percentile of latency is 54 to 91 µs compiled and 77 to 477 µs on the JIT. The medians are 21 to 40 µs and 21 to 34 µs. ## Where it loses A warm JIT still wins on untyped npm code. That's the second servers table. Koa, Fastify and Express are untyped JavaScript in many small packages. A JIT learns which objects reach a property access by watching the code run. The compiler has to prove it at build time, and where it can't, the access is a lookup by name. [Full types](#full-types) are the way out we're betting on. Code that a package generates at runtime is interpreted, for as long as the program runs. zod 4, ajv, fast-json-stringify and Elysia generate their hot code with `new Function`. That's most of Elysia's loss above. For validation there's a way that needs no generated code: [`JSON.parse<T>` and `validate<T>`](#what-it-does-with-types). Old, untyped JavaScript is slower than on the JIT: constructor functions, class helpers, asm.js output, arithmetic on whatever comes out of a plain array. A JIT watches `a + b | 0`, sees two int32s every time and compiles an add. The compiler has to prove that they're int32s, and where it can't, the `+` is a call to a generic add. JetStream2, the benchmark that JavaScriptCore's JIT is tuned on, is mostly code like this, and most of its tests are slower compiled. Everyday idioms are mixed. `test/aot/perf` has 1,814 of them. Each is compiled twice, as written and with its types erased, and the slower of the two is compared with the same `bun`'s warm JIT, by instructions. | | ahead | within 5% | behind | median ratio | | --- | ---: | ---: | ---: | ---: | | each idiom as a small program | 916 | 223 | 675 | 0.94 | | compiled as in a very large program | 467 | 118 | 1,229 | 1.44 | `bun test/aot/perf/idioms-against-the-jit.ts`, and again with `--in-a-very-large-program`. Linux x64, AMD EPYC 9R14, 2026-10-24. User-mode instructions per iteration, with startup and warm-up subtracted. In the first row over a third are behind, 180 of them by more than 2x. And that row flatters the compiler. Each idiom is a program of its own, so its loop counts as hot and the idiom is inlined into it. A very large program can't have everything hot, and with no profile the compiler has to guess from loops and the call graph. The second row compiles the same idioms the way they'd be compiled there, and calls each through a function that's found at runtime, on the JIT too. Two thirds are behind, 687 by more than 2x. Builds are slow. The build type checks the whole program and compiles every function that can be reached. There's no cache between builds, so a one-line change builds everything again. A large application takes tens of seconds and gigabytes of memory. Executables are big. A small server is 69 MB. That's half of what `--compile` makes, and it's still 69 MB. [`--trim`](#smaller-executables) helps. `bun` itself [grows by about 30 MB](#what-plain-bun-pays). Go and Rust still win on small programs. A hello world in either starts sooner, holds a few megabytes and is a file of a few megabytes. Ours carries Bun's runtime. ## Why When a JavaScript tool gets too slow, the usual fix is to port it to Go or Rust. esbuild, SWC, Biome, oxc, Rolldown, Turbopack and now TypeScript itself went that way. It works, but it takes years, and afterwards most of the tool's users can't read or patch it. JavaScript engines were designed for browsers. Code arrives as text from somewhere you don't trust, has to start right away, and can do anything. So the engine interprets it, watches which types show up, guesses, compiles the guess, and throws the compiled code away when the guess was wrong. All of that happens on the machine that runs the program, every time it starts. A server or a CLI isn't in that position. You have all the source before you ship it. If it's TypeScript, you've also written down what everything is. Then the build erases that, and every process spends its first seconds working it out again. `--aot` keeps the types and does the work once, at build time. ## What it does with types TypeScript is unsound on purpose. `any`, `as`, `!`, `JSON.parse` and `.d.ts` files all let you call something a `number` when it isn't. So the compiler reads a type in two ways. For what your program makes, the type is a fact. `new Vec(1, 2)`, or an object literal of a declared type, gets the layout of that type. For what your program is given, the type is a prediction. An argument from a caller that the build can't see, or the result of `JSON.parse`, is tested once where it comes in. A value that fails the test takes a slow path. ```ts function double(n: number): number { return n * 2; } const body: any = JSON.parse('{"visits":"3"}'); console.log(double(body.visits)); ``` ``` $ bun app.ts 6 $ bun build --compile --aot app.ts && ./app 6 ``` `double` takes a raw number, so it can't take the string `"3"`. A guard at its entry sees the string and sends the call to a second, slow version of `double`, which multiplies the way plain JavaScript does. The string stays a string. The next call with a number takes the fast version again. The rules we hold the compiler to: 1. A program that runs under `bun run` does the same thing compiled. 2. Nothing throws at runtime because of a type. `as`, `any` and `!` never convert a value and never throw. 3. New errors show up at build time, with a message. The exceptions are listed under [What changes](#what-changes). JavaScript files are compiled the same way. A class has a fixed layout there too, and a flow analysis over the whole program follows objects from where they're created to where they're used, with or without annotations. Every property access ends up in one of three states: | A property access is | When | It costs | | --- | --- | --- | | free | The compiler proved which objects get here: from the types, from following every object through the program, or from a test earlier in the function | One load or store | | tested | A value that nobody checked can get in | A shape test, then the load. If the test fails, the slow path | | by name | Nothing is known | A lookup at runtime, through a cache | The build prints the split, and `--explain` lists what's behind the slow ones, worst first, with a fix for each. ``` 92.4% of property accesses are one load: 1,204 free, 61 tested, 38 by name (1,303) ``` An object with a fixed layout is still an object. `Object.keys`, `JSON.stringify`, spread and `for...in` see it like any other. You can add properties that the class didn't declare, delete the ones it did, and store a string in a `number` field through an `any`. That costs speed and nothing else. `JSON.parse<T>(text)` (`JSON` from `"bun"`) and `validate<T>(value)` (from `"bun:bundle"`) check data against a type you've already declared, and make objects with the layout of `T`. These two do throw. That's what they're for. `import type { int32 } from "bun"` is a 4-byte integer field. To TypeScript and under `bun run` it's a `number`. The build has to prove every store into one, and stops with an error where it can't. ## What changes Each of these is there because the compiler can't generate good code without it. `docs/bundler/aot-compatibility.mdx` has the smaller differences and a list of packages that are known not to work. | Change | What it buys | | --- | --- | | Type errors stop the build. They're TypeScript 7's errors, from the checker behind `bun check` | The types that the compiler starts from agree with each other | | Existing members of built-in objects can't be replaced. `Array.prototype.push = fn` throws a `TypeError`, in sloppy mode too. Adding is fine: `Array.prototype.last = fn`. `Object.prototype`, `RegExp.prototype`, `Promise.prototype` and the generator prototypes can't be changed at all. The prototypes of `Request`, `Response` and five other Bun classes are read-only | `arr.push(x)`, `map.get(k)` and `Math.floor(x)` compile to the operation, without checking that nobody replaced it | | A program with no call to `eval` or `new Function` is closed. The compiler finds those calls by following the functions as values, so `const run = eval` counts. A call that it couldn't follow, like `globalThis[name](code)`, throws an `EvalError` | Every caller of a function is known. Calls are direct, guards go away, and what nothing reaches isn't compiled | | The executable holds what your program can reach and nothing else. `import()` and `require()` of a computed path find the files that match its constant parts, and the entry points. Other code throws, like a file that doesn't exist. Data files still load | One file, with no `node_modules` beside it. Less code to compile, map and page in | | A module in the executable runs at most once and isn't in `require.cache` | An import is a direct reference to a variable or a function | | Each thread can have 32 GB of objects. `--aot-huge-heap` lifts that | References between objects are 32 bits | You shouldn't have to find any of this in production: | | your program | | --- | --- | | `bun check --aot` | isn't run. Reports everything the build would report. With `--explain=guards` it lists every guard that can fail | | `bun run --aot`, `bun test --aot` | is checked the way the build checks it, then runs from source with built-in objects read-only | | `--aot-log-type-mismatches` | is compiled, and the executable logs every guard that does fail | ## What it doesn't do There's no JIT in the executable. All machine code is in the file, mapped read-only and executable like the `.text` of a C program. On Linux 6.7 and later the executable calls `prctl(PR_SET_MDWE)` once its code is mapped, and from then on the kernel refuses the process any new executable memory. `dlopen()`, `bun:ffi` calls and up to 256 open `JSCallback`s keep working. `cc()` from `bun:ffi` and native addons that generate machine code don't work there. Text that shows up at runtime is interpreted. `eval`, `new Function` and `node:vm` run in JavaScriptCore's interpreter, in the same process, on the same objects as the compiled code. A `RegExp` built from a string at runtime is matched without a JIT. WebAssembly is compiled if it's a `.wasm` file that the program imports, and interpreted otherwise. `--disallow-code-generation-from-strings` turns `eval` off, like Node's flag. The interpreter is never a fallback for code that the build saw. A function that can't be compiled is a build error. Code from disk is opt-in. By default a dynamic `import()` or `require()` only loads code that's in the executable. `--aot-allow-code-from-disk` lets it load a plugin or a config file from disk, which is then interpreted. `--external <package>` does that for one package. The flag has a price, because the compiler can prove less about anything such a file can get hold of: with it, warm Express serves 13% fewer requests per second than in the table above and Koa 16% fewer. The other five don't move. A program with no dynamic import is the same executable either way. There's no profile and no heap snapshot. The compiler never sees your program run. What's hot is estimated from loops and the call graph. None of your program runs at build time, except macros. There's no cross-compilation, and breakpoints don't work. The build makes code for the platform it runs on. In an executable built with `--compile-inspector`, the console, heap snapshots and CPU profiles work. Breakpoints and stepping don't. | Platform | State | | --- | --- | | Linux x64, glibc | Tested | | Linux arm64, Linux musl, macOS arm64, macOS x64 | Tested. Some tests still fail | | Windows x64, Windows arm64 | Tested. Many tests still fail | | FreeBSD, Android | Bun builds there. Not tested yet | ## Full types Publishing a TypeScript library to npm strips the types out of its code. The `.d.ts` keeps only the ones on its exports. So when you compile an app, your code has types and `node_modules` doesn't, and that's where a lot of programs spend their time. Full types is one more file in the package: `index.js` with the types left in. | | code | types | built | | --- | --- | --- | --- | | `src/index.ts` | all of it | all of them | no | | `index.js` | all of it | none | yes | | `index.d.ts` | none | exports only | yes | | **`index.full.ts`** | **all of it** | **all of them** | **yes** | It sits behind a new condition in `exports`. Tools skip conditions they don't ask for, so Node, `tsc`, editors, Vite, webpack, esbuild, `bun run` and `bun build` resolve what they resolve today. ```json "exports": { ".": { "types": "./index.d.ts", "full-types": "./index.full.ts", "default": "./index.js" } } ``` There's one rule: strip the types and you get `index.js`, exactly. The build checks that every time, against the `index.js` that's installed. So the file can't hide code and can't go stale, and if you patch `index.js`, your patch wins. A full-types file that doesn't match or has a type error gets a warning, and the package is compiled without types. A dependency's types never fail your build. `bun compile --library` builds the whole package from your TypeScript: `index.js`, the `.d.ts` files, `index.full.ts` and a `package.json` with the condition in it. `packages/full-types` has full types for packages that don't ship their own, such as react-dom, Express, Fastify and Koa: annotations added, nothing else changed. `docs/bundler/aot-libraries.mdx` has the rest. ## Smaller executables A `--compile` executable is a copy of `bun` with your program attached, so hello world ships a package manager, a test runner, a type checker and several JIT compilers. Nothing in a compiled program can call those, so `--aot` always leaves them out. `--trim` also leaves out the APIs that your program never uses: `bun:sqlite`, `Bun.Image`, `HTMLRewriter`, `Intl` data by locale, WebAssembly, the bundler, the debugger. It removes code, not names. Every global, class and property of a removed feature is still there, so `typeof`, `in` and `instanceof` answer as before. A call into one throws `ERR_FEATURE_REMOVED` and says how to keep it. `--explain=trim` lists each feature, kept or removed, and why. You can't delete code from a linked binary, because dead functions are scattered all through `.text`. So the work happens when `bun` itself is linked. It's linked once, and the reference graph is read out of the object files' relocations, C++, C and Rust alike. `src/trim/features/*.def` name the features and the functions you have to pass through to reach each one. Then it's linked again with the removable sections grouped into page-aligned ranges of their own, and a table of them goes into `bun`. After that, `bun build --compile` just doesn't map those ranges. Nothing that stays moves, so you don't need a linker or a toolchain. ## How it works Under `--aot` nothing is bundled. Each module stays a module, a CommonJS body is a function, and Bun's own `node:` and `bun:` modules are compiled from their TypeScript sources along with your program. 1. `src/aot/load` walks the imports from the entry points with Bun's resolver, loaders, macros and `Bun.build` plugins, and parses each file once. 2. `src/aot/modules` works out the module graph: ES links, which names a CommonJS file exports, the order of evaluation. `src/sema/fold` folds `process.platform` and the like, and the code behind them is dropped. 3. `src/sema`, the type checker behind `bun check`, checks each file. `src/aot/lower` turns its syntax tree and types into an IR in SSA form, file by file, on all threads. 4. `src/aot/link` merges the files into one program. `src/aot/emit` writes it in a format that both repos generate from one schema (`aot/ir/AOTSchema.def` in the engine). 5. JavaScriptCore runs a flow analysis over the whole program, optimizes the IR, generates machine code through B3 and writes an image: code plus read-only data. 6. `src/bundler/aot`, `src/exe_format` and `src/trim` put the image into a copy of `bun`, with symbols for the compiled functions and without what's trimmed. One table says what every built-in takes, returns, reads, writes and may call back. `src/codegen/builtins` makes it from the engine's descriptions and from Bun's own class and module definitions. Bun compiles it, so a change to one of Bun's native classes needs no new build of WebKit. To start reading: `src/aot/compile.rs` and `src/aot/compile/*_stage.rs` are the driver, stage by stage. `test/aot/ir/README.md` and the files under `test/aot/ir/lower/` show a small program and the IR it has to lower to. `src/aot/lower/convert` and `src/aot/lower/layouts` are where types turn into IR. ## What plain Bun pays There's one `bun`, and the compiler is in it. Everyone who installs Bun gets it, whether they compile or not. That's about 30 MB more on disk, and about 23 MB on musl. An executable from plain `bun build --compile` is a copy of `bun`, so each of those grows by as much. This PR also changes code that runs without `--aot`: `node:http`, streams, events and `process.nextTick` in `src/js`, the checker in `src/sema`, the executable format, and shared parts of JavaScriptCore. Review matters most there. ## Testing Most of the added lines are tests. `test/aot/README.md` lists every group of tests, the command that runs it and where a new test goes. | | | | --- | --- | | `test/aot/programs` | Programs, and `test/aot/generate` writes more when the tests run. Each runs three ways that have to agree: regular Bun, compiled, and compiled with its types erased | | `test/aot/programs/nothing-throws-because-of-a-type`, `test/aot/values-against-their-types` | Values that don't fit their types, everywhere a type can appear | | The `variant()` rows of `test/aot/parts.mts` | The programs again with every pass verified, every function hot, every function cold, no inlining, no flow results, 64-bit references, another code layout | | `--aot-validate` | Compiled code checks every fact it relies on at runtime and stops on a miss. The programs run under it too | | `test/aot/collector-stress` | The programs again with a garbage collection at every third slow-path allocation | | `test/aot/other-suites` | test262, JavaScriptCore's tests, Node's tests and Bun's own `test/js`, each file as an executable against regular Bun on the same file | | `test/aot/frameworks` | Builds and serves Next, Nuxt, Astro, SvelteKit, Angular and React Router apps | | `test/aot/servers`, `test/aot/perf` | The servers and idioms above | | `test/aot/analysis-scaling`, `build-scaling`, `startup-scaling` | Build time and startup can't grow faster than the program | | `test/aot/trimmed-against-whole` | Each program built with and without `--trim`. Both have to print and exit the same | | `test/aot/fuzz` | Generators and mutators, against the shipping build, a build with assertions and the validating build | | `test/aot/engine-tests.test.ts` | The engine's own native and IR text tests, linked with Bun's table of built-ins | | `test/cli/aot-docs.test.ts` | Runs every command in the docs that's followed by its output | Of test262's 47,536 files, 33,467 can run as module code, and 31,231 of those pass. Most of the rest try to change a built-in. All of it takes 130 hours on one core. So each run takes a share: one program in ten, picked by the commit's hash, so every commit runs a different tenth. CI's blocking steps keep the pick fixed, so the same tree always runs the same tests. `AOT_SHARE=all` runs everything. Fuzzers still find programs that it compiles wrong. Known failures are `todo`s in `test/aot`, with the reason next to each. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Pins `WEBKIT_VERSION` to oven-sh/WebKit `main` at fcc1ea0973a4 (oven-sh/WebKit#914). oven-sh#45631 was merged on the preview build of that pull request. This is the same source, built from `main`.
…ory, and bound sparse holes in `files()` (oven-sh#47106) ### What does this PR do? I went through the CVEs and security fixes in libarchive's history (and HdrHistogram's) and turned the ones that could apply to `bun_tar`, `bun_archive` and `bun_histogram` into test cases. Two problems came out of it. Both are also in the last libarchive-based canary, so neither is a regression from oven-sh#46638, which itself has only shipped in canary builds. The third change gives `Bun.serve` directory routes the rules they have on Linux with `openat2` everywhere else but on Windows. The fourth stops `fs.realpath` from opening its path on macOS. #### 1. `extract()` followed symlinks that were already in the destination The extractor checks the text of each entry's path (no `..`, not absolute) and the text of each symlink's target. Where a path leads also depends on the symlinks already on disk, and those were followed. This is the class of CVE-2016-4679 in libarchive, and of CVE-2020-11736 and CVE-2020-36314 in file-roller, which uses it. - With `dest/out -> ../outside` in place, an entry `out/new.txt` was written to `outside/new.txt`. The same held for directories, for parents created on the way, for symlink entries, and for a file entry over a symlink (`victim.txt -> ../outside/victim.txt` was truncated and rewritten). - No symlink has to be there beforehand. Three archives extracted to one directory are enough: `a/up -> ..`, then `a/up/esc -> ..`, then `esc/new.txt`. Each target is inside going by the path of its symlink, but `a/up/esc` is really `dest/esc`, so its `..` is the parent of the destination. - On macOS, `mkdirat` and `symlinkat` follow a dangling symlink when the name ends in `/`, which the name of every directory entry does. `dangling/` created a directory outside. The fix is the new `bun_sys::beneath` (`src/sys/beneath.rs`), which the POSIX arms of `Archiver::extract_to_dir`, `create_deferred_symlinks` and `extract_to_disk_filtered` now call in place of `openat`, `mkdirat`, `symlinkat`, `unlinkat` and `make_path`: | | resolved by | | --- | --- | | Linux 5.6+ | `openat2(RESOLVE_BENEATH \| RESOLVE_NO_MAGICLINKS)` | | macOS 26+, FreeBSD | `O_RESOLVE_BENEATH` | | macOS before 26 | `O_NOFOLLOW_ANY`, on a path that `readlinkat` has resolved if it has symlinks | | Linux before 5.6, Android, `openat2` blocked by seccomp | one `openat(O_NOFOLLOW)` per component, on the same | All of them follow a relative symlink that stays inside and refuse every other one. - `mkdirat`, `symlinkat` and `unlinkat` take no flag that constrains resolution. They are given a single name, without a trailing slash, in a parent that was opened under the constraint. - The parent of a new symlink is reached without following any symlink (`RESOLVE_NO_SYMLINKS`, `O_NOFOLLOW_ANY`), because `is_symlink_target_safe` counts the `..` of a target from where the symlink's path says it is. If a symlink is in the way, the parent is resolved as below, the target is checked again from the resolved path, and the symlink is made there, again without following any (`beneath::symlinkat_where_safe`). So under `current -> releases/v1`, the entry `current/bin/x -> ../lib/x` lands in `releases/v1/bin` as it does on `main`, and `a/up/esc -> ..` is refused. This replaces the `st_dev`/`st_ino` comparison in `create_deferred_symlinks`, which only noticed a parent that changed between its two passes. - `beneath::symlinkat` answers `ENOENT` for a name that ends in `/`. No kernel makes `a/l` of `a/l/`, and `is_symlink_target_safe` counts the `..` of such a target from `a/l`, not from `a`. What such an entry leaves behind is unchanged: a directory of that name, or nothing with a glob. - Without the kernel, a path is first opened with no symlink allowed: one `openat` with `O_NOFOLLOW` per component, or `O_NOFOLLOW_ANY` on macOS. That is all a path without symlinks costs. If it meets a symlink that may be followed, the path is resolved and opened again in the same way. Resolving asks `readlinkat` about each component and puts the target of a symlink in its place. A `..`, which only a target can have, drops the name before it and is `EXDEV` when there is none. Nothing depends on the disk staying as it was read: whatever the resolved path names by then, the second open follows no symlink either. It gives up with `ELOOP` after as many symlinks as the kernel follows (40 on Linux, 32 elsewhere). - The walk keeps what a last `/` means to the kernel: with `O_CREAT` it is `EISDIR`, otherwise the name has to be a directory. An empty path is `ENOENT`. - Parents are opened with `O_PATH` / `O_SEARCH`. A symlink entry in a directory that its archive made `0300` used to be dropped, because the parent was opened for reading. - `bun install` pays nothing. It extracts to a directory it has just created, and symlink entries are created after everything else, so there is no symlink to meet. `ExtractOptions::new_destination` says so and keeps the plain syscalls (`Symlinks::Follow`). `TarballStream` is not touched. - With a glob, a file is now opened first and its parents are created only if that fails, as without a glob. That is one `mkdirat` fewer per file. `open` ignores the flags it does not know, so an `O_RESOLVE_BENEATH` that a macOS kernel lacks would silently do nothing. Support is therefore taken from a refusal: the first call asks for `..` of the directory it was given, and only `ENOTCAPABLE` counts. macOS 26 answers that before any permission check (tried with mode `0000` on the directory and on its parent, and on a deleted directory). An error that says nothing about the flag (`EMFILE`, `ENFILE`, `ENOMEM`, `EBADF`, `ENOTDIR`, `EIO`, `ESTALE`, `ETIMEDOUT`) fails that one call and is not remembered. Linux needs no such probe, because a missing `openat2` fails. `openat2_in_root`, which only directory routes called, is now `beneath::openat` with `Symlinks::FollowAsRoot`. Its probe and its "unavailable" cache are in `openat2_if_available`, which all modes share. That also retries `EAGAIN`, which `openat2` answers when anything on the system is renamed while it resolves a `..`, and after four attempts the path is resolved without the kernel. `openat2_in_root` did neither, so a route could answer `404` for `-> sub/../file` now and then. `BUN_FEATURE_FLAG_DISABLE_RESOLVE_BENEATH=1` makes every `openat2` caller behave as on a kernel without it. It is there for seccomp policies that kill the process on `openat2`, which no check from inside the process can find out, and the tests use it to cover the fallbacks. `docs/runtime/archive.mdx` names it. #### 2. `files()` allocated whatever a sparse map declared `MemoryDecoder::read_data` fills the holes of a sparse member with zeros, up to a declared size of 2^60. Holes take no space in the archive, so the usual bound (output is at most the input) does not hold. A 3,584-byte tarball that declares a 3 GiB file made `files()` on that canary return a 3 GiB `File` and grow RSS by 3,076 MiB. `read_data` now counts the holes it has filled per archive and fails with `HolesTooLarge` above 1,024 times the size of the archive as it was given, but not below 64 MiB or above 1 GiB (`holes_limit` in `src/tar/decoder.rs`). 1,024 times is about what gzip makes of the same input, so a hole is worth no more to an attacker than compressed zeros. That covers all of its callers: `files()`, the plucker, `bun pm diff`, `bun publish` and `bun pm pack`. `extract()` is not limited, since it leaves holes unallocated on disk. #### 3. Directory routes followed symlinks out of `dir` without `openat2` `DirectoryRoute::open_beneath` uses `openat2(RESOLVE_IN_ROOT)` on Linux and used a plain `openat` everywhere else, as documented. On macOS, `public/escape -> ../secret.txt` is served with `200`. Directory routes have been in releases since 1.4.0. So did Linux before 5.6 or under a seccomp policy that blocks `openat2`, where `openat2_in_root` fell back to a plain `openat`. What a tree serves should not depend on what the kernel can enforce, so the rules of `RESOLVE_IN_ROOT` now hold on every system but Windows: `dir` is the root directory, an absolute target starts from it, and a `..` at `dir` stays there. Where the kernel does not do that, the resolver above does. For it, these rules are two match arms (`Symlinks::FollowAsRoot`). macOS 26 and FreeBSD still ask the kernel first: what `O_RESOLVE_BENEATH` opens is what `RESOLVE_IN_ROOT` would open, so only a path that it refuses for leaving `dir` is resolved by Bun. Nothing outside of `dir` is opened. On `main`, a request for a symlink to a FIFO outside of `dir` is a `404` that has opened the FIFO. `RESOLVE_BENEATH` alone would stop the same escapes on every system. But it refuses targets that `RESOLVE_IN_ROOT` clamps and serves, such as `-> /inside.txt` and `-> ../../inside.txt` (both checked with the raw syscall on Linux 7.0), and Linux has served those since 1.4.0. #### 4. `fs.realpath` on macOS opened what its path leads to This one is not from the CVE list. It is the same kind of problem as the FIFO in 3: opening something has effects, and needs permissions, that asking where it is should not. `realpath_inner` opens the path, asks the descriptor for its path, and closes it. On Linux the open is `O_PATH`, which gives a descriptor for the place without opening what is there. macOS has no `O_PATH`, so there it was a real `open(O_RDONLY | O_NONBLOCK | O_NOCTTY)`, then `fcntl(F_GETPATH)`, then `close`. So on macOS, for `fs.realpath`, `fs.realpath.native`, their sync forms and `fs.promises.realpath`: | | `main` | this branch | | --- | --- | --- | | a file of mode `000`, or a symlink to it | `EACCES` | the path | | a directory of mode `100` | `EACCES` | the path | | a Unix socket, or a symlink to it | `EOPNOTSUPP` | the path | | any path, with no descriptor left | `EMFILE` | the path, or `ENOENT`, `ENOTDIR` or `ELOOP` | | a FIFO that a writer is blocked opening | the path, and the writer is released | the path, and the writer stays blocked | Node 25.6 answers the first three with the path too. It now asks `getattrlist()` for `ATTR_CMN_FULLPATH` (`bun_sys::get_full_path`): one system call, by path, with no descriptor. `getattrlistbulk()` lists a directory and needs a descriptor for it, so it does not fit here. `getattrlist()` does not always answer what `open` and `F_GETPATH` answer. These are the cases I found, and they still open the path as before: | | `getattrlist()` alone | | --- | --- | | `/.nofollow/…` and `/.resolve/1/…` | repeats the prefix, so `/.nofollow/private/tmp` would be its own real path | | `/dev/fd/3`, `/dev/stdin` | `/dev/fd/3`, and not the file that is open there | | `file/..namedfork/rsrc` | `EPERM` | | an ACL that denies `readattr` and nothing else | `EACCES` | | a path of 1,020 bytes behind `/.resolve/0/` | `ENOSPC`, because the prefix takes room in the kernel's buffer | There may be more, such as a volume that answers `ENOTSUP` or `EINVAL`, which `getattrlist(2)` documents and I have no such volume to try. So the rule is the other way around: `ENOENT`, `ENOTDIR` and `ELOOP` are returned at once, because `open` only repeats them, and every other error falls back to `open`. With that, the only answers that change are errors that become paths, and `EMFILE` for a path that does not exist, which is `ENOENT` now. Node 25.6 answers the same for an absolute path. For a relative one its `process.cwd()` needs a descriptor. It is also faster. In an optimized build, 200,000 calls each on three paths, three rounds: `open`, `F_GETPATH` and `close` took 19 to 52 µs per call, and `getattrlist()` 0.7 to 1.4 µs, which is 16 to 36 times less. `open` is where antivirus and endpoint-security software hooks in, so the ratio depends on the machine. #### Behavior changes - `extract()` rejects (or, with a glob, skips the entry) where it used to write through a symlink that leaves the destination. Directory and symlink entries that need one are skipped. The error says `A symlink in the destination is absolute or leads outside of it`. Its `code` is `"EXDEV"`, and its `path` is where the entry would have been written, relative to the destination, such as `out/new.txt`. The message is a fixed string, so no byte of the archive is in it. - `extract()` never follows a symlink with an absolute target, even if it points inside. Both kernels refuse those. - Directory routes on macOS, on FreeBSD and on Linux without `openat2` resolve symlinks as Linux with `openat2` does. They used to follow them anywhere. A symlink that leads outside of `dir` is no longer served. Neither is one that leads back into `dir` by its real path, such as `-> "$PWD/public/logo.png"` or `-> ../public/logo.png`: those are looked up under `dir` and are a `404`, as they have been on Linux. `-> logo.png`, `-> sub/../logo.png` and `-> /logo.png` are served. So is `-> /etc/hosts` if `dir` has an `etc/hosts`, and `-> ..` is `dir` itself. - On macOS, `fs.realpath` and its variants answer with the path where they used to fail with `EACCES` (what the path leads to cannot be read), `EOPNOTSUPP` (it is a socket) or `EMFILE`. #### Not changed - **Windows** is the one system where the rules still differ. A directory route follows symlinks and junctions wherever they lead. Extraction never creates symlinks there, so an archive cannot plant one, but nothing checks for junctions and symlinks that are already in the destination. None of this was run on Windows, so its arms are as they were. `OBJ_DONT_REPARSE` may be the equivalent to look at. - Every other extraction failure still reaches JavaScript as `ReadError`. That includes a symlink that cannot be resolved at all: a loop, too long a chain, or a target with a missing directory in it. The recorded archives of `archive-golden` expect that message. - `extract()` without a glob still counts the entries it skips. - `fs.realpath` on FreeBSD still opens its path with `O_RDONLY`. `O_PATH`, or libc's `realpath`, may do there. I could only compile that arm, so it is as it was. - `fs.realpath("/dev/stdin")` is still `EBADF` on macOS when stdin is a pipe. Node answers `/dev/fd/0`. - A path that only a compiled program has (`bun build --compile --aot`) still resolves to itself. That rule followed the `open` that macOS no longer does, so there it now follows `getattrlist()`. - Three other places on macOS open a path only to learn where it is: the module resolver for each symlinked entry (`lstat`, `open`, `fstat`, `F_GETPATH`, `close`), `Bun.main`, and `fs.watch` when it starts. `get_full_path` would fit all three. The resolver is a hot path and deserves its own change and benchmark. #### Where resolving without the kernel differs from the kernel None of these leaves the directory. - `fs.protected_symlinks` and the `nosymfollow` mount option apply when the kernel follows a symlink, not when Bun reads one and goes on. - The resolved path is given to the kernel from the destination, so it cannot be longer than `PATH_MAX`. The kernel itself gets deeper than that through symlinks. - On macOS, a symlink with mode `000` and a target that names a resource fork (`f/..namedfork/rsrc`) are followed by the kernel and refused without it. - Some refusals have another error. A symlink that is not followed in the middle of a path is `ELOOP` for `openat2` and `ENOTDIR` without it. A chain of 39 symlinks that ends outside is `ELOOP` and `EXDEV`. To create `./` exclusively is `EEXIST` and `EISDIR`. To create `link/` is `EISDIR` for `openat2`, and without it whatever following the symlink comes to. #### What else was tested and found fine The corpus has 1,282 cases (293 for extraction, 989 for the readers). Each runs through two to four of `extract()`, `extract({ glob })`, `files()` and an entry listing. That is 4,288 jobs in separate processes on macOS, with the destination's surroundings, exit status, time and RSS checked. | | escapes | crashes | hangs | | --- | --- | --- | --- | | `main` (debug + ASAN) | 30 | 2 | 0 | | 1.4.4-canary.1+963452075 (libarchive) | 30 | 0 | 0 | | this branch | 0 | 0 | 0 | | this branch, `BUN_FEATURE_FLAG_DISABLE_RESOLVE_BENEATH=1` | 0 | 0 | 0 | The two crashes are ASAN aborting on a 1 TiB allocation, which is problem 2. The six cases with a 1 TiB sparse file were left out of the run on the canary. - **Readers:** every typeflag, size, time and mode fields at and past their limits in octal and base-256, malformed pax records and out-of-range pax numbers, `L`, `K`, `x`, `g` and `V` headers that are oversized, negative, repeated or cut short, old GNU, pax 0.0, 0.1 and 1.0 and `SUN.holesdata` sparse maps, ACL text, `._` entries, names that are not UTF-8, members whose data is missing, and layered, cut short and damaged gzip. No crash, hang or unbounded allocation other than problem 2. - **Writer** (CVE-2025-5917 and the fixes around it): 3,969 names around the ustar limits of 100, 155 and 255 bytes, with the slash at each position that matters. Bun and `bsdtar` both read all of them back unchanged. - **Histogram:** 2,137 cases from the four HdrHistogram CVEs and the fix history of HdrHistogram_c (bucket configurations, counts that overflow through `add`, top-bucket values, `percentile` inputs, wrong receivers, `structuredClone`). No crash, negative value or percentile out of order. The two decoding CVEs do not apply, because Bun has no decoder. - The records about formats Bun does not read (rar, 7zip, zip, iso9660, cab, lha, mtree, xar, warc, cpio) do not apply. ### How did you verify your code works? New tests in `test/js/bun/archive.test.ts`. The symlink ones run without a glob and with one, by default and with the kernel feature turned off: - `path safety > symlinks in the destination > nothing is written outside of it`: 18 cases - `… > one that stays inside`: 8 cases, among them 30 and 41 symlinks in a row, one to itself and one with `..` in its target - `… > a symlink entry below one that stays inside`: 4 cases, three that are made where they really are and one whose target leads outside from there - `… > directories that no entry names are 0755` (the class of CVE-2023-30571) - `… > in a directory that cannot be listed` - `… > a symlink whose name ends in a slash is not made` - `… > one that goes back and forth 400 times` - `sparse members > more holes than files() reads` In `test/js/bun/archive-golden/archive-golden.test.ts`: - `extract() when the kernel is not asked to resolve paths` runs the recorded archives again with the flag set, so the fallbacks are held to the same results as the kernels. It is a control too: the flag means nothing to the system Bun. In `test/js/bun/http/serve-directory-routes.test.ts`: - `symlinks … > only lead to what is inside the root` takes the place of `rejects symlinks that leave the root`, which was Linux only. It runs everywhere but on Windows, by default and with the flag set, and expects the same 16 answers on every system: symlinks that stay inside, absolute targets at the top and in a subdirectory, a `..` that is the root, real paths that lead back into the root and are not found under it, six ways to leave (one to a sibling whose name starts with the name of the root), and a loop. With `USE_SYSTEM_BUN=1` on macOS both fail. In `test/js/node/fs/fs.test.ts`: - `realpath does not open what its path leads to`: a file of mode `000`, a directory of mode `100`, a Unix socket, a symlink to each and a file in that directory, for all five variants. It first checks that `open` fails on them. It fails with `USE_SYSTEM_BUN=1` on macOS. - `realpath needs no file descriptor`: with none left, a file resolves, and a missing path, one below a file and a loop are `ENOENT`, `ENOTDIR` and `ELOOP`. All eight answers are `EMFILE` with `USE_SYSTEM_BUN=1`. - `realpath leads from /dev/fd to the open file` and `realpath resolves what macOS only resolves for open()` are controls: they pass on `main`, and hold the cases that still open to what `main` answers. The rows with `/.nofollow/` and `/.resolve/` only run where the kernel has those prefixes, which is from macOS 15.4. CI runs macOS 26 and macOS 14, so both the `O_RESOLVE_BENEATH` path and the path of an older kernel run there. The tests in `archive.test.ts` are 34. With `USE_SYSTEM_BUN=1` (1.4.4-canary.1) on macOS, 16 fail and 18 pass. The 18 are controls for the new code rather than for the old bugs: `0755` (4), `ends in a slash` (4), `goes back and forth` (4), `cannot be listed` with a glob (2), `a symlink entry below one` with a glob (2), and `extract()` still writing more than 1 GiB of holes (2). On this branch all 170 tests of the file pass. By hand, with archives made by the system `tar` and the same script on `main` and on this branch: - Each of the escapes above happens on `main` and not here. A normal archive extracts identically, and `bun install ./pkg.tgz` produces the same tree. - While another process keeps swapping `dest/d` between a directory and a symlink to `../outside`, 80 extractions of 200 files under `d/` left 201 entries outside on `main`, and none here, with or without the flag. - 20 extractions over symlinks that are already there: five kinds that stay inside (to a directory, a chain of three, `..` in the target, a file over a symlink, `node_modules/x -> ../store/x`) are followed, and five that do not (absolute, leaving, leaving and coming back, going down first, to itself) are refused. The output with the flag set is the same. - `files()` on a 3,072-byte tarball of a real sparse file reads one of 32 MiB, and rejects one of 65 MiB and one of 3 GiB with no growth in RSS. The canary returns the 3 GiB and grows by 3,075 MiB. - 34 requests to a directory route on macOS 26, on `main` and on this branch. 19 change. 7 reached something outside of `dir` (`SECRET`, `/etc/hosts`) and are a `404` now. 11 led back into `dir` by its real path and are a `404` now, as on Linux. 1, `sub/alias -> /inside.txt`, was a `404` and is served now, as on Linux. The other 15 are unchanged: plain files, redirects, index files, relative symlinks, a `Range` request through a symlink, a loop and dangling symlinks. With the flag set, all 34 answers are the same. - A writer that is blocked opening a FIFO outside of `dir` is released by a request for a symlink to it on `main`, which answers `404`. Here it stays blocked, with or without the flag. - With the fd table full at the first request to a route, so that the probe gets `EMFILE`, that request is a `404`. Once descriptors are free again, the symlinks that leave are still a `404` and the one that stays inside is a `200`. For `fs.realpath`, on macOS 26.7: - I linked `bun_sys` into a small program that resolves a path both ways and ran it over 607,490 real paths, 66,305 of them symlinks: `/usr`, `/private/etc`, `/opt/homebrew`, the system frameworks, `/Applications`, `/System/Volumes/Data`, `/tmp`, `/var`, `/etc` and this repository. 605,295 get the same path and 2,193 the same error. 2 were `EACCES` and get a path now. None gets a different path, and none that had a path gets an error. - With each of the five clauses of `get_full_path` removed in turn (three prefixes, the fallback on an error, the errors returned at once), the same program reports a difference for the cases that clause is there for, and for no other. Those cases are the rows of the tests above, and of the existing `realpath reports ENAMETOOLONG…`. - 22 paths with `realpathSync` and `realpathSync.native`, on 1.4.4 and on this branch: 10 of the 44 answers change, all from an error to the path that Node 25.6 gives. Letter case, firmlinks (`/System/Volumes/Data/Users` is `/Users`), `ENOENT`, `ELOOP` and `ENOTDIR` stay as they were. - `/dev/stdin` and `/dev/fd/0` answer the same on `main` and here with stdin a file, `/dev/null`, a pipe, and closed. - A writer that is blocked opening a FIFO is released by `realpathSync` on `main`, and stays blocked here. With no descriptor left, `main` answers `EMFILE` and this branch the path. A path of 862 bytes resolves and one of 1,063 bytes is `ENAMETOOLONG`, on both. - `fs.test.ts` (622 pass), Node's five `test-fs-realpath*.js`, `promises.test.js` (30) and `fs-mkdir.test.ts` (21) pass. The Linux and FreeBSD arms are not compiled on macOS, so: - `bun_sys`, `bun_tar` and `bun_archive` build for `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-musl`, `aarch64-linux-android`, `x86_64-unknown-freebsd` and `x86_64-pc-windows-msvc`, and `bun_runtime` for `x86_64-unknown-linux-gnu` and `x86_64-unknown-freebsd`. - I linked `bun_sys::beneath` into a static Linux binary and ran 123 calls of it in a container on Linux 7.0, as a non-root user, three times: with `openat2`, with the flag set, and under a seccomp profile that answers `openat2` with `ENOSYS`. Nothing was created outside in any of them. - A fuzzer in the same three situations takes the kernel for the reference. It builds 6,000 forests of directories and symlinks, some with chains around the limit of 40 and targets close to `PATH_MAX`, and opens 60 paths in each to read, search, create or create exclusively: 192,029 with `FollowInside`, 96,025 with `FollowAsRoot` and 71,946 with `Refuse`. Whenever either side succeeds, so does the other, and they have opened the same file: 31,817 times, 1,806 of them something other than what the path says literally, so through a symlink. 15,902 are refused for leaving. Nothing outside was opened or changed. About 1,300 of the 288,054 that may follow symlinks are refused with another error than the kernel's, as listed above. The number moves by a few from run to run, because the kernel's answer for a long chain that ends outside does. - The number of system calls is linear. For one open through `-> x/../x/../…/real`, `strace` counts `2 × pairs + depth + 3` calls of `readlinkat` and `2 × depth + 4` of `openat`: 327, 527, 927 and 1,727 in all for 100, 200, 400 and 800 pairs 40 directories down, and 1,637, 1,667, 1,727 and 1,847 for 800 pairs at 10, 20, 40 and 80. Each `readlinkat` is given a path from the destination, so the kernel's own work per call grows with the depth, as it does for `realpath(3)`. - The tree and the 16 requests of the new route test, opened as the route opens them on Linux, get the answers the test expects, and the same ones in all three situations. macOS gives those answers too. - I linked `bun_archive` the same way and ran `Archiver::extract_to_disk`, which is `extract()` without a glob, on the archives of `archive-golden` in the same three situations. 1,098 of the 1,253 can be compared, and all 1,098 leave what was recorded. The other 155 reach zlib or another library that the binary does not link. Earlier states of this branch differed in 24 of them by default and in another 18 without `openat2`, all from names that end in `/`. CI caught the first kind, in `archive-golden`. - Linked the same way, `extract_to_disk` ran on seven sets of archives with symlink entries below symlinks: the four cases of the new test, `current -> releases/v1`, an entry through a symlink that leaves, and the three archives of `a/up/esc`. In four, the entry stays inside and is made where it really is. In the other three, the entry that would lead outside is not made. Nothing is written outside, and the output is the same in all three situations. - The FreeBSD arm has only been compiled. The macOS constants and semantics (`O_RESOLVE_BENEATH` is `0x1000` and fails with `ENOTCAPABLE`, 107, neither of which the pinned `libc` has) come from the macOS 26 SDK headers and from C probes on macOS 26.7. The macOS 15.4 SDK has the flag too, documented differently and without `ENOTCAPABLE`. That kernel is untested, so only the macOS 26 behavior counts as support and 15.4 takes the `O_NOFOLLOW_ANY` path. Also run on macOS, before the `fs.realpath` change: the source lints (186 pass), `symlink-path-traversal` (93), `bun-pm-cache-pack` (206), `bun-install-streaming-extract` (134), `bun-install-tarball-integrity` (165), `archive-golden` (150), `serve-directory-routes` (31, its 3 Linux tests skipped), `bun-pack` (362) and `bun-create` (131 of 140). The nine failures are the same nine on `main` on the same machine: the LeakSanitizer test times out, and the others run `git commit`, which could not sign on this machine at the time. `perf_hooks` (24) and `bun-install-git-deps` (196 of 200, the same four as on `main`) were run on the first commit only.
…ven-sh#47235) ### What does this PR do? `CrashReason::Panic` held a `&'static [u8]`, but three of its callers only have a borrowed message: `panic_impl(msg: &[u8])`, the Rust panic hook (a stack buffer) and `Bun__crashHandler(ptr, len)`. Each erased the lifetime with `detach_lifetime` in an `unsafe` block. This gives the enum the lifetime it was missing: `CrashReason<'a>` with `Panic(&'a [u8])`. `crash_handler` takes `CrashReason<'_>`, and `TraceString<'a>` stores `CrashReason<'a>`. Nothing keeps a `CrashReason` past the call that made it, so the borrow checker accepts all three callers as they are. The fourth `detach_lifetime` in this file, in `CrashHandler__beginDlOpenAction`, was redundant. `ffi::cstr` turns the raw pointer into a `&CStr` whose lifetime its caller chooses, so the lifetime that the C++ caller's contract allows is now chosen there, in the `unsafe` block that was needed anyway. **4 fewer `unsafe` blocks**, no new code, no cost. The crate no longer uses `detach_lifetime`. `ZigError` keeps `&'static [u8]`: every caller passes a literal. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux glibc and musl, macOS, Windows, FreeBSD and Android. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/cli/run/run-crash-handler.test.ts` | 30 | 1 | yes | | `test/cli/run/crash-report-command-char.test.ts` | 3 | 0 | yes | | `test/js/node/process/dlopen-duplicate-load.test.ts` | 17 | 0 | yes | | `test/regression/issue/15753.test.ts` | 6 | 0 | yes | Total: 56 pass, 1 fail. The one failure is `crash trace starts at the crash site, not inside the crash handler`. It fails the same way on `main`: it looks for `js_panic` in the symbolized trace, and the frame is printed as `bun_runtime::api::crash_handler_jsc::panic`. Of the four call sites, these tests run `panic_impl` and `CrashHandler__beginDlOpenAction`. The Rust panic hook and `Bun__crashHandler` are covered by the type check only. I also triggered a panic, a root error, out-of-memory and a segfault through `bun:internal-for-testing`, and called `process.dlopen` on a missing file and on a file that is not a library, on this build and on `main`. With addresses and timings masked, the first lines of the output and the exit code are the same on both builds in all six cases. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…#47238) ### What does this PR do? The shell reference-counts `IOReader`, `IOWriter`, `PipeReader` and `BuiltinBlob` with `std::sync::Arc`, although none of them leaves its thread. To get that past clippy's `arc_with_non_send_sync`, which is `deny` here, it carried: - `unsafe impl Send` and `unsafe impl Sync` for `IOReader` and `IOWriter`, each commented "shell is single-threaded; `Arc` is used purely for refcounting"; - `#[allow(clippy::arc_with_non_send_sync)]` on `PipeReader`; - a module of trait tricks asserting at compile time that `Arc<PipeReader>` is *not* `Send` or `Sync`; - a compile-time assertion that `BuiltinBlob` *is* `Send + Sync`. This uses `std::rc::Rc` (and `rc::Weak`), which says the same thing in the type. All of the above is deleted. **4 fewer `unsafe impl`s.** Reference counts are plain increments instead of atomic ones. The rest of the diff is the rename (`Arc` to `Rc` in code and comments, `arc_as_mut_ptr` to `rc_as_mut_ptr`) and the two lists that track such impls: the four entries leave `vm-thread-door.inventory.json`, and the two for these files leave `mordant-baseline.toml`. **Does anything carry one to another thread?** The compiler now rejects that wherever `Send` / `Sync` are derived. It cannot see through raw pointers or hand-written `Send` / `Sync` impls, so I read those: - **Thread-pool tasks.** None of the 12 structs that run on the pool has a field of one of these types, or of a type that contains one: `ShellCpTask`, `ShellLsTask`, `ShellMkdirTask`, `ShellMvCheckTargetTask`, `ShellMvBatchedTask`, `ShellRmTask`, `DirTask`, `ShellTouchTask`, `CondExprStatInner`, `ShellGlobTask`, and `cp`'s `NewAsyncCpTask<true>` and `CpSingleTask<true>` in `node_fs.rs`. They hold ids, byte buffers, fds, atomics and raw pointers. - **The interpreter pointer.** The interpreter does own these `Rc`s, and nine of those structs embed a `ShellTask`, which holds a `*mut Interpreter`. None of the functions that run on the pool (`run_from_thread_pool`, `run_from_thread_pool_impl`, `work_pool_callback`, `cp_on_copy`) dereferences it. - **`JsCell`** is `Send + Sync` whatever it holds, so the compiler would not object to moving a struct that keeps an `Rc` in one. In the shell, the only ones that do are fields of `Interpreter`, covered above. - **Callbacks.** `IOReader`, `IOWriter` and `PipeReader` give a raw pointer to themselves to their `bun_io` reader or writer, and a subprocess gives one to its `Process` for the exit callback. The event loop that owns the shell dispatches those callbacks on its own thread. The GC's size callback returns a cached number and does not walk these objects. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux glibc and musl, macOS, Windows, FreeBSD and Android. - On a debug + ASAN build: All 44 test files under `test/js/bun/shell/` (including `commands/`): **1053 pass, 3 fail**. The three failures are timeouts: `leak.test.ts` `memleak_Blob_something` and `memleak_Blob_nothing` (100 s limit) and `shell-load.test.ts` `immediate exit` (90 s limit). Run alone, they time out the same way on a debug + ASAN build of `main` on this machine, which is too slow for those limits. Because that leaves the `Blob` leak tests unfinished here, I ran a smaller loop on both builds: 480 shell commands with a 128 KB `Blob` on stdin, after a warm-up, with ASAN's quarantine turned down to 1 MB. RSS changed by -0.4 MB on this build and +2.4 MB on `main`. Leaking the blob once per command would be about 60 MB. I also ran one script on both builds that covers pipelines between builtins and processes, redirects to and from files, a `Response` on stdin, a `Buffer` on stdout, command substitution, `ls -R`, `cp`, `mv`, `rm`, a 20,000-line pipe, 20 concurrent commands, `.quiet()` and `.lines()`. The output is the same on both. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…-sh#47231) ### What does this PR do? `uws::Loop` mirrors the C `us_loop_t` field by field, and its methods took `&mut self`. So `tick(&mut self)` claimed exclusive access to the loop for the whole tick, while C and the callbacks it dispatches write the same fields and reach the loop again through `Loop::get()`. The comment on the `extern "C"` block named that problem as the reason every declaration stayed `*mut Loop`, which put an `unsafe` block in every method. This puts every field of `PosixLoop`, `WindowsLoop` and `InternalLoopData` in a `Cell`. `Cell<T>` has the layout of `T`, so the structs still match C, and the layout assertions are unchanged. A `&Loop` now claims nothing about the bytes behind it, so: - every method takes `&self`; - a C function is declared `safe fn(&Loop)` where a live loop on its own thread is all it asks for (six of them), and the `unsafe` block around each call is gone; - code that kept a raw loop pointer for a whole function (`auto_tick`, `auto_tick_active`, the timer code, the HTTP thread) borrows it once as `&Loop`, instead of dereferencing it in an `unsafe` block per call. Four C functions that take only a loop are still `unsafe` to call, because they ask for more: - `uws_loop_date_header_timer_update` and `uws_loop_flush_pending_writes` read uWS's `LoopData` behind the loop. Every loop has one except spawnSync's, which `Loop::create` makes without it. `SpawnSyncEventLoop::uws_loop()` is private, so that loop does not leave its file. - `us_internal_free_closed_sockets` must not run while the loop is dispatching events. - `us_wakeup_loop` is the one that other threads call, so it takes a raw pointer. **29 fewer `unsafe` blocks.** No new `unsafe fn`, `unsafe impl` or helper. `Cell::get` / `set` are plain loads and stores, so the generated code is no worse. What else changes, each needed or made possible by the above: - Fields that nothing outside the crate uses are private. Other threads write `pending_wakeups` and `dns_ready_head`, so Rust must not read them. - `fd`, `head`, `quic_next_tick_us` and (Windows) `uv_loop` are private with read-only getters. C follows these, and `uv()`'s SAFETY comment relies on `uv_loop` never being reassigned, which a `pub` `Cell` would let safe code do. - libuv's `Loop::active_handles` is a `Cell`, and `inc` / `dec` / `add_active` / `sub_active` take `&self`. `WindowsLoop::uv_mut`, which turned the raw pointer into `&mut uv::Loop`, is deleted. - `SpawnSyncEventLoop::uws_loop_mut` is deleted. It now ticks through `uws_loop()`, whose borrow is bound to `&self`. - `HTTPThread::uws_loop_mut`, which returned a `&mut Loop` of any lifetime and held it across `tick()`, is replaced by `uws_loop_ref(&self) -> &Loop`. - `WindowsWaker::wait` calls `run()`. It called the C function directly only to avoid a `&mut WindowsLoop`, so `us_loop_run` is no longer re-exported. - `posix_event_loop.rs`'s private `loop_add_active` / `loop_sub_active` are deleted in favour of the `Loop` methods their comment was waiting for. - `DateHeaderTimer` calls `should_enable_date_header_timer()` instead of reading the field. Not changed: `Loop::get()` still returns `*mut Loop`, and a caller that holds a raw pointer still turns it into a reference in `unsafe`. The other accessors that return `&mut Loop` (`VirtualMachine::uws_loop_mut` and `platform_loop_opt`, `bun_io`'s `platform_event_loop`, `loop_mut` and `uws_loop_mut`) are untouched. ### How did you verify your code works? Release builds behave the same, so there is no new test. In debug builds, activating a `FilePoll` now goes through `Loop::add_active` / `sub_active`, which log under the `Loop` scope. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux glibc and musl, macOS, Windows, FreeBSD and Android. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`. Each file has the same passes and the same failing tests, by name, on both builds. | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/js/bun/spawn/spawnSync.test.ts` | 18 | 0 | yes | | `test/js/bun/spawn/spawn.test.ts` | 182 | 0 | yes | | `test/js/web/timers/setTimeout.test.js` | 35 | 3 | yes | | `test/js/web/timers/setInterval.test.js` | 7 | 1 | yes | | `test/js/web/timers/setImmediate.test.js` | 3 | 0 | yes | | `test/js/bun/http/serve.test.ts` | 334 | 0 | yes | | `test/js/bun/http/serve-http3.test.ts` | 84 | 0 | yes | | `test/js/bun/http/bun-serve-date.test.ts` | 1 | 0 | yes | | `test/js/web/fetch/fetch.test.ts` | 397 | 1 | yes | | `test/js/bun/net/socket.test.ts` | 194 | 1 | yes | | `test/js/bun/net/tcp-server.test.ts` | 11 | 0 | yes | | `test/js/web/websocket/websocket.test.js` | 97 | 2 | yes | | `test/js/node/dns/node-dns.test.js` | 144 | 28 | yes | | `test/js/node/quic/quic-endpoint.test.ts` | 9 | 0 | yes | | `test/js/node/quic/quic-stream.test.ts` | 13 | 0 | yes | | `test/js/bun/udp/udp_socket.test.ts` | 234 | 0 | yes | | `test/js/node/worker_threads/worker_threads.test.ts` | 161 | 1 | yes | | `test/cli/test/isolation.test.ts` | 50 | 0 | yes | Total: 1974 pass, 37 fail. The 37 failures are the same 37 tests on `main` on this machine, so none comes from this change. I also ran one script on both builds that serves HTTP and a WebSocket, fetches, spawns (sync and async), uses an unref'd interval, a timeout, an immediate and a Worker, then stops the server. Both print the same output and exit 0 on their own, which they would not if the loop's ref counts were off. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…sh#47247) ### What does this PR do? Removes seven `unsafe { detach_lifetime(..) }` blocks. In each case the data already lives as long as the code needs. The signature, field or local just didn't say so, and once it does the borrow checker accepts the code as it is. | Where | Before | After | |---|---|---| | `P::value_for_define` | took `&DefineData` and erased `original_name()` to `'a` | takes `&'a DefineData`, which is what all three callers have (`p.define: &'a Define`) | | `ColonListValue::resolve_value` (`--define`, `--loader`) | took `&[u8]` and erased it to `'static` | takes `&'static [u8]`: the caller's input is already `'static` (argv) | | `edit_trusted_dependencies` | borrowed from a `Query` that died at the end of the `if let`, and erased the borrow | keeps the `Query` in a local for as long as the slice is used, and reuses it where the function looked the same key up a second time | | `DirEntryResolveQueueItem::safe_path` | `RawSlice<u8>`, erased back to `'static` when read | `&'static [u8]`, the type of the `DirEntry::dir` it is set from | | `PackageManager::global_link_dir_path` | interned the path, copied the result into a `Box<[u8]>`, then erased a borrow of the box | keeps the interned `&'static [u8]` | | `AsyncModule::resume_loading_module` (2 blocks) | took `parse_result` out of `self`, put it back, took it out again, and erased two borrows of `self` to get past the write | takes it out once | **7 fewer `unsafe` blocks**, +39/−80. No new `unsafe` and no new helper. Nothing does more work at run time, and three places do less: `global_link_dir_path` saves an allocation and a copy, `edit_trusted_dependencies` a property lookup, and `resume_loading_module` a placeholder `ParseResult`. `unsafe_path`, the other field of `DirEntryResolveQueueItem`, stays a `RawSlice`: it points into a thread-local buffer. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux, macOS and Windows. The change has no platform-specific code. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/bundler/esbuild/default.test.ts` | 155 | 0 | yes | | `test/bundler/transpiler/transpiler.test.js` | 258 | 0 | yes | | `test/cli/install/bun-prune.test.ts` | 177 | 0 | yes | | `test/cli/install/bun-pm.test.ts` | 154 | 0 | yes | | `test/js/bun/resolve/resolve.test.ts` | 107 | 1 | yes | | `test/cli/install/bun-link.test.ts` | 57 | 0 | yes | | `test/bundler/cli.test.ts` | 44 | 0 | yes | | `test/cli/install/bun-install-link-path.test.ts` | 49 | 0 | yes | | `test/bundler/bundler_loader.test.ts` | 62 | 0 | yes | | `test/cli/install/isolated-relink.test.ts` | 6 | 0 | yes | | `test/js/bun/resolve/import-meta.test.js` | 34 | 0 | yes | | `test/js/bun/resolve/resolve-ts.test.ts` | 27 | 0 | yes | | `test/js/bun/resolve/resolver-permission-denied-ancestor.test.ts` | 2 | 0 | yes | | **Total** | **1132** | **1** | | The one failure is `loads reflect-metadata before tsyringe`. It needs the `tsyringe` package, which is not installed on this machine, so it fails the same way on `main`. I also ran these on both builds and compared the output, which is the same in all of them: - **Defines and loaders:** `--define` with identifier, dotted, string and number values, at run time, in `bun build` and through `Bun.Transpiler`, including assignment and `delete` targets. `--loader` with two mappings. The error messages for a malformed `--define` and `--loader`. - **Resolver:** `Bun.resolveSync` from six directories of a nested project, a `FileSystemRouter` over part of it, an entry point deep in the tree, and `bun build` from a nested directory. A breakpoint confirms that this reaches the branch that sets a non-empty `safe_path`. - **`bun link`:** register, use, `--save`, reinstall, the isolated linker, a package that was never linked, and `bun unlink`. - **`trustedDependencies`:** `bun add --trust` and `bun pm trust` from 14 starting states: no list, an empty list, a list of other names, a name that is already listed, several names at once, `--all`, an unknown name, and a list that is a string, an object or holds non-strings, which are all rejected. **Not covered by any run:** `resume_loading_module`. Nothing reaches it today. It only runs for a module with `pending_imports`, and nothing fills that list, so auto-install at run time goes through the synchronous path (I checked with `BUN_DEBUG_AsyncModule=1`). That change is checked by the compiler only. `self.parse_result` is left as `ParseResult::empty` on both the success and the early-error path, as before. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ntext (oven-sh#47251) ### What does this PR do? While it prints a file, the bundler's printer asks the linker about other files through `RequireOrImportMetaCallback`, a type-erased context pointer plus a function pointer. Two things about that type were wrong, and `print_code_for_file_in_chunk_js` paid for both: - **It demanded `&mut`.** `init` took `&mut T` and the trait method took `&mut self`, but the linker's implementation only reads four graph columns. - **It had no lifetime.** `init` and `call` are safe functions, yet nothing tied the stored pointer to the borrow it came from. Because of the `&mut self`, every other borrow of `self` that goes into the same `js_printer::Options` (`ts_enums`, `import_member_bindings`, the line offset table, `mangled_props`) had its lifetime erased with `detach_lifetime_ref`, and `parse_graph` was dereferenced raw instead of through its accessor. Now the callback is `RequireOrImportMetaCallback<'a>` and holds a `&'a T`, using the `'a` that `Options<'a>` already has, and the trait method takes `&self`. All the borrows are shared, so the borrow checker accepts them as written and the 28-line workaround goes away. With that in place: - The two functions that build a callback, `print_code_for_file_in_chunk_js` and `generate_entry_point_tail_js`, take `&LinkerContext`, which is all they need. - `Options` holds `Option<RequireOrImportMetaCallback<'a>>`, like the `Option<SourceMapHandler<'a>>` next to it. Before, "no callback" was a null context plus a `noop` function that was never called, tested with `is_none()` and then `unwrap()`. - `LinkerContext::require_or_import_meta_for_source` had one caller, the trait method that forwarded to it. Its body is now that method. **5 fewer `unsafe` blocks**, +59/−113, part of which is that moved body. The one `unsafe` block that remains is the cast back from the erased pointer, and its SAFETY comment no longer has to rely on what callers promise. There is no cost. `NonNull` gives the `Option` a niche, so the field is still two words, and a call is still one null test and one indirect call through the same kind of function pointer. I kept the hand-rolled function pointer. `Option<&'a dyn RequireOrImportMetaSource>` would remove the last block as well, but each call would then load the function pointer from a vtable first. Two other lifetime workarounds nearby are left alone: the `Renamer` transmute in the same function and the `import_records` erasure in `postProcessJSChunk.rs`. Both come from `Renamer<'r, 'src>` being invariant in `'src`, which is a separate problem. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes on Linux. The change has no platform-specific code. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/bundler/esbuild/default.test.ts` | 155 | 0 | yes | | `test/bundler/esbuild/importstar.test.ts` | 99 | 0 | yes | | `test/bundler/esbuild/ts.test.ts` | 59 | 0 | yes | | `test/bundler/bundler_cjs2esm.test.ts` | 113 | 0 | yes | | `test/bundler/esbuild/splitting.test.ts` | 26 | 0 | yes | | `test/bundler/bundler_merged_enum.test.ts` | 2 | 0 | yes | | `test/bundler/bundler_minify.test.ts` | 67 | 0 | yes | | `test/bake/dev/bundle.test.ts` | 26 | 28 | yes | | `test/bake/dev/esm.test.ts` | 11 | 7 | yes | | **Total** | **558** | **35** | | All the failures are in the two `test/bake/dev` files. Those tests drive a browser-like client that needs the `happy-dom` package, which is not installed on this machine, so they fail the same way on `main`. The tests in those files that only need the server pass. I also ran 14 `bun build` commands on both builds over a small project, and then ran five of the bundles. The project has `require()` of ESM and of CommonJS, `import()` of a local file, `import *`, TypeScript enums used across files, a JSON import, a CSS module and top-level await. The commands cover `--format` esm, cjs and iife, `--target` bun, node and browser, `--minify`, `--minify-syntax --minify-whitespace` and `--minify-identifiers`, inline and external source maps, `--splitting` with two entry points, and `--format=internal_bake_dev`. One command, CommonJS output of a module with top-level await, is rejected with the same error. All 4,407 lines of output, which include the bundles and the source maps, are the same on both builds. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…en-sh#47253) ### What does this PR do? The EVP-backed hashers in `bun_sha_hmac` (`SHA1`, `SHA256`, `SHA512`, ...) have a one-shot `hash(bytes, out, engine)`. It was an `unsafe fn` for one reason: `engine` is a `*mut ENGINE`. So all 15 call sites were `unsafe` blocks, 14 of them only to pass a null pointer, several under a paragraph explaining why null is fine. BoringSSL never reads that argument. `EVP_Digest` hands it to `EVP_DigestInit_ex`, whose body does not mention it (`vendor/boringssl/crypto/fipsmodule/digest/digest.cc.inc`). The parameter exists for OpenSSL source compatibility. This drops the parameter. `hash(bytes, out)` passes null itself and is a safe function, with the same signature as the non-EVP hashers in the same file. The one caller that passed a real engine is `Bun.SHA256.hash()` and its seven siblings in `CryptoHasher.rs`. They no longer fetch the VM's engine first, which on the first call also meant creating it. `Bun.CryptoHasher` goes through `EVP.rs` and is not touched: dropping the engine there reaches into the VM's `RareData` and three functions exported to C++, so it is a change of its own. Also gone, because nothing is left for them to do: - `s3_signing`'s `boring_engine()` helper, which returned null, and the `ENGINE` re-export that only it used. That was the last use of `sha::ffi` from another crate, so the module is `pub(crate)` now. - `exe_format`'s `sha256_hash`, which wrapped the `unsafe` call. Its six callers call `SHA256::hash` directly. One of them, the Mach-O signer, hashed each page into a temporary and copied it into the signature. It now hashes straight into the slot. **15 fewer `unsafe` blocks and 1 fewer `unsafe fn`** (a macro stamps it out once per hasher), +34/−102. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes on Linux. The change has no platform-specific code. - `cargo test -p bun_exe_format` passes (237 tests). It covers the hashing of Mach-O code signatures and of build ids. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/js/bun/util/bun-cryptohasher.test.ts` | 109 | 0 | yes | | `test/cli/install/bun-install-tarball-integrity.test.ts` | 165 | 0 | yes | | `test/cli/install/bun-pm.test.ts` | 154 | 0 | yes | | `test/js/bun/s3/s3-storage-class.test.ts` | 9 | 0 | yes | | `test/js/bun/s3/s3-requester-pays.test.ts` | 10 | 0 | yes | | `test/js/bun/s3/s3-list-objects.test.ts` | 40 | 0 | yes | | `test/js/sql/postgres-auth.test.ts` | 101 | 0 | yes | | `test/js/sql/sql-mysql-auth-short-nonce.test.ts` | 10 | 0 | yes | | `test/js/sql/sql-mysql-duplicate-auth-switch.test.ts` | 6 | 0 | yes | | `test/js/sql/postgres-duplicate-auth-request.test.ts` | 17 | 0 | yes | | `test/js/sql/wire-frames.test.ts` | 10 | 0 | yes | | `test/js/sql/sql-mysql.auth.test.ts` | 37 | 0 | yes | Total: 668 pass, 0 fail. I also ran two scripts on both builds, with the same output: - Each of the eight `Bun.SHA*` / `Bun.MD*` classes on five inputs (empty, 3 bytes, 64 and 65 bytes, 1 MB): `hash()` to hex and into a caller's buffer, compared with the incremental digest. Also `Bun.CryptoHasher`, and `node:crypto` as a cross-check. - `bun install` of a local tarball, which computes its sha512. `bun pm hash` and `bun pm hash-string`. A `--frozen-lockfile` reinstall, which verifies the integrity. The same reinstall after I corrupted the integrity in `bun.lock`, which fails with `Integrity check failed`. `postgres-auth.test.ts` authenticates with SCRAM-SHA-256 against a real Postgres server, and `sql-mysql.auth.test.ts` with `mysql_native_password` and `caching_sha2_password` against a real MySQL server. Those cover the call sites in `SASL.rs` and `Auth.rs`. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…afe (oven-sh#47250) ### What does this PR do? `Entry::kind` and `Entry::symlink` stat a cached directory entry the first time someone asks. Both were `unsafe fn`s because they took `fs: *mut R` and formed `&mut *fs` to call `EntryKindResolver::resolve_kind(&mut self, ..)`. So each of their 22 call sites was an `unsafe` block with a comment saying that the pointer is the global `RealFS`. `resolve_kind` does not need `&mut`. Its only implementation, `RealFS::kind`, reads `self.cwd`, calls `self.need_to_close_files()` (a `&self` method) and makes syscalls. So: - `RealFS::kind` and `EntryKindResolver::resolve_kind` take `&self`. - `Entry::kind` and `Entry::symlink` take `fs: &R` and are safe functions. Their `# Safety` section asked for two things. A valid pointer is what the reference now states. That `resolve_kind` does not re-enter the entry's mutex is about a deadlock, not about memory safety, and is now a doc comment on `resolve_kind`. - Callers pass a shared reference: `Resolver::fs_ref()`, `vm.fs()` or a local `&FileSystem` where they have one, and `FileSystem::get()` where they had only the raw pointer. In the resolver it is spelled `self.rfs_ref()`, a one-line safe accessor for `&self.fs_ref().fs`. **One caller needed more than that.** `bun test`'s file scanner calls `kind` from the per-entry hook of `RealFS::read_directory_with_iterator(&mut self)`, so while the hook runs, that `&mut RealFS` is live further up the stack and no independent `&RealFS` can honestly exist. The hook is now lent the borrow: `readdir`, which never used its `&mut self`, takes `&self` and passes itself to `DirEntryIterator::next`. For that, the hook call moves from `DirEntry::add_entry_with_store` up into `readdir`, its only caller with a real hook. `add_entry_with_store` returns the entry it stored and loses its type parameter, and `IS_VOID` and the forwarding `impl DirEntryIterator for &T` have no use left. The hook gets the entry as `&Entry`. It only calls `&self` methods on it, and the map's key can point into the entry, so a `&mut` there would be wrong. This also removes one source of aliased `&mut`. `Entry` is built for use from several threads: it has a per-entry mutex and a lock-free fast path, and most `kind` calls in the resolver run outside the global resolver mutex. Two threads that stat different entries each made their own `&mut` to the one global `RealFS`. Other code still forms `&mut RealFS` from a raw pointer, for `read_directory` and the entries map. That is not changed here, except for two places next to a `kind` call that formed one only to lock `entries_mutex`, which takes `&self`. With `kind` an ordinary `&self` method, some code around it is no longer needed: - Four locals in the resolver that held `rfs_ptr()` only to feed `kind`, three of them with a paragraph on Stacked Borrows. A fifth, in `load_as_file`, is down to one use and is inlined, without its paragraph. - Four `let entry = lookup.entry();` locals that were used once, under a `SAFETY` comment although the call is safe. - `ResolverLike::fs_impl`, which has no users left. - In `RouteLoader::load`, three separate `unsafe { &*entry_ptr }` reborrows that existed "so `&` reads and the `kind()` call do not overlap". One remains. - `Route::parse` was an `unsafe fn` taking `*mut Entry` because "a `&mut Entry` parameter would alias" its other arguments. All it ever did with the pointer was `&*entry`, so it takes `&Entry` and is safe. Its two remaining internal `unsafe` blocks depend only on lengths that it computes from its own buffer. **30 fewer `unsafe` blocks and 3 fewer `unsafe fn`s**, +91/−251 over 8 files. No new `unsafe`. A reference and a raw pointer are passed the same way, and the hook call is in the same loop as before, so there is no cost. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux, macOS and Windows. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/js/bun/resolve/resolve.test.ts` | 107 | 1 | yes | | `test/cli/test/bun-test.test.ts` | 107 | 2 | yes | | `test/js/bun/util/filesystem_router.test.ts` | 51 | 0 | yes | | `test/cli/run/filter-workspace.test.ts` | 131 | 0 | yes | | `test/bake/framework-router.test.ts` | 37 | 0 | yes | | `test/cli/bun.test.ts` | 48 | 0 | yes | | `test/js/bun/resolve/import-meta.test.js` | 34 | 0 | yes | | `test/js/bun/resolve/resolve-ts.test.ts` | 27 | 0 | yes | | `test/js/bun/resolve/resolver-permission-denied-ancestor.test.ts` | 2 | 0 | yes | | `test/bundler/esbuild/packagejson.test.ts` | 94 | 0 | yes | | `test/bundler/esbuild/tsconfig.test.ts` | 21 | 0 | yes | | **Total** | **659** | **3** | | The three failures are the same three on `main`: - `loads reflect-metadata before tsyringe` needs the `tsyringe` package, which is not installed on this machine. - `instances of a class nested too deeply to print are a RangeError` and `the data of events nested too deeply to print are a RangeError` test a stack overflow in the value printer, which this PR does not touch. I also ran these on both builds, in directories that contain plain files, directories, symlinks to files, symlinks to directories and dangling symlinks. The output is the same: - **`bun test` discovery**, which is the path that goes through the hook: ten runs over a tree with nested directories, `node_modules` and dot directories, a directory named like a test file, a FIFO, and symlinks that lead out of the project. No filter, name filters, a path, a filter that matches nothing, `--path-ignore-patterns`, and runs from two subdirectories. Every test file prints its own path, so this compares the set of files found. - **Resolver:** `Bun.resolveSync` on 38 specifiers (a package `main` without extension, `exports`, `dir/index`, a file and a directory with the same name, `.js` that resolves to `.ts`, `tsconfig` paths), with and without `--preserve-symlinks`, then the same program through `bun build` and run again. Also a build with missing imports, an unreadable directory, and a file used as a directory. - **`Bun.FileSystemRouter`:** its routes, nine `match()` calls, and `reload()` after a file is added. - **`bun run --filter`** with workspace globs, five filters. - **`bun getcompletes`** for bins, scripts and files. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…h#47249) ### What does this PR do? `bun_core::ffi::zeroed::<T>()` is the safe way to zero-initialize a value: it needs `T: Zeroable`, and `impl_zeroable!(T)` only compiles if rustc can prove from the layout that all-zero is a valid `T`. 27 places zeroed a value with `unsafe` and a hand-written SAFETY comment, through `zeroed_unchecked()` or `MaybeUninit::zeroed().assume_init()`, for types that need no escape hatch. - **14 sites only needed the safe call**, because the type is already `Zeroable`: `sockaddr_storage` (3), `addrinfo`, `statfs`, `[pollfd; 2]`, `EVP_MD_CTX`, and on Windows `FILETIME` (4), `CONSOLE_SCREEN_BUFFER_INFO`, `BY_HANDLE_FILE_INFORMATION` and the directory iterator's name buffer. That last one is an associated type, which now has the bound `NameData: Zeroable`. - **7 sites needed the proof to be declared:** `libc::sockaddr_un`, `libc::rusage_info_v4`, `uv_utsname_t` (2), BoringSSL's `SHA_CTX` and `SHA256_CTX` (one site, in a macro), lsquic's `NqTransportParams` and `bin::Value`. Each is one `impl_zeroable!` entry that the compiler checks. For `bin::Value`, a union with a `()` member, the check proves little. Its other members are `Zeroable` through impls of their own. - **6 sites read better without zeroing at all:** `timespec { sec: 0, nsec: 0 }` in `Futex.rs`, a three-field literal for `Iccp` (the file already builds one that way in the encoder), `LastResult { e: 0 }` in `NativeBrotli.rs`, `static MUTEX: Mutex = Mutex::new()`, a literal for `uv_stdio_container_t`, and `EventLoopHandle::init(null)` for the placeholder in `WindowsOptions::default()`. That last one zeroed a `repr(Rust)` enum and relied on which variant is discriminant 0. `init(null)` is the sentinel that type documents for this use. The hashers in `sha.rs` zero their inner C context, `Self { hasher: zeroed() }`. The public wrapper types are not `Zeroable`, so `init()` stays the only way to make one. Four more values, three `sockaddr_storage` in `node_cluster_binding.rs` and a `winsize`, sit inside `unsafe` blocks that are still needed for the call next to them. They use the safe call too, and the count does not include them. **27 fewer `unsafe` blocks**, +75/−106 over 19 files. No hand-written `unsafe impl`, no new helper, and the same bytes as before. The 9 `zeroed_unchecked()` calls that remain hold pointers or are generic, which is what the escape hatch is for. Comments, in the files this touches: ten `// SAFETY: all-zero is valid` comments sat above calls that were already the safe `zeroed()` and are removed, and six `// SAFETY:` comments in `node_cluster_binding.rs` that stopped in the middle of a sentence are completed. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux glibc and musl, macOS, Windows, FreeBSD and Android. For the eleven Windows-only sites and the one macOS-only site this type check, which includes the layout proofs, is the only verification I could do. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/js/bun/image/image.test.ts` | 104 | 0 | yes | | `test/js/node/zlib/zlib.test.js` | 473 | 0 | yes | | `test/js/bun/udp/dgram.test.ts` | 62 | 0 | yes | | `test/js/bun/util/bun-cryptohasher.test.ts` | 109 | 0 | yes | | `test/js/bun/spawn/spawnSync.test.ts` | 18 | 0 | yes | | `test/js/node/cluster/worker-set-up-by-the-first-to-ask.test.ts` | 1 | 0 | yes | | `test/cli/install/bun-patch.test.ts` | 232 | 0 | yes | | `test/bundler/cli.test.ts` | 44 | 0 | yes | | `test/js/web/websocket/websocket-accept-header-validation.test.ts` | 2 | 0 | yes | | **Total** | **1045** | **0** | | I also ran a script on both builds. Its output is the same: - **Hashers:** the eight `Bun.SHA*` and `Bun.MD*` classes, one-shot and incremental, over five inputs from empty to 1 MiB. `Bun.CryptoHasher` with eight algorithms, including `copy()`. `node:crypto` hashes and HMACs. A WebSocket handshake, where the client checks `Sec-WebSocket-Accept` with the `SHA_CTX` hasher. - **`bun patch --commit` with `BUN_FEATURE_FLAG_DISABLE_MEMFD=1`.** On Linux the output of `git diff` normally goes to a memfd. With that off it is read through the `[pollfd; 2]` loop, and `strace` confirms the `poll` calls. The patch is then hashed with the `SHA_CTX` hasher. This compares the patch file, `package.json`, and a reinstall that applies the patch. - **`bun build` with `BUN_FEATURE_FLAG_FORCE_IO_POOL=1`**, the only mode that uses the `static MUTEX`: 41 modules, and the bundle is then run. - **`fs.statfsSync`**, also with `bigint` and on a missing path, and a **Brotli** round trip and decode error. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
oven-sh#47273) ### What does this PR do? Five separate changes. Each has its own commit, and a sixth commit holds small follow-ups to 2, 4 and 5. Each change removes hand-written pointer code. Most of it becomes plain borrows that the compiler checks. The rest goes through safe constructors and accessors that already exist. Together: **70 fewer `unsafe` blocks**, +299/−765 over 58 files. There is no new `unsafe`, no safe function became `unsafe`, and nothing is added at run time. | # | Change | `unsafe` blocks | Diff | |---|---|---|---| | 1 | The bundler reads source files through `&mut cache::Fs` | −8 | +35/−97, 6 files | | 2 | `NonNull`s are made from references and boxes without `new_unchecked` | −16 | +56/−148, 17 files | | 3 | Four callback contexts borrow instead of holding raw pointers | −7 | +36/−78, 5 files | | 4 | JS codegen workers borrow the linker, the chunk and the renamer with `&` | −17 | +108/−285, 12 files | | 5 | Readers of the `FileSystem` singleton call `FileSystem::get()` | −22 | +67/−160, 22 files | The commits can be reviewed one at a time. Four files are touched by two commits: `bundler/transpiler.rs`, `resolver/lib.rs` and `resolver/resolver.rs` by 1 and 5, and `jsc/VirtualMachine.rs` by 3 and 5. Commits 1 to 3 come from oven-sh#47261, oven-sh#47265 and oven-sh#47266, which this PR replaces. #### 1. The bundler reads source files through `&mut cache::Fs` `cache::Fs::read_file_with_allocator` took `_fs: &mut FileSystem`. All it does with it is call `need_to_close_files()`, which takes `&self`, and all five callers passed the process-wide singleton. To hold a `&mut` to that singleton next to the `&mut` borrow of the cache, they went through `unsafe`. - **The parameter is gone.** The function asks `FileSystem::get()`, the existing shared accessor. - **The bundler's source readers take what they use.** `get_code_for_parse_task`, `get_code_for_parse_task_without_plugins` and `OnBeforeParsePlugin` carried a `*mut Transpiler` and a `*mut Resolver`. The pointers were raw because the resolver is a field of the transpiler, so a `&mut` to each would alias. They used the transpiler only for `fs` and the resolver only for `caches.fs`, so they now take `&mut cache::Fs`. - **`get_source_code` is plain safe code.** It borrows `transpiler.options.loaders`, then `transpiler.resolver.caches.fs`, then calls `transpiler.reset_store()`, and the borrow checker accepts that as written. - `get_code_for_parse_task_without_plugins` also took a `_loader` that it never read. That is gone too. - `PackageJSON::parse` still needs a `&FileSystem` for paths. It gets it from `FileSystem::get()` instead of `unsafe { &mut *r.fs() }`. #### 2. `NonNull`s from references and boxes without `new_unchecked` 22 calls of `NonNull::new_unchecked(p)` were on a pointer that the code had just taken from a reference, a slice, a `Box`, a `Vec` or a `RefPtr`, with a SAFETY comment saying that such a pointer is never null. The standard library and `bun_core::heap` already have safe constructors for exactly that. The repo calls `NonNull::from` about 150 times and `heap::alloc_nn` / `heap::into_raw_nn` about 30 times. `NonNull::from_ref` and `from_mut` are the same conversion as `NonNull::from`, and unlike it they can be called in a `const fn`, which four of these sites are. | From | Was | Now | |---|---|---| | a reference or slice (`StoreRef::from_static`, `StoreStr::new`, `StoreSlice::new` and `new_mut`, `StringHashMapKey::borrowed`, two in `finish_arena_contents`, `crypto.randomBytes`) | `new_unchecked(s.as_ptr().cast_mut())` | `NonNull::from_ref(s).cast()` / `from_mut` | | a `Box` that is given up (`BSSList`'s overflow block, `StringHashMapKey::owned`) | `new_unchecked(Box::into_raw(b))` | `Box::into_non_null(b)` / `Box::into_non_null_with_allocator(b).0` | | a new heap object (`ConcurrentTask::create`, the spawn wait queue, `readdir`'s result list, the FSEvents loop, `IPCInstance::new`, `WatcherAtomics::init`) | `new_unchecked(heap::into_raw(Box::new(..)))` | `heap::alloc_nn(..)` / `heap::into_raw_nn(..)` | | a `Vec` that is given up (`Encoded::from_owned`) | `ManuallyDrop` and `new_unchecked(slice_from_raw_parts_mut(v.as_mut_ptr(), v.len()))` | `v.into_parts()` and `NonNull::slice_from_raw_parts` | | a `RefPtr` (two in `Bun.spawn`) | `new_unchecked(p.as_ptr())` | `p.as_non_null()` | - `heap::alloc_nn` and `heap::into_raw_nn` were themselves `NonNull::from(Box::leak(b))`. The documentation of `Box::leak` says to prefer `Box::into_non_null` for memory that is freed later, so they now call that. It compiles to the same pointer copy. - `ConcurrentTask::new`, which only `create` called, is folded into it. The FSEvents loop boxed a placeholder task and then overwrote every field through a helper with one caller. It now builds the task directly. - The job behind `crypto.randomBytes(n, callback)` kept a `JsPtr<u8>` and a length, and rebuilt the slice with `from_raw_parts_mut`. It keeps a `JsPtr<[u8]>`, like the scrypt job in the same file. - Almost all `new_unchecked` calls that remain start from a raw pointer: an FFI result, a `container_of` walk, pointer arithmetic, or the argument of an `unsafe fn`. #### 3. Four callback contexts borrow instead of holding raw pointers Four places build a context on the stack and pass it through a `void*` to a callback that runs before the call returns. Each kept raw pointers to other locals in that context, and the callback turned each pointer back into a reference in its own `unsafe` block. The context only lives for one synchronous call, so it can hold the borrows themselves. The other contexts of this kind are already written that way: `TablePrinter`'s `Ctx<'c, 'a>`, `URLSearchParams`' `Wrap<'a, Ctx>`, and the `ExpectedEntry<'a>` of `toBeOneOf` and `toContainEqual`. | Context | Used for | Held | Holds now | |---|---|---|---| | `AggCtx` | printing the members of an `AggregateError` | `*mut Formatter`, `*mut Writer`, `*mut ExceptionList` (null for none) | `&'c mut Formatter`, `&'c mut Writer`, `Option<&'c mut ExceptionList>` | | `ExpectedEntry` | `expect(iterable).toContain()` | `*const JSGlobalObject`, `*mut bool` | `&'a JSGlobalObject`, `&'a mut bool` | | a tuple | `AnyWebSocket::cork` | `(*mut C, fn(&mut C))` | nothing: `cork` takes the closure itself, as `Response::corked` does | | `Erased` | `PackageManager::sleep_until` | `*mut C` | `&'c mut C` | The one `unsafe` block left in each callback is the cast of the context pointer itself. A reference has the layout of a raw pointer and `Option<&mut T>` is a nullable pointer, so each callback does the same loads and the same null test. `cork` had one caller, which passed a closure and `|call| call()`, so each corked WebSocket event now makes one indirect call fewer. A comment above `cork` pointed to `NewWebSocket::cork`, which does not exist. It is removed, with the empty section heading of the same deleted type. #### 4. JS codegen workers borrow the linker, the chunk and the renamer with `&` Each part range of a JS chunk is printed by its own task on the thread pool. Every such task formed a `&mut LinkerContext` and a `&mut Chunk` from the shared raw pointers, so while a chunk is printed, many threads hold a `&mut` to the same two objects. The code said so itself: the worker's SAFETY comment noted that peer tasks hold their own `&mut` views "for read-only printer use", and the note on `unsafe impl Sync for Chunk` listed the renamer as an exception whose borrow "should become `&'r`". Nothing on that path writes through either object. The `&mut` came from a few signatures that asked for more than they use: - **`Renamer` held `&mut` to the underlying renamer.** All three `name_for_symbol` implementations only read. `MinifyRenamer`'s took `&mut self` because `TinyString::slice` did, under a comment carried over from Zig about the compiler copying the struct, which cannot happen behind a Rust `&self`. `Renamer<'a>` now holds shared references and is `Copy`. - **`should_remove_import_export_stmt`, `convert_stmts_for_chunk`, `convert_stmts_for_chunk_for_dev_server` and `generate_code_for_file_in_chunk_js`** took `&mut LinkerContext`, and two of them `&mut Chunk`. They take `&`. - **`module_preload_registration`** took both by `&mut`, next to a `&[Chunk]` that contains the same chunk. - **`Printer::mangled_prop_name`** took `&mut self`, and `post_process_js_chunk` borrowed the runtime's module scope with `&mut`. Both only look names up. With those fixed, the workarounds around them are not needed: | Where | What goes away | |---|---| | `Printer::name_for_symbol` | It erased the name's lifetime through a raw pointer, because a borrow of `&mut self.renamer` got in the way of the next `print` call. `Renamer::name_for_symbol` now returns `&'a [u8]`. | | `print_code_for_file_in_chunk_js` | A `transmute` of the `Renamer` to other lifetimes. The `&mut` inside made it invariant. | | `generate_entry_point_tail_js` | `detach_lifetime` on `import_records`, for the same reason, and an explicit `'a` on four parameters. | | `generate_compile_result_for_js_chunk_impl` | `chunk.renamer` split off through a raw pointer. | | `generate_code_for_file_in_chunk_js` | `parts` held as `*mut [Part]` and dereferenced in four places, `parts_live` held as a `BackRef`, and `parse_graph` read through the raw field, all to get around its own `&mut c`. `ExportHoist`, the context of a callback in that function, held `BackRef`s to the arena and to `c.graph.symbols`. It now borrows both. | | the JS and CSS workers | `c_ptr` and `chunk_ptr` dereferenced by hand. They use `ctx.c.get()` and `ctx.chunk.get()`, as the HTML worker already does, and `pending_part_range_prologue` no longer returns `c_ptr`. | | the CSS worker and `prepare_css_asts_for_chunk_impl` | A raw-pointer cast of `&c.graph.symbols` from `symbol::Map` to `symbol::Map`, once in each. In the first, also a `Vec` made with `from_borrowed_slice_dangerous` only to be passed as a slice. Without it the `Layers` and `ExternalPath` arms were identical, so they are one arm now. In the second, also `parse_graph` read through the raw field. | The old SAFETY comment in `Printer::name_for_symbol` said that the bytes always live in the AST arena or in the source. A minified name of up to 15 bytes is stored inside the renamer itself. The new signature ties the slice to the borrow of the renamer, which covers that case. Of the 17 blocks, 11 are removed outright. The other six were hand-written dereferences that now go through the existing safe accessors `ParentRef::get`, `BackRef::get` and `LinkerContext::parse_graph()`. Two things are left as they are: - For `--format=internal_bake_dev`, `convert_stmts_for_chunk_for_dev_server` still updates the file's own import records. It does that through its local `ast` view, not through `c`. In the dev server each file has one part range, so one task. `bun build --format=internal_bake_dev` with several entry points and no `--splitting` can print one file in two chunks, so there two tasks can still touch the same records. This PR does not change that. - `post_process_js_chunk` keeps `&mut Chunk`. It runs once per chunk and really writes to it. It also still takes `&mut LinkerContext` through `ctx.c()`, one task per chunk, so that later phase still has the aliasing that this change removes from printing. #### 5. Readers of the `FileSystem` singleton call `FileSystem::get()` `Transpiler` and `Linker` each kept a raw copy of the pointer to the process-wide `FileSystem`. Code that wanted the working directory or one of the string stores dereferenced a copy by hand. None of that code writes to the `FileSystem`. - **`Transpiler.fs` and `Linker.fs` are gone**, and with them `Transpiler::fs()`, `Transpiler::fs_mut()` and `Linker::fs()`. `fs_mut` gave out a `&mut FileSystem` with an unbounded lifetime from a `&self`. Its two callers, both in `process.chdir()`, only read the working directory. - **Readers call `FileSystem::get()`**, the existing shared accessor, as 45 other places on `main` do. That covers six sites in `bun run`, one in `bunx`, one in the bake router, `VirtualMachine::fs()`, `Scanner::fs()` and the former callers of the accessors. The sites that only wanted the filename store call `FilenameStore::instance()`. - **`bun create` and `bun upgrade`** turned the pointer that `FileSystem::init` returns into a reference, three times out of four a `&mut`. They only read, so they call `init` and then `get()`. `Example::fetch_all_local_and_remote` and `FileSystemTmpdirExt::tmpdir` take `&`, so the other two callers of `tmpdir` no longer need `FileSystem::instance()`, which returns a `&'static mut`. - **`BundleOptions::from_api`** took `&mut FileSystem` and passed `&mut fs.fs` through `init_external_modules` to `validate_path`, which ignored it. The parameter is gone from all three. - `wire_after_move` copied the pointer into the resolver again. `Resolver::for_worker` has already done that. The resolver keeps its pointer, because it really writes to the directory cache. So does the scanner of `bun test`, for `read_dir_with_name`. It copied `Transpiler.fs` and now asks `Resolver::fs()`, which returns the same value. No other crate reads the field `Resolver.fs` or the alias `fs::Implementation` any more, so both are `pub(crate)`. The compiler does not prove these reads safe. They now rest on the one `unsafe` inside `FileSystem::get()`, which is the accessor of a `static`, instead of on a separate `unsafe` at each site. In a release build `get()` is the address of that static, so no reader does more work than before, and `Transpiler` and `Linker` are each one pointer smaller. The readers that go through `FileSystem::instance()` are a much larger set and are left for a follow-up. ### How did you verify your code works? Behaviour is unchanged, so there is no new test. - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux, macOS and Windows. The FSEvents loop is macOS-only, one half of the IPC setup is Windows-only, and `Encoded::from_owned` is compiled for macOS and Windows only, so for those three this type check is the only verification I could do. - On a debug + ASAN build I ran the test files below, and ran the same files on a debug + ASAN build of `main`: | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/bundler/native-plugin.test.ts` | 19 | 0 | yes | | `test/bundler/bundler_plugin.test.ts` | 68 | 0 | yes | | `test/aot/build-plugins-native.test.ts` | 8 | 0 | yes | | `test/bundler/esbuild/packagejson.test.ts` | 94 | 0 | yes | | `test/bundler/esbuild/tsconfig.test.ts` | 21 | 0 | yes | | `test/bundler/esbuild/css.test.ts` | 68 | 0 | yes | | `test/js/bun/resolve/resolve.test.ts` | 107 | 1 | yes | | `test/bundler/esbuild/default.test.ts` | 155 | 0 | yes | | `test/js/node/fs/fs.test.ts` | 631 | 0 | yes | | `test/js/bun/spawn/spawn.test.ts` | 182 | 0 | yes | | `test/bundler/transpiler/transpiler.test.js` | 258 | 0 | yes | | `test/js/node/child_process/child_process_ipc.test.js` | 1 | 0 | yes | | `test/js/node/crypto/crypto-random.test.ts` | 27 | 0 | yes | | `test/js/bun/test/expect.test.js` | 427 | 0 | yes | | `test/js/bun/util/inspect-error.test.js` | 46 | 0 | yes | | `test/js/bun/util/reportError.test.ts` | 7 | 0 | yes | | `test/js/bun/websocket/websocket-server.test.ts` | 218 | 0 | yes | | `test/cli/install/bun-add.test.ts` | 220 | 0 | yes | | `test/bundler/bundler_cjs2esm.test.ts` | 113 | 0 | yes | | `test/bundler/bundler_merged_enum.test.ts` | 2 | 0 | yes | | `test/bundler/bundler_minify.test.ts` | 67 | 0 | yes | | `test/bundler/esbuild/importstar.test.ts` | 99 | 0 | yes | | `test/bundler/esbuild/splitting.test.ts` | 26 | 0 | yes | | `test/bundler/esbuild/ts.test.ts` | 59 | 0 | yes | | `test/bundler/bundler_html.test.ts` | 26 | 0 | yes | | `test/bake/dev/bundle.test.ts` | 26 | 28 | yes | | `test/bake/dev/esm.test.ts` | 11 | 7 | yes | | `test/cli/install/bun-run.test.ts` | 319 | 0 | yes | | `test/cli/run/run_command.test.ts` | 33 | 0 | yes | | `test/cli/install/bun-create.test.ts` | 139 | 1 | yes | | `test/js/node/process/process.test.js` | 194 | 4 | yes | | `test/cli/test/bun-test.test.ts` | 107 | 2 | yes | | **Total** | **3778** | **43** | | The 43 failures are the same 43 tests on both builds: - 35 in `test/bake/dev` drive a browser page through the `happy-dom` package, and 1 in `resolve.test.ts` needs `tsyringe`. Neither package is installed on the machine I used. - 1 in `bun-create.test.ts` runs `bun create` with no template, which downloads the list of examples. - 4 in `process.test.js` check the ICU version, the host's Node.js version, `process.getgroups()` and `process.execArgv`. - 2 in `bun-test.test.ts` expect a `RangeError` for values nested too deeply to print. I also ran these on both builds. The output, 5,473 lines in all, is the same: - **Bundler (1, 2, 4):** 14 `bun build` commands over a small project, across formats, targets, minify flags, source maps, `--splitting` and `--format=internal_bake_dev`, and then five of the bundles. The output includes the bundles and the source maps. - **CSS (4):** bundles of a stylesheet that imports external URLs and local files with `layer()`, `supports()` and media conditions, and of a CSS module, plain and minified. - **Resolver (1):** `Bun.resolveSync` on 38 specifiers in a project with several `package.json` and `tsconfig.json` files, with and without `--preserve-symlinks`, then the same program through `bun build`. - **`NonNull` sites (2):** - sources that are plain, start with a UTF-8 BOM, are UTF-16 with a BOM, and are empty - `require` in a directory of 12,000 files, which reaches `BSSList`'s overflow block - recursive `fs.promises.readdir` - 200 async file operations in flight at once - 24 children at once with `BUN_FEATURE_FLAG_FORCE_WAITER_THREAD=1` - a child that talks to its parent over IPC, JSON and advanced, from `Bun.spawn` and from `child_process.fork` - `crypto.randomBytes` and `crypto.randomFill` with a callback - **Callback contexts (3):** - `AggregateError`s printed, reported, rejected and thrown - `toContain` on sets, generators and map iterators, passing and failing - `ws.cork()` in `open` and `message` handlers, with a return value, a throw and a `close()` inside the callback - `bun install`, `add` and `remove` with local tarballs - **`FileSystem` readers (5):** - `bun run` of scripts, files, `.`, local binaries, a system binary and missing names, from the project root and from a subdirectory - `bun x` of local binaries - `process.chdir()` that succeeds and that fails - `bun build --external` with relative, absolute and wildcard paths - `bun create` from local templates in three places - `bun getcompletes` - `Bun.FileSystemRouter` - `bun test` file discovery over files, directories and symlinks --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…compiler docs (oven-sh#47129) Follow-up to oven-sh#45631. Upgrades WebKit to oven-sh/WebKit `main` at f495db1d2488 (oven-sh/WebKit#930), which holds the engine's half of this work. Most of this is behind `--aot`. The first two sections list what is not. ### Changes that reach Bun without `--aot` - **`node:path`:** the native functions take the width of a substring's characters from its base's storage as it is now. `test/js/node/path/string-forms.test.ts` covers it. - **Heap snapshots:** `Bun.generateHeapSnapshot()`, its V8 form and the heap profiler release the profiler's snapshot once the JSON is built. - **`execve`:** the suspension lock is held over `execve`. - **`process.env` on Windows:** `Object.preventExtensions()`, `Object.seal()` and `Object.freeze()` throw a `TypeError`, as in Node. - **Module loader:** follows `VM::synchronousModuleQueue()` from the WebKit upgrade. - **Windows:** the crash handler registers with `WTFSetAbortReporter()`. - **Parser:** a class records whether `useDefineForClassFields: false` turned its fields into assignments in the constructor. `declare` is a name on the left of `for-of`, `for-in` and `for await`, and before every kind of line break. Any of the language's line breaks may follow `global`. - **`bun check`:** a ring of functions without annotations resolves in linear time. Only callable built-ins join a function type. The type of a well-known symbol is `symbol`. `--timing` prints the number of queries. ### `bun build --compile` - **`Bun.build({ compile })`:** `import.meta.main` is inlined for the entry point, as on the command line. - **Windows:** where the section table has no free slot for the payload's header, the sections' data is laid out again behind a table that has one. - **macOS:** the executable is built and signed over a private mapping of the template and written page by page, so the process never holds the whole file in memory. - **macOS:** a compiled executable no longer starts a background thread at boot to load the system CA certificates. They load on first use, on the thread that asks, as before oven-sh#45631. The flag for it, and the search of the output for `getCACertificates` that set the flag without `--aot`, are removed with their tests. - **`node:fs` on paths inside an executable:** the asynchronous calls on Windows name each path in an error as the caller wrote it. Asynchronous `statfs()` answers for the file system that holds the executable. - **`--trim`:** - On macOS, stack traces and the allocator's override know the several code segments of a trimmed executable. - The two TLS hooks that only the WebSocket client calls go with it. - On Windows, the tool tells a member of an archive from a file of the same name. - On Windows, a build on a trim template fills holes and joins pieces over them, so it cuts what a build on Bun cuts. ### The ahead-of-time compiler - **Modules:** - One set of `__esModule` marks serves both kinds of module. - Export tables of re-exporting files are limited like those of other files. - A name taken from a JSON import loads the files in between, so files that a package lists under `sideEffects` run. - A name that a library module lacks can be read off its namespace. - **CommonJS:** under `"type": "module"`, a default import holds what the link captured. It takes the short path only where nothing stores to `module.exports` after the body ends. - **Module names from lists and tables:** a walk of a list (`forEach`, `map`, `filter`, `find` and so on) counts as leaving the list unchanged only under these conditions: - its callback is written in the call - the callback has no rest parameter - the callback cannot reach itself through `arguments`, a direct `eval` or its own name - **Projects with a second `tsconfig.json`** take what the application's load found: `import.meta.main`, file identities, erased sites and folded constants. - **Classes:** - An unused class is removed whole only where its heritage is proven. - Methods stored on a constructor function's `prototype` are members of its layout. - **Workers:** - Whether the files and texts that Workers start from form one program with the rest is decided after the front end, before the engine's first run. - A Worker that finds no room for its thread waits for siblings that are ending, and is tried again on a timer. - **`node:http`:** only what `new ServerResponse()` makes takes the compact form. An instance of a subclass, and an object that `ServerResponse` is called on, keeps its state on itself. - **Build:** a third list of the engine's sources is compiled for minimum size. Dependencies that no code of a crate uses are gone from 13 `Cargo.toml` files. ### Docs `docs/compiler` is rewritten and about 5,800 lines shorter. Each page keeps a runnable example, which `test/cli/aot-docs.test.ts` runs. `docs/snippets/compiler-internals.jsx` had no importer left and is removed. The table of libraries known not to work moves next to its test. ### Tests and CI The ahead-of-time compiler's tests run in the ordinary `test-bun` shards. The `test-aot` and `test-aot-drawn` steps, their path triggers, draws and shares are gone, and so is every step that was allowed to fail. Every pull request gets the same pipeline. - **Lanes:** every test of the compiler runs on the two Alpine lanes. The other lanes run the ones that ask something of the system: processes, Workers, the environment, files, sockets, memory limits, the image, native code. Windows also runs the `.ir` tests. The lane with AddressSanitizer runs all but a list of files that feed it nothing new, on a copy of Bun without its debug information. The lists are in `scripts/runner.node.ts`, and a test file that says `// Runs alone:` stays out of the bucket of files that run side by side. - **Whole programs** are small files under `test/aot/parallel/`. Each passes when its executable exits and prints as `bun <file>` does, on Alpine also with its types erased. The runner runs them side by side, as it does Node's tests. - **Source to IR:** files under `test/aot/ir/` with `// RUN: lower` and `// CHECK:` lines. `cargo test` runs them in one Buildkite step. `AOT_IR_UPDATE=1` writes everything below a `// GENERATED:` line again. - **IR to IR, and IR to B3:** `.ir` files under `test/aot/engine-tests/`, run by `testaot` with Bun's table of built-ins. `bun test/aot/tools/update-ir-tests.ts` writes a file's input from its source and its expectation from its input. - **Servers, frameworks and packages** are ordinary test files: `test/aot/servers/`, `test/aot/frameworks/`, `test/aot/backend-packages/`. - **Frameworks:** Next and Angular compile to one file. Nuxt and SvelteKit compile with `--compile-in-place`. Each of these four has caught a compiler bug that no smaller test showed. The Astro and React Router apps never did, and building the site is most of what these tests cost, so those two apps are removed. The twelve units (six apps, two forms each) become four test files. - **Generators** moved to `test/aot/fuzz/`, where `bun test/aot/fuzz/run.ts` draws from them. No test file starts one. - The stored programs were cut to those that cover an edge of the compiler that no cheaper one covers, and those that came in with a fix. - `bun run rust:test` runs the Rust crates' unit tests in the same Buildkite step. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
…7299) `cargo clippy` fails on `main`: ``` error[E0599]: no method named `fs` found for mutable reference `&mut transpiler::Transpiler<'_>` --> src/bundler/aot/testing.rs:84:39 ``` | PR | What it did | |---|---| | oven-sh#47273 | Removed `Transpiler::fs()` and rewrote its callers to `FileSystem::get()` | | oven-sh#47129 | Added `src/bundler/aot/testing.rs`, which calls `transpiler.fs()` | Each passed CI alone, and they merged one after the other. The file is behind `#[cfg(feature = "test_support")]`, so the build of bun is unaffected. The line now reads as oven-sh#47273 left the same field in `src/bundler/aot/mod.rs:119`. It is the only remaining call. Not compiled locally: this PR's `cargo clippy` is the check.
…hat values already have (oven-sh#47291) ### What does this PR do? 19 separate changes, one commit each. They follow oven-sh#47273 and work the same way: hand-written pointer code becomes plain borrows that the compiler checks, signatures name the lifetime that the value already has, and what was never needed is deleted. Together: **166 fewer `unsafe` blocks, 4 fewer `unsafe fn`s and 1 fewer `unsafe impl`**, +897/−1862 over 141 files. There is no new `unsafe`, and no safe function became `unsafe`. Nothing is added at run time except one null test per REPL start. **[Where the compiled code changes](#where-the-compiled-code-changes) lists every place where the program does different work.** | # | Change | `unsafe` | Diff | |---|---|---|---| | 1 | semver: the cached tail pointers come from plain borrows | −3 blocks | +43/−27, 2 files | | 2 | bake: the dev server's route trampoline forms its two references once | −16 blocks | +14/−36, 2 files | | 3 | crypto: no `ENGINE` is allocated or passed, because BoringSSL never reads it | −2 blocks | +43/−125, 7 files | | 4 | Readers of the `FileSystem` singleton use `get()`, not `instance()` | 0 | +129/−140, 61 files | | 5 | bundler: the arena and the chunk contexts are borrowed for as long as they live | −14 blocks, −2 `unsafe impl` | +79/−210, 8 files | | 6 | follow-ups to 2, 3, 4 and 5 | −2 blocks | +9/−37, 9 files | | 7 | bun test: the file scanner borrows its filters | −7 blocks | +31/−89, 3 files | | 8 | install: `LoadResult` no longer borrows the lockfile it was loaded into | −21 blocks | +141/−249, 16 files | | 9 | Return a borrow from one branch and keep going in the other | −5 blocks | +26/−100, 6 files | | 10 | Bun.openInEditor: return `Option`, own the argv, own the editor's name | −8 blocks | +80/−255, 3 files | | 11 | install: the cache directory lookup takes `&mut PackageManager` | −11 blocks, −2 `unsafe fn` | +19/−59, 1 file | | 12 | Use the borrow at hand in places that went through a raw pointer | −7 blocks | +60/−99, 4 files | | 13 | Declare 25 foreign functions without preconditions as `safe fn` | −32 blocks | +71/−107, 23 files | | 14 | Foreign functions that only need a live handle take a reference and are `safe fn` | −19 blocks | +66/−138, 9 files | | 15 | bundler: `LinkerGraph` keeps its own `Sync` impl | +1 `unsafe impl` | +7/−0, 2 files | | 16 | http: `build_request` returns `Request<'a>`, the lifetime its parts already have | −12 blocks, −1 `unsafe fn` | +15/−85, 6 files | | 17 | http: `HTTPClientResult::detach_lifetime` was safe code | −2 blocks, −1 `unsafe fn` | +9/−20, 5 files | | 18 | Follow-ups from a second review of 7 to 17 | −3 blocks | +124/−146, 25 files | | 19 | Follow-ups from a review of 18 | −2 blocks | +23/−32, 7 files | The commits can be reviewed one at a time. Commits 6, 18 and 19 hold review follow-ups, which are described with the change each belongs to. Commit 15 corrects 5. Change 1 also fixes undefined behaviour that Miri reports on `main`. Worth knowing before reading on: - **Changes 9 and 10 rely on the Polonius borrow checker**, which the pinned nightly has on by default. Change 9 says what that means for a toolchain bump. - **Change 8 leaves an aliasing problem in the lockfile migration as it is on `main`**, and now marks it with a `FIXME` instead of a `SAFETY` comment that denied it. ### Where the compiled code changes Nothing here is visible to a JS program or to someone using the CLI, apart from memory use in change 10. These are the places where the machine is asked to do different work: an allocation, a copy, a lookup, a branch, a store, or their order. They are what the tests and the output comparisons below are aimed at. | Change | What differs | Effect | |---|---|---| | 1 semver | `List::and_range` and `Group::or_range` take the cached tail pointer after the new `Box` is in the chain, not before. | Same address. Fixes undefined behaviour that Miri reports on `main`. | | 3 crypto | No `ENGINE` is allocated per VM, looked up in `RareData` per hash operation, or freed at teardown. The FFI calls pass null, and three C++ exports lose a parameter. | One allocation fewer per VM that hashes. BoringSSL never read the argument. | | 4 `FileSystem` | In builds with debug assertions, `get()` asserts that `init()` has run. | None in release builds: both functions are the address of the same static. | | 5 bundler | The linker graph is built once. It was built twice, and one copy dropped. | Less work per bundle. | | 7 `bun test` | One `Vec` fewer for the name filters (two fewer on Windows). The ignore patterns are built after the branch that parallel workers take, so workers skip them. The `concurrentTestGlob` list is moved, not borrowed, so bunfig's `Vec` is freed at start-up. | Same filters, same matches. | | 8 install | `bun install` and `bun pm scan` reset the cached root package id before they load the lockfile, as the other commands do. Seven commands read `options.node_linker` or `log_level` after the load instead of before. `LoadResult` is one pointer smaller. | The id is recomputed to the same value, and nothing writes those options during a load. | | 9 | `Snapshots::get_or_put` does one map lookup instead of two. `get_node_path` builds its string once. `ArrayHashMap::get_or_put_value` no longer recomputes its result from the index. `PathPair`'s iterator is `once().chain().filter()`. In debug builds `ZStr::from_buf` asserts the terminator. | Same results. | | 10 `Bun.openInEditor` | The argv is built on the JS thread and moved to the editor's thread, which frees it. The scratch buffer is freed when `open` returns, not when the editor exits. The editor's path is an owned `Vec`, not a copy in the dirname store. An absolute editor name skips the path buffer and the read of the working directory. | Same editor, same argv. A change of editor no longer leaves a copy that is never freed. | | 12 install | A `NonNull::new(..).expect(..)` is gone. The streaming extract builder reads `task_id` before it builds the task, not after. | Same values. | | 14 | Four functions are called through the generated `bun_jsc::cpp` wrappers instead of hand-written declarations: the same call and the same exception check, and in debug builds another recorded source location. `vm.global()` tests for null: the REPL's set-up makes that test once, where it made it only if `TZ` is set, and `Run::start` has one such call instead of eight. | The pointer is never null there. | Changes 2, 6, 11, 13, 15, 16 and 17 change only types, lifetimes, declarations and comments, and ask for the same operations in the same order. I have not compared binaries: a reference where there was a raw pointer can let the optimizer choose differently, and every moved line changes the panic locations. ### The changes in detail #### 1. semver: the cached tail pointers come from plain borrows `List` and `Group` are linked lists of boxes that also cache a pointer to their last node. **`Clone`** clones the boxes and then walks the new chain to find that node. The three walks (two in `List::clone`, one in `Group::clone`) kept the cursor as a `NonNull` and turned it back into a `&mut` on every step. The cursor is now a `&mut`, and it becomes a `NonNull` once, where it is stored: ```rust if let Some(mut tail) = out.head.next.as_deref_mut() { while let Some(ref mut next) = tail.next { tail = next; } out.tail = Some(NonNull::from(tail)); } ``` **`List::and_range` and `Group::or_range`** took the pointer from the new `Box` and then moved the `Box` into the chain. Under Tree Borrows the move makes that pointer stale, and the next append writes through it. The header of `test/internal/source-lints/frozen-nonnull-reborrow.test.ts` states the rule: derive the pointer after any move of the owning `Box`. On `main`, `cargo miri test -p bun_semver` fails for this reason as soon as a test parses a range with two `||`. The pointer is now taken from the box's final place: ```rust self.tail = Some(NonNull::from(&mut **last_tail.next.insert(tail))); ``` With that the crate's tests pass under Miri, so `bun_semver` joins the crates that `bun run rust:miri` covers in CI. A new unit test checks that every cached pointer of a clone is the last node of the clone's own chain. No test covered that. #### 2. bake: the dev server's route trampoline forms its two references once `dev_route_tramp` is the `extern "C"` function that uWS calls for each of the dev server's seven HTTP routes. It kept the `DevServer` and the `Request` as raw pointers and wrote `unsafe { &*dev }` or `unsafe { &mut *req }` at each of 17 uses, under an `#[allow(clippy::undocumented_unsafe_blocks)]`. It now makes `&mut DevServer` and `&mut Request` once at the top, in one `unsafe` block with one SAFETY comment. The two guards (`is_allowed_dev_host`, `is_allowed_dev_origin`) take `&` and borrow from them, and then exactly one handler gets both. The `#[allow]` is gone. Each handler already held both `&mut` for its whole run, so the only new overlap is with the two guards, which only read request headers and the server's configured address. What this does not change: a handler still holds `&mut DevServer` while it runs JavaScript, and the file watcher's thread forms its own. That comes from the handlers' signatures. The comment above the `route!` macro in `set_routes` described an older trampoline, keyed on a handler type, and named an `AppRouteExt` that no longer exists. It now says what the code does. #### 3. crypto: no `ENGINE` is allocated or passed, because BoringSSL never reads it `EVP_DigestInit_ex` and `EVP_Digest` have an `ENGINE*` parameter for OpenSSL compatibility. BoringSSL ignores it. In `vendor/boringssl/crypto/fipsmodule/digest/digest.cc.inc` the body of `EVP_DigestInit_ex` never mentions it, `EVP_Digest` only hands it to `EVP_DigestInit_ex`, and BoringSSL's own `EVP_DigestInit` passes `nullptr`. Bun still allocated one `ENGINE` per VM on first use, kept it in `RareData`, freed it on teardown, and passed it through six of `EVP`'s functions. - `EVP::init`, `reset`, `hash`, `final`, `copy` and `by_name_and_engine` (now `by_name`) lose the parameter, and the FFI calls pass a null pointer. - `RareData::boring_engine()`, its field and its `ENGINE_free` are gone, and so are `boring_engine` and `engine_of` in `CryptoHasher.rs`. `engine_of` needed an `unsafe` block to get at the global object from a `JsHeap`. - The three exports that `node:crypto`'s `Hash` calls from C++ (`Bun__CryptoHasherExtern__getByName`, `getFromOther`, `digest`) took the global object only to find the engine. They lose that parameter on both sides. - `ENGINE_new` and `ENGINE_free` have no callers left, so their declarations are removed. - Two calls of `EVP::init`, and `HMAC::init`, cast `Algorithm::md()` to `*const EVP_MD`, under a comment saying that it returns another crate's opaque type. It returns that very type, so the casts and the comments are removed. Creating, copying and finishing a hash no longer looks the engine up in `RareData`, and a VM that hashes allocates one object fewer. #### 4. Readers of the `FileSystem` singleton use `get()`, not `instance()` `bun_resolver::fs::FileSystem::instance()` is a safe function that returns a `&'static mut` to a process-wide singleton. Every call makes another `&mut` to the same object, on whichever thread it runs. Of its 128 call sites, 120 only read: `top_level_dir()`, the dirname and filename stores, path joins (`abs`, `abs_buf`, `relative`, `relative_to`), `need_to_close_files()` and taking `entries_mutex`. All of those take `&self`. Two more, in `PackageManager::init` and `init_with_runtime_once`, kept the result in a local, read `top_level_dir()` from it and then called `read_directory` on it. The reads now go through `FileSystem::get()`, which returns `&'static FileSystem`: 122 conversions in 55 files. Its documentation already says to prefer it for read-only access. In a release build both functions are the address of the same static. In a build with debug assertions `get()` also asserts that `init()` has run. I traced the earliest converted site of each command and found none that runs before `init()`. Eight callers stay on `instance()`, because they go through `RealFS::entries`, whose methods all take `&mut self`: - three `read_directory` calls in `PackageManager.rs` - `DirEntryAccessor::openat` - the hot reloader - three accessors in `dir_info.rs` `instance()` now has a doc comment that says so. In the two functions that did both, the working directory is read first, through `get()`, so the function's own shared reference is no longer in use when `read_directory` forms the `&mut`. This commit removes no `unsafe` block. It is here because it ends the `&mut`/`&mut` aliasing among readers, and it cuts what is left of `instance()` from 128 call sites to eight in four files. The shared borrows still coexist with the `&mut` of those eight, so nothing becomes provably sound yet. The follow-up is to give the directory cache a `&self` interface under `entries_mutex` and delete `instance()`. Eight `// SAFETY:` comments that stood above safe `get()` calls are removed. `bun_paths::fs::FileSystem` is a different, lower-level type with its own `instance()`, and is not touched. #### 5. bundler: the arena and the chunk contexts are borrowed for as long as they live **The linker's arena.** `LinkerGraph<'a>` kept the bundle's arena as a `BackRef<Arena>` that started out dangling and was filled in after `BundleV2` had been built. So `LinkerGraph::arena()` and `LinkerContext::arena()` could only return a borrow of `&self`, and code that allocated from the arena and then called a `&mut self` method had to erase that lifetime first. `BundleV2::init_without_starting_threads`, the only place that builds a linker, has the `&'a Arena` at hand. `LinkerGraph` now holds it in a private field, `arena()` returns `&'a Arena`, and `LinkerGraph::new(arena)` and `LinkerContext::new(arena)` replace the two `Default` impls. The three `detach_lifetime_ref(this.arena())` in `generate_code_for_lazy_export` become `this.arena()`, and `take_ast_ownership` no longer needs the arena passed in. **The bundle's arena.** `BundleV2::arena()` already returned `&'a Arena`, so seven workarounds around it did nothing: | Where | Was | Now | |---|---|---| | `arena_create`, a helper with ten callers | `detach_lifetime_mut(self.arena().alloc(value))`, returning an unbounded `&'r mut T` | removed; callers call `alloc` on the arena | | the client transpiler's setup | `detach_lifetime_ref(self.arena())` to `&'a` | `self.arena()` | | the server component manifest | `detach_lifetime_ref(self.arena())` to `&'static` | `self.arena()` | | the dev server's list of reachable files | `&*ptr::from_ref(self.arena().alloc_slice_copy(..))` | `self.arena().alloc_slice_copy(..)` | | the dev server's chunks | `&mut *ptr::from_mut(self.arena().alloc_slice_fill_iter(..))` | `self.arena().alloc_slice_fill_iter(..)` | | the runtime's parse task | kept as `*mut ParseTask`, fields written with `(*task).x = ..` | a `&mut ParseTask` | | `enqueue_parse_task2` | the same, and `&mut *task` to pass it on | a `&mut ParseTask` | In the last two, one `unsafe` block stays: the back pointer from the task to the bundle, whose lifetime is erased as in every other `ParseTask`. **The chunk contexts.** `PendingPartRange<'a>` held `&'a GenerateChunkCtx<'a>`. `'a` is the bundle's lifetime and `GenerateChunkCtx` is invariant in it, so no borrow of the local `Vec` of contexts could have that type, and all three places that build a `PendingPartRange` erased the lifetime. The borrow gets its own parameter, `PendingPartRange<'c, 'a>`, and the compiler now checks that the contexts outlive every `PendingPartRange`. The loop that builds them only reads, so it iterates with `iter()` instead of `iter_mut()`. The two `ParentRef::from_raw_mut(ptr::from_mut(c))` in the same function, and the two in `find_all_imported_parts_in_js_order`, are the safe `ParentRef::from_ref_mut(c)`. **`unsafe impl Send` and `Sync` for `LinkerGraph`.** Nothing needs them. The graph is only shared as a field of `LinkerContext`, which has its own impls. They are removed, and so is the entry in `mordant-baseline.toml` that allowed one of them. (Change 15 puts the `Sync` impl and the baseline entry back, and says why.) Their field-by-field notes were out of date: - They described a thread pool in `compute_cross_chunk_dependencies`, which is a sequential loop now, so `Map::assign_chunk_index` is not called by workers either. - They said that the arena is frozen while workers run. The bundle thread allocates from it while source map tasks run. What holds is that workers never allocate from the arena. That sentence is now in the SAFETY comment of `LinkerContext`'s impls, next to the one about disjoint rows. An unused `let _arena` in `compute_chunks`, whose comment described the `BackRef`, is removed. A `&'a Arena` is one pointer, like the `BackRef` it replaces, and the linker graph is now built once instead of being built twice with one copy dropped. #### 7. bun test: the file scanner borrows its filters `Scanner<'a>` has a lifetime for its two filter lists, but `TestCommand::exec` erased both lists, and every element, to `'static`, on the grounds that the frame never returns. That was not true of the filter names: they lived in the `else` block that set them, and that block ends while the scanner is still in use. So `scanner.filter_names` was left dangling. Nothing read it afterwards. - The scanner's `filter_names` is a `&'a [Box<[u8]>]` and borrows `ctx.positionals` directly, so the `Vec` that only changed the element type is gone. On Windows, where the filters are rewritten to backslashes first, it borrows that rewritten list. - The ignore patterns are declared before the scanner and borrowed normally. - A third list was erased the same way: `concurrentTestGlob` from bunfig. It cannot be borrowed, because the `TestRunner` that holds it is published through a static. The runner owns it instead. Nothing else reads `ctx.test_options.concurrent_test_glob`, so the list is moved out of it, not copied: the only allocation is still the one `Vec` of prepared patterns. #### 8. install: `LoadResult` no longer borrows the lockfile it was loaded into `Lockfile::load_from_cwd(&mut self, ..)` returned `LoadResult<'a>`, whose `Ok` variant held `lockfile: &'a mut Lockfile`: a second `&mut` to the very object the caller had called it on. For most callers that is `manager.lockfile`, so as long as the result was alive the whole `PackageManager` was borrowed. Callers got around that with a raw `*mut PackageManager` that they dereferenced at each use (16 `unsafe` blocks in `pm_trusted_command.rs` alone). `install_with_manager` also kept the result while `clean_lockfile` replaced the `Box` in `manager.lockfile`. After that the `&mut` pointed at the pre-clean lockfile, which `install_with_manager` also owns and reads as `lockfile_before_clean`. `LoadResultOk` now holds only what the load found out (`migrated`, `serializer_result`, `format`), and `LoadResult` has no lifetime. What was loaded is in the `Lockfile` that the load was called on. - `choose_config_version` and `node_linker` take `&Lockfile`. The former is merged with its `_with_saved` variant. - The four migrators (npm, pnpm, yarn, yarn berry) no longer pass the lockfile back through the result, and `finish_migration` takes it as a parameter. - `PackageManager::load_lockfile_from_cwd` is now used by `install_with_manager` and `bun pm scan` too, which had their own copies of it. For both that also resets the cached root package id, which is derived from the lockfile that is being replaced. - The nine places that read through the field read the lockfile they already have: `manager.lockfile` in most commands, a local one in `bun patch --commit`. - `LoadResult::ok()` and `ok_mut()` are gone. The last four reads are `matches!(&load_result, LoadResult::Ok(ok) if ..)`. - The `SAFETY` comments on the two split borrows that remain (`load_lockfile_from_cwd`, `bun pm migrate`) said that the callee never reaches `manager.lockfile`. During a migration it does: `Package::parse_with_json` (yarn v1) reads it, and `populate_manifest_cache` (npm, pnpm, yarn) reads it and stores to `scratch.dependency_list_queue`, all while the `&mut Lockfile` is live. **That aliasing is on `main` too and this PR does not fix it.** The comment now says what holds (two separate allocations) and marks the rest with a `FIXME`. The code there is as it was. #### 9. Return a borrow from one branch and keep going in the other The pinned toolchain's borrow checker (Polonius, which `rustc -Z help` lists as on by default) accepts a function that returns a borrow on one path and uses the borrowed value again on the other. The older checker did not, and the code worked around it: | Where | Workaround | Now | |---|---|---| | `which_win`, `search_bin_in_path_list` (Windows) | `&mut *ptr::from_mut(buf)` before each call | `buf` | | `Snapshots::get_or_put` | `contains_key`, then `get(..).unwrap()` | one `get` | | `Loader::get_node_path` | the length computed in a closure, the `ZStr` built twice | built once | **This is new:** `main` still compiles with `-Zpolonius=off`. After this commit these four functions do not, and after commit 10 a fifth, `Editor::by_fallback`. The default is nightly-only (stable 1.98 rejects these shapes), so upstream may still switch it off again. A toolchain bump would then need `-Zpolonius=next` in the rustflags, which the pinned `rustc` accepts. Three more workarounds go away without needing the new checker: - `PathPairIter` (resolver) passed its `&mut Path` through a raw pointer. The type is replaced by `once(&mut primary).chain(&mut secondary).filter(..)`, and the method is called `iter_mut` now, because that is what it is. - The first of the three Windows sites, in `which_win`'s `has_dir` branch, where both paths return. - `ArrayHashMap::get_or_put_value`, which recomputed its result from the index. The absolute-path branch of `which` built its `ZStr` with the unsafe `from_raw_mut`. It uses `ZStr::from_buf`, like the other two branches. That function checks its precondition with debug assertions only, so here the gain is the lifetime, which is now the buffer's. `bun_which` has no `unsafe` left on any platform and is now `#[forbid(unsafe_code)]`, so the six `// SAFETY:` labels on safe calls in it are dropped or turned into plain comments. Left alone on purpose: `is_valid` in `which` and `parse_value` in the env loader also compute a length first and build the slice later. Neither has any `unsafe`, and reshaping them would only add functions that need Polonius. #### 10. Bun.openInEditor: return `Option`, own the argv, own the editor's name - **The lookups.** `by_path_for_editor` returned `bool` and wrote the path through `out: &mut &'a [u8]`. That ties `buf` to `out` even after a miss, so each of the four calls re-created `&mut buf` from a raw pointer. It returns `Option<&'a [u8]>`, `by_fallback_path_for_editor` returns `Option<&'static [u8]>`, and `by_fallback` returns `Option<(Editor, &'a [u8])>` (its loop is the fifth function of item 9). "In `PATH`, else the usual install locations" was spelled out three times and is one helper now. - **The argv.** `Editor::open` filled a boxed `SpawnedEditorContext` with `(pointer, length)` pairs, some of them into the box itself, passed the box to the new thread as a `usize`, and the thread turned each pair back into a slice and copied it into a `Box<[u8]>`. `open` now builds that `Vec<Box<[u8]>>` itself and moves it into the thread. - The number of allocations and the bytes copied are the same. They are made on the JS thread now, and the argv is freed on the editor's thread. - The scratch buffer is still one heap allocation of the same size, so a path that is too long still fails with `WriteFailed`. It is freed when `open` returns instead of when the editor exits. - **The name and the path.** `EditorContext::name` was a `&'static [u8]` into a `Vec` that `BunObject.rs` kept beside it in a wrapper struct. It is the `Vec` now, and the wrapper is gone. `path` was a `&'static [u8]` too, copied into the dirname store, which is never freed, because the editor thread could still be reading it when a later call replaced the name. The thread has its own copy of the argv now, so `path` is a `Vec` as well, and a change of editor no longer leaves a dead copy in the store. - An absolute editor name is handled before the path buffer and the working directory are fetched, which that branch never used. #### 11. install: the cache directory lookup takes `&mut PackageManager` `get_cache_directory_raw` and `ensure_cache_directory` were `unsafe fn`s on a `*mut PackageManager`, "for callers that hold a disjoint `&mut this.manifests` borrow". There are no such callers left. The only one is `try_get_cache_directory`, which has a `&mut PackageManager`. Both are safe now, and the first is folded into its caller. #### 12. Use the borrow at hand in places that went through a raw pointer - The lockfile's metadata backfill after a migration, and the manifest-cache lookup when a dependency is enqueued, read `manager.options` and write `manager.manifests` through raw pointers. They are two fields, so the code borrows the two fields. A `NonNull::new(..).expect(..)` goes away with it. A third lookup, in `get_or_put_resolved_package`, reads `options` plainly now but keeps its raw `manifests`: the manifest is held across calls that take the whole manager, and cloning it would add an allocation. - `ConsoleObject::init_in_place` assigned two fields of the `&mut ConsoleObject` it had just made through a raw copy of it. - Three task builders in `PackageManagerEnqueue.rs` (git checkout, manifest parse, streaming extract) and the recursive `readdir` subtask build their `ParentRef` with the safe `from_ref_mut`, as the other builders in `PackageManagerEnqueue.rs` already do. In two of them the `unsafe` block then wrapped a whole struct literal for one raw dereference, and now holds only that. Its `SAFETY` comment called the network task "freshly vended", which is true for one of three callers. It now says what all three rely on: the pool slot is not returned before the task is done. This is tidying, not a borrow the compiler checks: `ParentRef` has no lifetime, and what it owes (the parent outlives the task) is unchanged. #### 13. Declare 25 foreign functions without preconditions as `safe fn` Each of these takes only integers, or nothing, and can be called with any value from any thread. Most of the comments at their call sites said so ("no preconditions", "pure getter", "pure size query"). The repository already declares over 1,100 foreign functions `safe fn`; these were left over. | Kind | Functions | |---|---| | pure queries | `us_nq_vtable_size`, `us_nq_tp_size`, `us_nq_settings_size`, `us_nq_spec_stride`, `simdutf__base64_length_from_binary`, `bun_image_resize_scratch_size`, `uv_get_total_memory`, `uv_translate_sys_error` | | Win32 clipboard queries | `IsClipboardFormatAvailable`, `GetClipboardSequenceNumber` | | lookups that return a static string or null | `SSL_get_group_name`, `X509_verify_cert_error_string`, `Bun__X509__validationErrorCode`, `gnu_get_libc_version` | | getters of a process-wide pointer | `mi_heap_main`, `Bun__getStandaloneModuleGraphMachoLength`, `Bun__getStandaloneModuleGraphELFVaddr` | | constructors that return a new object (or null) | `mi_heap_new`, `spng_ctx_new`, `tj3Init`, `WebPNewInternal`, `JSC__IdentifierArray__create`, `CreateMemoryResourceNotification` | | other | `uv__winsock_ensure` (`uv_once`), `Bun__abortForRemovedFeature` (bounds-checks its index, never returns) | Commits 18 and 19 add two more, `WaitForSingleObject` and `SetEvent` in `memory_pressure.rs`. They take a handle, which the kernel validates: a bad one gives `WAIT_FAILED` or `FALSE`. Two other declarations of the first are already `safe fn`. The C or C++ body of each was read, or its documented contract for the OS functions. Four that look the same stay `unsafe`, and a line on each declaration now says why: - The two Windows counterparts of the module graph getters fill two plain statics on first use, which is not safe to do from two threads. - `bun_coregraphics_clipboard_change_count` messages `NSPasteboard`, which that file treats as JS-thread only. - `uv_os_getppid` calls through a pointer that only `uv__winapi_init` fills. The `SAFETY` comments at their call sites said "FFI calls", "pure getter, no preconditions" and "trivial libuv accessor, no preconditions". They now say what the caller relies on. Also removed: three stale `// SAFETY: pure FFI getter` lines in `node_os.rs` above calls that were already safe, and a second, unused declaration of `uv_disable_stdio_inheritance`. What a caller then does with a returned pointer is still `unsafe`. Where a removed comment also justified such a later block ("as above"), the justification moved there. #### 14. Foreign functions that only need a live handle take a reference and are `safe fn` 16 functions (the last one in commit 18) were `unsafe` to call, most because they were declared with `*const` or `*mut JSGlobalObject` (or a pointer to a source provider). They now take the reference, as most of the bindings do, which has the same ABI: `Expect__getPrototype`, `ExpectStatic__getPrototype`, `ExpectMatcherUtils__getSingleton`, `AsymmetricMatcherConstructorType__fromJS`, `Process__emitMemoryPressureEvent`, `HandleScopeImpl__open`, `Bun__createNodeModuleSourceMapOriginObject`, `Bun__createNodeModuleSourceMapEntryObject`, `Bun__FFI__closeFunctions`, `Bun__FFI__CStringConstructor`, `BakeSourceProvider__getSourceSlice`, `BakeSourceProvider__getBunVM`, `DevServerSourceProvider__getSourceSlice`, `DevServerSourceProvider__getSourceMapJSON`, `BakeGlobalObject__isBakeGlobalObject`, `Bun__JSCFFICallbackClose`. What they need beyond a live handle is what every safe `fn(&JSGlobalObject)` in the tree already assumes: the VM's own global, on its JS thread under the API lock, and non-empty `JSValue`s. Those that take a `JSValue` check its type on the C++ side (`dynamicDowncast`, `isObject()`) or only store it. Two neighbours do not and stay `unsafe`: `Bun__JSWrappingFunction__getWrappedFunction` calls `asCell()` unchecked, and `Bun__JSWrappingFunction__create` expects an object. A line on each declaration says so. They, `HandleScopeImpl__close` and `BakeGlobalObject__getPerThreadData` take the reference too, but stay `unsafe` for their other argument or for an unchecked downcast. Five more hand-written declarations duplicated safe wrappers that `bun_jsc::cpp` generates from `[[ZIG_EXPORT]]`: `Bun__ExposeNodeModuleGlobals` (twice), `JSC__JSGlobalObject__addGc`, `JSMockFunction__getCalls` and `JSMockFunction__getReturns`. They are deleted, with their two forwarding functions, and the generated wrappers are called. Three call sites passed the raw field `vm.global`. `Run::start` and the REPL's set-up now call `vm.global()`, which tests for null, once each and use the result throughout. `Run::start` already made that call for the API lock, and seven more times further down, next to one `unsafe { &*vm.global }`. The REPL's set-up made it only when `TZ` is set, so that is the one test this PR adds, once per REPL start. #### 15. bundler: `LinkerGraph` keeps its own `Sync` impl A correction to change 5, which removed both `unsafe impl Send` and `unsafe impl Sync` from `LinkerGraph` because only `LinkerContext` shares it and that has impls of its own. For `Sync` that made things worse: `LinkerContext`'s impl then also had to vouch for what is inside the graph, not only for its own raw pointers, and the `mordant` job reported exactly that (`unsafe_send_covers_more_than_pointers`). The `Sync` impl is back on the type that needs it. `Send` stays removed, because nothing needs it. `mordant-baseline.toml` is as on `main` again, and the job passes. Its `SAFETY` comment is new, because the 29 lines on `main` were out of date (change 5 lists how). It names what is not `Sync` on its own and what the code relies on for each: - The `MultiArrayList`s: workers write their columns through `split_raw()` and `items_raw()`, and each task writes only the rows of its own file or chunk. Nothing writes `Part.scopes` or the `symbols` column of `ast` while workers run. - The `BitSet`s: workers only read them. - `Symbol.link`, a `Cell`: `follow()` writes it only to shorten a chain, and `LinkerContext::link()` shortens them all with `follow_all()` before workers follow any. The dev server skips `link()`. Its links come from the parser and stay within one file, and tasks that run at the same time work on different files. The arena is `Sync` by itself. The comment keeps the rule that goes with it: only the bundle thread allocates from it. #### 16. http: `build_request` returns `Request<'a>`, the lifetime its parts already have `HTTPClient<'a>::build_request` returned `picohttp::Request<'static>`, and erased the path and the headers to get there, "so callers don't pin `&mut self` for the rest of their fn". `'a` gives callers the same thing: `header_buf`, `url` and `if_modified_since` are all `&'a`, and `'a` is independent of the `&mut self` borrow. - `build_request` returns `Request<'a>`. Its path is `self.url.pathname`, and its headers are a plain reborrow of the HTTP thread's scratch buffer, which is `&'static mut` already. - The `if_modified_since` assignment was erased "only because we don't yet thread struct lifetime params". The struct has the parameter, and the field is `&'a`. - `HTTPContext::connect` takes `hostname: &'a [u8]`, which is what its callers pass, and assigns it. - `picohttp::Header::new` and `RawSlice::new` take any lifetime, so the `unsafe` reborrows in front of four of their arguments did nothing. - `h2_build_request`, which only erased the lifetime once more, and `picohttp::Request::detach_lifetime`, which only it called, are deleted. The request is never stored: at its four call sites it is a local that never leaves the function. What no lifetime expresses is unchanged: the headers are valid until the next `build_request` on the HTTP thread (a doc line now says so), the `Content-Length` value points into the client itself (a doc line on the field says so), after a redirect the path and `Host` point into `self.redirect`, and `state.request_body` still points into `compressed_request_body` or the thread's `shared_buffer` (the function's doc and a one-line comment say how long each is valid). #### 17. http: `HTTPClientResult::detach_lifetime` was safe code It was an `unsafe fn` with the contract "caller must not read `.body` from the returned value". Its body has no unsafe operation: it sets `body: &[]` and moves the other fields, which are all owned, so reading `.body` afterwards is harmless. It is a safe fn now, named for what it does (`without_borrowed_body`), because every other `detach_lifetime` in the tree is a real eraser. Its body is `HTTPClientResult { body: &[], ..self }`, which only compiles as long as `body` is the one borrowed field. One of the removed `SAFETY` comments (S3) described a borrow that does not exist. #### 18. Follow-ups from a second review of 7 to 17 Each item is described above, with the change it belongs to. In short: - **Comments that were wrong:** the two `SAFETY` comments on the lockfile split borrows (8), the one on `unsafe impl Sync for LinkerGraph` (15), one about `compressed_request_body` (16), and three at the call sites of functions that stay `unsafe` (13). - **The same pattern, a few lines further on:** `scope` in `get_or_put_resolved_package` (12), `WaitForSingleObject` (13), `Bun__JSCFFICallbackClose` (14). - **Cost:** the null tests that 14 added are down to one per REPL start, and 10 no longer reads the working directory for an absolute editor name. - **Simpler:** `EditorContext::path` is a `Vec` (10), `..self` in `without_borrowed_body` (17), two `unsafe` blocks hold one dereference instead of a struct literal (12), `PathPair::iter_mut` (9). #### 19. Follow-ups from a review of 18 - **Comments next to the ones that 18 rewrote:** a stale note above the manifest lookup and a `SAFETY` line that pointed at it (12), "freshly vended" on the two network task dereferences (12), the call site of `Bun__JSWrappingFunction__create` (14), a pointer from `LinkerGraph`'s comment to a stale note (15), and a line on the clipboard getter that contradicted the header of its section (13). - **The same pattern, once more:** `SetEvent` is a `safe fn` (13), and `Run::start` uses the `global` it has bound instead of seven more `vm.global()` calls and one `unsafe { &*vm.global }` (14). The two `Bun__JSWrappingFunction__*` declarations take `&JSGlobalObject` and stay `unsafe` (14). - A test for an empty editor name that `is_absolute` already makes (10). ### How did you verify your code works? Change 1 fixes a bug that only Miri can see. With `MIRIFLAGS=-Zmiri-tree-borrows`, as `bun run rust:miri` sets it, `cargo miri test -p bun_semver` fails on `main`: ``` test intersects::tests::is_subset_of ... error: Undefined Behavior: reborrow through <195137> at alloc69209[0x8] is forbidden --> src/semver/SemverQuery.rs:1029:8 help: the accessed tag <195137> was created here, in the initial state Reserved --> src/semver/SemverQuery.rs:535:9 535 | prev_tail.next = Some(new_tail); help: the accessed tag <195137> later transitioned to Disabled due to a foreign write access at offsets [0x8..0x10] ``` With this PR all 7 tests of the crate pass under Miri, the new one included. No other change is visible from outside, so there are no other new tests. The runs below are aimed at the changes listed under "Where the compiled code changes". - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux, macOS and Windows. - I ran the test files below on a debug + ASAN build of this branch and on a debug + ASAN build of `main` as it was when the branch started, and compared the names of the failing tests. The second table is from commit 18, the last one before `main` was merged in. Commit 19 changes comments, two declarations' parameter types, one more `safe fn`, and which expression names the global object in `Run::start`, so I only ran the static checks above on it. <details> <summary>Changes 1 to 6: 34 test files, 3,430 pass, 47 fail on both builds</summary> | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/cli/install/semver.test.ts` | 357 | 0 | yes | | `test/cli/install/overrides.test.ts` | 124 | 0 | yes | | `test/cli/install/catalogs.test.ts` | 155 | 0 | yes | | `test/bake/dev/bundle.test.ts` | 26 | 28 | yes | | `test/bake/dev/esm.test.ts` | 11 | 7 | yes | | `test/bake/dev/html.test.ts` | 6 | 9 | yes | | `test/bake/dev/sourcemap.test.ts` | 1 | 1 | yes | | `test/js/bun/util/bun-cryptohasher.test.ts` | 109 | 0 | yes | | `test/js/node/crypto/crypto.test.ts` | 403 | 0 | yes | | `test/js/node/crypto/node-crypto.test.js` | 217 | 0 | yes | | `test/js/node/crypto/crypto-lazyhash.test.ts` | 4 | 0 | yes | | `test/js/node/crypto/crypto-oneshot.test.ts` | 35 | 0 | yes | | `test/js/node/crypto/hash-and-buffer-short-roads.test.ts` | 22 | 0 | yes | | `test/cli/install/bun-add.test.ts` | 220 | 0 | yes | | `test/cli/install/bun-remove.test.ts` | 37 | 0 | yes | | `test/cli/install/bun-link.test.ts` | 57 | 0 | yes | | `test/cli/install/bun-patch.test.ts` | 232 | 0 | yes | | `test/cli/install/bun-workspaces.test.ts` | 340 | 0 | yes | | `test/cli/install/bun-update.test.ts` | 175 | 0 | yes | | `test/cli/install/migration/pnpm-migration.test.ts` | 14 | 0 | yes | | `test/cli/install/migration/yarn-berry.test.ts` | 116 | 0 | yes | | `test/js/node/watch/fs.watch.test.ts` | 49 | 0 | yes | | `test/js/node/watch/fs.watchFile.test.ts` | 10 | 0 | yes | | `test/js/bun/util/filesystem_router.test.ts` | 51 | 0 | yes | | `test/cli/test/bun-test.test.ts` | 107 | 2 | yes | | `test/bundler/bun-build-api.test.ts` | 91 | 0 | yes | | `test/bundler/esbuild/default.test.ts` | 155 | 0 | yes | | `test/bundler/esbuild/css.test.ts` | 68 | 0 | yes | | `test/bundler/esbuild/splitting.test.ts` | 26 | 0 | yes | | `test/bundler/esbuild/loader.test.ts` | 10 | 0 | yes | | `test/bundler/bundler_loader.test.ts` | 62 | 0 | yes | | `test/bundler/css/css-modules.test.ts` | 46 | 0 | yes | | `test/bundler/bundler_plugin.test.ts` | 68 | 0 | yes | | `test/bundler/bundler_html.test.ts` | 26 | 0 | yes | | **Total** | **3430** | **47** | | The 47 failures are the same 47 tests on both builds: - 45 in `test/bake/dev` drive a browser page through the `happy-dom` package, which is not installed on the machine I used. - 2 in `bun-test.test.ts` expect a `RangeError` for values nested too deeply to print. `Bun.build can be called thousands of times in one process without crashing`, in `bun-build-api.test.ts`, needs about as long as its own limit of 180 s on these builds. In other runs, with the machine busy, it timed out once on each build. </details> <details> <summary>Changes 7 to 18: 86 test files, 9,415 pass, 29 fail on both builds</summary> | Test file | Pass | Fail | Same on `main` | |---|---|---|---| | `test/cli/test/bun-test.test.ts` | 107 | 2 | yes | | `test/cli/test/path-ignore-patterns.test.ts` | 10 | 0 | yes | | `test/cli/test/pass-with-no-tests.test.ts` | 5 | 0 | yes | | `test/cli/test/test-shard.test.ts` | 29 | 0 | yes | | `test/cli/test/test-changed.test.ts` | 21 | 0 | yes | | `test/cli/test/concurrent-test-glob.test.ts` | 6 | 0 | yes | | `test/cli/install/bun-pm.test.ts` | 154 | 0 | yes | | `test/cli/install/bun-pm-why.test.ts` | 116 | 0 | yes | | `test/cli/install/bun-pm-licenses.test.ts` | 85 | 0 | yes | | `test/cli/install/bun-pm-scan.test.ts` | 19 | 0 | yes | | `test/cli/install/bun-pm-diff.test.ts` | 75 | 0 | yes | | `test/cli/install/bun-prune.test.ts` | 177 | 0 | yes | | `test/cli/install/bun-patch.test.ts` | 232 | 0 | yes | | `test/cli/install/bun-install-patch.test.ts` | 53 | 0 | yes | | `test/cli/install/bun-lock.test.ts` | 317 | 0 | yes | | `test/cli/install/bun-lockb.test.ts` | 66 | 0 | yes | | `test/cli/install/lockfile-only.test.ts` | 34 | 0 | yes | | `test/cli/install/config-version.test.ts` | 8 | 0 | yes | | `test/cli/install/migrate-bun-lockb-v2.test.ts` | 2 | 0 | yes | | `test/cli/install/frozen-lockfile-outdated.test.ts` | 107 | 0 | yes | | `test/cli/install/bun-install-lifecycle-scripts.test.ts` | 434 | 0 | yes | | `test/cli/install/bun-dedupe.test.ts` | 82 | 0 | yes | | `test/cli/install/bun-audit.test.ts` | 260 | 0 | yes | | `test/cli/install/bun-update.test.ts` | 175 | 0 | yes | | `test/cli/install/bun-add.test.ts` | 220 | 0 | yes | | `test/cli/install/bun-remove.test.ts` | 37 | 0 | yes | | `test/cli/install/bun-workspaces.test.ts` | 340 | 0 | yes | | `test/cli/install/isolated-install.test.ts` | 283 | 0 | yes | | `test/cli/install/migration/migrate.test.ts` | 108 | 0 | yes | | `test/cli/install/migration/pnpm-migration.test.ts` | 14 | 0 | yes | | `test/cli/install/migration/pnpm-lock-v9.test.ts` | 168 | 0 | yes | | `test/cli/install/migration/yarn-berry.test.ts` | 116 | 0 | yes | | `test/cli/install/migration/yarn-lock-migration.test.ts` | 69 | 0 | yes | | `test/cli/install/migration/npm-arborist.test.ts` | 57 | 0 | yes | | `test/cli/install/migration/complex-workspace.test.ts` | 38 | 0 | yes | | `test/js/bun/test/snapshot-tests/bun-snapshots.test.ts` | 6 | 0 | yes | | `test/js/bun/test/snapshot-tests/existing-snapshots.test.ts` | 1 | 0 | yes | | `test/js/bun/test/snapshot-tests/new-snapshot.test.ts` | 2 | 0 | yes | | `test/bundler/esbuild/packagejson.test.ts` | 94 | 0 | yes | | `test/js/bun/resolve/resolve.test.ts` | 107 | 1 | yes | | `test/js/bun/util/which.test.ts` | 8 | 0 | yes | | `test/cli/install/bun-pm-cache-pack.test.ts` | 206 | 0 | yes | | `test/cli/install/bun-install-git-deps.test.ts` | 182 | 19 | yes | | `test/cli/install/bun-install-offline.test.ts` | 12 | 0 | yes | | `test/cli/install/bunx.test.ts` | 237 | 1 | yes | | `test/js/node/fs/fs.test.ts` | 631 | 0 | yes | | `test/js/bun/test/expect.test.js` | 427 | 0 | yes | | `test/js/bun/test/expect-extend.test.js` | 29 | 0 | yes | | `test/js/bun/test/expect-extend-asymmetric-match-throw.test.ts` | 1 | 0 | yes | | `test/js/bun/test/expect-extend-matcher-utils-throw.test.ts` | 1 | 0 | yes | | `test/js/bun/test/expect-toHaveReturnedWith.test.js` | 13 | 0 | yes | | `test/js/bun/test/mock-fn.test.js` | 100 | 0 | yes | | `test/js/node/module/module-sourcemap.test.js` | 4 | 0 | yes | | `test/js/node/module/sourcemap.test.js` | 14 | 0 | yes | | `test/js/bun/ffi/ffi.test.js` | 151 | 0 | yes | | `test/js/bun/ffi/cc.test.ts` | 25 | 0 | yes | | `test/js/bun/ffi/closed-jscallback.test.ts` | 1 | 0 | yes | | `test/js/bun/image/image.test.ts` | 104 | 0 | yes | | `test/js/bun/image/image-adversarial.test.ts` | 103 | 0 | yes | | `test/js/bun/image/image-allocator.test.ts` | 0 | 0 | yes | | `test/js/bun/repl/repl.test.ts` | 129 | 0 | yes | | `test/cli/run/run-eval.test.ts` | 45 | 0 | yes | | `test/js/node/process/process-memory-pressure.test.ts` | 8 | 0 | yes | | `test/js/node/quic/quic-endpoint.test.ts` | 9 | 0 | yes | | `test/js/node/quic/quic-sni.test.ts` | 11 | 0 | yes | | `test/js/node/quic/quic-stream.test.ts` | 13 | 0 | yes | | `test/js/bun/util/open-in-editor-gc.test.ts` | 5 | 0 | yes | | `test/js/web/fetch/fetch.test.ts` | 397 | 1 | yes | | `test/js/web/fetch/fetch.stream.test.ts` | 121 | 0 | yes | | `test/js/web/fetch/fetch-http2-client.test.ts` | 88 | 0 | yes | | `test/js/web/fetch/fetch-http3-client.test.ts` | 82 | 0 | yes | | `test/js/web/fetch/fetch-compress.test.ts` | 32 | 0 | yes | | `test/js/web/fetch/fetch-redirect.test.ts` | 30 | 0 | yes | | `test/js/web/fetch/fetch-url-after-redirect.test.ts` | 2 | 0 | yes | | `test/js/web/fetch/fetch-keepalive.test.ts` | 58 | 0 | yes | | `test/js/web/fetch/content-length.test.js` | 1 | 0 | yes | | `test/js/web/fetch/fetch-connection-header.test.ts` | 16 | 0 | yes | | `test/js/bun/http/fetch-header-count-limit.test.ts` | 3 | 0 | yes | | `test/js/bun/http/proxy.test.ts` | 0 | 1 | yes | | `test/js/bun/http/proxy.test.js` | 16 | 0 | yes | | `test/js/bun/s3/s3-connection-close.test.ts` | 7 | 0 | yes | | `test/js/bun/s3/s3-insecure.test.ts` | 1 | 0 | yes | | `test/cli/install/bun-install-registry.test.ts` | 492 | 0 | yes | | `test/cli/install/bun-install.test.ts` | 768 | 4 | yes | | `test/cli/install/bun-publish.test.ts` | 279 | 0 | yes | | `test/cli/install/bun-run.test.ts` | 319 | 0 | yes | | **Total** | **9415** | **29** | | The 29 failures are the same 29 tests on both builds: - 23 (19 in `bun-install-git-deps.test.ts`, 4 in `bun-install.test.ts`) run `git tag` in a fixture repository, which the git configuration of the machine I used refuses ("no tag message?"). - 2 in `bun-test.test.ts` expect a `RangeError` for values nested too deeply to print. - 2 need packages that are not installed on that machine (`axios` in `proxy.test.ts`, `tsyringe` in `resolve.test.ts`), and 1 in `fetch.test.ts` needs `example.com`. - 1 in `bunx.test.ts` runs into its 300 s limit. </details> I also ran these on both builds (commit 18 again) and compared the output. All 9,683 lines are the same: - **Dev server routes (2):** a dev server with an HTML route, asked over raw sockets for 13 paths that reach six of the seven handlers, each with 12 combinations of `Host` and `Origin` headers, allowed and forbidden. The seventh handler, the framework's catch-all, is covered by `test/bake/dev`. - **Hashing (3):** - every name in `Bun.CryptoHasher.algorithms` and in `crypto.getHashes()`: incremental, one-shot, `copy()`, a copy of a copy, use after `digest()`, output into a buffer, HMAC - `shake128` and `shake256` with six output lengths - unknown and oddly spelled names - a `Hash` used as a stream - four workers, each a VM that is created and torn down - 3,000 hashers, a third of them never finished, then a forced GC - **`FileSystem` readers (4):** - `bun run` of scripts, files, `.`, local binaries, a system binary and missing names, from the project root and from a subdirectory - `bun run --filter`, `bun x`, `process.chdir()`, `bun build --external`, `bun create` from local templates, `bun getcompletes`, `Bun.FileSystemRouter` - `bun test` file discovery - `Bun.resolveSync` on 38 specifiers - `bun add --trust` and `bun pm trust` on 14 projects with `file:` dependencies - **Bundler (5):** - 25 `bun build` commands over two small projects, across formats, targets, minify flags, source maps, `--splitting`, HTML entry points and `--format=internal_bake_dev`. The output includes the bundles and the source maps. - JSON, TOML, YAML and text imports by default, by name and by namespace, which is what `generate_code_for_lazy_export` handles - CSS modules with `composes` within a file, across files and from `global` - 12 modules that import each other dynamically, as three and as four entry points - `Bun.build` with in-memory files, with `onResolve` and `onLoad` plugins, with `data:` URLs, with errors, and six builds at once - **`Bun.openInEditor` and the `node` lookup (9, 10):** - which editor `Bun.openInEditor` starts and with which arguments, with fake editors in six different `PATH`s: ten values of `$EDITOR`, `$VISUAL`, 18 values of the `editor` option (names, aliases, absolute paths, unknown names), line and column, and several calls in one process - `bun run` of a script and of a file with a `node` shebang, with `NODE` and `npm_node_execpath` unset, empty, and set to paths that exist or do not - **The cache directory (11):** `bun install`, `bun add`, `bun pm cache` and `bun pm cache rm` in 22 set-ups: `BUN_INSTALL_CACHE_DIR` (absolute, relative, empty), `BUN_INSTALL`, `XDG_CACHE_HOME`, bunfig, `--cache-dir`, `--no-cache`, a path that is a file, is below a file or is too long, no `HOME`, and `--linker=isolated` --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…malloc to v3.6.0 and ICU to 79.1 (oven-sh#47281) Three things that go together. All three dependencies' PRs are merged, and both pins are commits of their main branches. | | What | From | |---|---|---| | 1 | The address space ballast is removed, and `patches/mimalloc/` with it | oven-sh/WebKit#937, oven-sh/mimalloc#63 | | 2 | mimalloc goes from v3.5.3 to v3.6.0 | oven-sh/mimalloc#64 | | 3 | ICU goes from 78.3 to 79.1 (Unicode 18.0, CLDR 49) | oven-sh/WebKit#936 | **About 3:** oven-sh/WebKit#936 merged just before oven-sh/WebKit#937, so no commit of WebKit's `main` has one without the other. It changes some locale text (`fr` hours, `th` short time, `zh-Hant` short zone names) and makes `EST5EDT`, `CST6CDT`, `MST7MDT` and `PST8PDT` resolve to the zones the IANA database links them to. What people will see, all of it CLDR 49 (an ICU 79.1 that is not Bun's gives the same). Node has ICU 78.3, so Bun differs from Node in these until Node moves: | | ICU 78.3 | ICU 79.1 | |---|---|---| | `toLocaleDateString("ko")` | `1970. 1. 1.` | `1970/1/1` | | `en`, `{ month: "short", year: "2-digit" }` | `Jan 70` | `Jan ’70` | | `timeZoneName: "long"`, `Europe/London` and `Europe/Dublin` in winter, every locale tried | `Greenwich Mean Time` | `GMT+00:00` | | `timeZoneName: "long"`, `Pacific/Honolulu` | `Hawaii-Aleutian Standard Time` | `Hawaii Time` | oven-sh#47314 is the same ICU upgrade by itself. The test changes here that overlap with it are the same lines. The `Intl` snapshots run only on the ICU version they were made with, so they are regenerated (one value differs, the Korean date). They exist to catch a fault in Bun's compressed ICU data, so each value was also compared with a Node that links an ICU 79.1 whose data is not compressed: 385 of 389 are the same, and the other 4 differ in the case of the first letter of a display name, as they did with ICU 78.3. ## 1. The ballast and the patches ### What **The address space ballast is removed.** Under a limit on address space (`ulimit -v`), an executable compiled with `--aot` kept 8 MB of address space unused. When a `malloc` that may not fail found no memory, it released 1 MB and tried again, so that the thread could reach a point where it throws a `RangeError`. mimalloc is not built for that, and it took ten patches, each fixing what the one before exposed. It was still best effort: a request larger than what was left of the reserve crashed anyway. Such a `malloc` crashes on purpose again, as it does in plain Bun and in upstream JavaScriptCore. Allocations that can fail (strings, arrays, typed arrays, `ArrayBuffer`) still throw a `RangeError`. **`patches/mimalloc/` is gone.** We maintain a fork, so what is worth keeping is a commit there. | Patches | Count | Outcome | |---|---|---| | The ballast | 10 | Deleted | | `hand-out-last-freed`, `hand-out-resident-slices` | 2 | Deleted, see below | | `new-oom-hook`, `look-in-every-bin-of-an-exclusive-arena`, `secure-guard-pages-are-protected`, `out-of-memory-is-no-corruption` | 4 | Commits in the fork | | `environment-sets-sizes-and-pace-only` | 1 | A generic `MI_GETENV_ALLOWED` in the fork. The same 16 names now live in `scripts/build/deps/mimalloc.ts`. No change in behavior. | ### What stays The cage reserved in segments under a limit, the `RangeError` at the object limit, the reserve of the structure heap, and `new Worker()` throwing a `TypeError` when there is no room for another thread. ### The two `hand-out` patches They made the allocator hand out the memory that was freed last while an `--aot` build runs. Because `aot` is on by default, every thread-local `mi_free` in every Bun process paid for the test: 23 → 26 instructions on arm64, one of them a load of a global. `bun build --compile --aot` of `typescript/lib/_tsc.js`, macOS arm64, canary `8d5672c55`, six runs each in turn, switched off with `MIMALLOC_HAND_OUT_LAST_FREED=0`. Min / median / max: | | On | Off | |---|---|---| | Peak footprint (MB) | 553 / 585 / 676 | 523 / 530 / 542 | | Instructions (G) | 382.2 / 382.8 / 384.3 | 381.7 / 382.6 / 383.1 | | Cycles (G) | 175.8 / 176.8 / 182.2 | 177.7 / 179.2 / 179.9 | | System CPU (ms) | 1171 / 1418 / 1760 | 1213 / 1457 / 1484 | **Not measured on Linux, or on a large program.** If they show a real gain there, they can come back as fork commits behind a define. ### Tests removed All of them assert survival once address space is gone, which is no longer promised. | File | Removed | |---|---| | `test/aot/heap-limit-cases.ts` | 5 tests ("... where no address space is left", "... gets back only what malloc keeps, goes on", "... shapes of objects and of address space in one stretch ...") | | `test/aot/workers-within-address-space-limit-cases.ts` | 5 groups, 15 cases ("... find no room at once ...", "... has taken all the room, and the program goes on", "wave after wave ..."), and their 4 fixtures | | `test/aot/two-builds-give-the-same-bytes.test.ts` | 1 test that used the deleted option as its switch | | `test/internal/build-hardened-flags.test.ts` | 1 test about the `hand-out` patch list. The others now check the `MI_GETENV_ALLOWED` define. | ## 2. mimalloc v3.6.0 | Change | Why | |---|---| | `MIMALLOC_COMMIT` | The merge of upstream `v3.6.0` into the fork | | `purge_holes = 56` in `src/mimalloc_sys/mimalloc.rs` | The fork's options come after upstream's, and upstream added six. 50 is now `profile_time_interval` | Every other option number in that enum was compared with the merged header and is unchanged. To go back to v3.5.3 with the same fork commits, pin `6da53c355e1332db3161e0749111926ed35bc986` and set `purge_holes = 50`. What differs from upstream's defaults, and why, is in oven-sh/mimalloc#64: cache offsets only for blocks above 512 KiB, no purge on the thread that frees, `page_reclaim_on_free` at 0. ### Memory against the previous pin Linux x64, 64 cores, CI builds of this branch with v3.5.3 (`6da53c35`) and with v3.6.0 at an earlier commit (`eab19c23`, before the two changes to the defaults), 5 interleaved rounds, medians in MB. The two builds differ in mimalloc alone. | Workload | Peak, v3.5.3 | Peak, v3.6.0 | After going idle, v3.5.3 | After going idle, v3.6.0 | |---|---|---|---|---| | Startup | 16.7 | 16.7 | 14.4 | 14.6 | | 3M objects, 1 in 2000 kept | 583.4 | 578.9 | 96.6 | 91.4 | | Buffers of mixed sizes | 3170.5 | 3168.9 | 33.3 | 33.0 | | 48 MiB buffers in a loop | 66.5 | 66.3 | 18.8 | 18.7 | | 4 waves of 16 Workers | 582.1 | 587.9 | 36.9 | 36.1 | | `Bun.serve` under load | 54.4 | 52.6 | 32.7 | 32.4 | | Bundling a 17 MB file | 825.7 | 810.2 | | | The ranges of the five runs do not overlap for the bundle peak and for the objects after going idle. The Workers peak is noise (568 to 620 against 560 to 660). ### Costs | | v3.5.3 | v3.6.0 | |---|---|---| | `mi_malloc`, arm64 instructions | 22 | 23 | | `mi_free`, arm64 instructions | 35 | 35 | | `sizeof(mi_theap_t)` | 8240 | 8920 | | Binary size | | +24 KB Linux x64, +48 KB macOS arm64, +114 KB Windows x64 | ### Not measured Speed (the machine was busy, so the timings are rough and show no difference), memory on macOS, Windows or arm64, and a real application. ## Testing - macOS arm64, release build against the WebKit branch: it links, none of the removed symbols are in the binary, allocation-heavy JS and a Worker run, a huge `ArrayBuffer` is still a catchable `RangeError`, and a program built with `--compile --aot` runs. That build had the ballast removed but predates the `hand-out` removal and the move to the fork commit. - `test/internal/build-hardened-flags.test.ts` passes (11 of 11), and the build scripts typecheck. - CI on the final commit (build 134939): 180 jobs passed and none failed. That is every build lane, every Linux test job (Debian, Ubuntu and Alpine, x64 and arm64), the 20 ASAN jobs, the hardened lane, `cargo-test`, and Windows x64 and arm64. - **Not run:** the macOS test jobs. No machine picked them up in any build of this PR. The macOS build lanes pass, and the fork's own CI passes on macOS. - Five tests needed a retry (`testFFI`, `no-leak-records-at-exit`, `native-contract`, `decorators`). Each needed one as often in build 134852, which is `main` with a one-line fix. - `test/js/bun/wasm/compile-rss.test.ts` failed once and passed on retry in an earlier build with v3.6.0 on Alpine arm64: 10.8 MiB kept against a limit of 10. It did not in the final build. Its fixture, 24 runs per version on Linux x64, shows no difference (idle median -0.7 against -0.8 MiB, none over the limit). - The engine's own tests passed on Linux x64 and arm64 in oven-sh/WebKit#937, the hardened AOT lanes among them.
…s` (oven-sh#47326) ### What does this PR do? `test/bundler/compile-trim-capabilities.test.ts` fails on `main` since oven-sh#47291, on the two x64 glibc lanes: ``` expect(placesThatMakeSystemCalls(cut.file, startsAProgram.systemCalls)).toEqual([]); + [ { "address": 47263345, "number": 56 } ] ``` 56 is `clone`. oven-sh#47291 adds no call that starts a program, and what CI shows points at the scan, not at the executable: | Build | Commit | x64 glibc | aarch64 musl | |---|---|---|---| | 134869 | `main` before oven-sh#47291 | passes | passes | | 134909 | oven-sh#47291, one commit before its last | 47263345 and 47263370 | `vfork` at 107316644 | | 134932 | oven-sh#47291, last commit | 47263345 | passes | | 134948 | `main` with oven-sh#47291 | 47263345 | | | 134951 | oven-sh#47317, 14 more commits, mostly in the css, bundler and JS parser crates | 47263345 and 47263370 | | | **134973** | **this PR** | **passes** | **passes** | - The address is the same in four builds whose Rust code differs, so the code there does not change. The second place comes and goes, so some of its bytes do: they say where something else is. - The link step's report has the same number of entry points, edges between features, groups and cuttable ranges as on `main` before oven-sh#47291. On aarch64 musl, all its numbers are the same in 134909, which fails, and in 134932, which passes. `what-an-executable-can-call.ts` reads bytes, not instructions, and two of its rules match bytes that mean something else. What they match depends on where the linker puts things, so any change to compiled code can trip them. **x64: `0f 05` is not always `syscall`.** The scan took every `0f 05` in the code for one, and took its number from the nearest `b8` before it. But `0f 84 0f 05 00 00` is a jump of 0x50f bytes, `bf 0f 05 31 00` loads the address 0x31050f, and `c6 44 0f 05 2f` stores to `[rdi+rcx+5]`. A load of the number now counts only if the instructions from it on lead up to the `0f 05`. That takes the length of an instruction, so there is a function for it. Where it does not know an instruction, the place is reported as before. **arm64: a register that was overwritten.** To read a field of a global through the GOT, a compiler emits ``` adrp x8, <page of the GOT> ldr x8, [x8, #0x4a0] // the global's address ldr x0, [x8, #0x4c0] // its field ``` The scan paired the `adrp` with both loads, so the field's offset became an address in the GOT's page. If an import's slot is there, the function "calls" it. The scan now stops at the load or add that overwrites the register. **A reported place carries the bytes around it**, as `code`. Without them, and without the executable, a failure cannot be told from a false alarm. I have not seen the bytes at 47263345. CI on this PR was the test of this reading: the executable holds the same code as in build 134948, because only a test helper changes, and the test passes on both x64 glibc lanes and on the six other Linux lanes. ### Where the compiled code changes Nowhere. Only a helper of one test changes. ### How did you verify your code works? The test needs a release build that can trim, so CI runs it. What I checked is the scan, against `llvm-objdump`: - **Lengths.** On a release build of Bun 1.4.4 for Linux x64, the new function gives the disassembler's length for all but 7 of 20,163,723 instructions (the disassembler prints a `lock` prefix on a line of its own, and I counted it with the instruction it belongs to). Four are AVX-512 FP16 instructions, which it reports as unknown. At the other three the disassembler reads data. - **`syscall`.** Numbers the scan finds for the `0f 05` pairs in the code: | Build | `0f 05` pairs | of them `syscall` | given a number, before → after | the others | given a number, before → after | |---|---|---|---|---|---| | release, 1.4.4 | 948 | 700 | 700 → 700 | 248 | 85 → 5 | | debug + ASAN | 1,771 | 82 | 77 → 77 | 1,689 | 572 → 18 | "Given a number" means that the scan finds some load before it, not that the number is right. No `syscall` loses its number, and none gets a different one, except this: one in the release build gets the right one, 262, where a `b8` inside `be a2 b8 24 00` was read before. The numbers left on the others are above 300, and all but one above 700 million: there the `b8` is itself in the middle of an instruction. - The scan still finds `clone3` in `bun_clone3_vfork`, which the test asks for, and the same calls of `fork`, `vfork` and `execve`. - **arm64.** I compiled the three instructions above from C for `aarch64-linux-gnu`, next to a function that calls `vfork`, with the field at the offset of `vfork`'s slot. Before, the scan reports both functions. Now it reports the one that calls `vfork`. --- _Generated by [Claude Code](https://claude.ai/code)_ Co-authored-by: Claude <noreply@anthropic.com>
…an it has (oven-sh#47323) Follow-up to a review comment on oven-sh#47281. `ICU version does not regress` hands `process.versions.icu` to `Bun.semver.order`. On macOS the version is what the system's ICU reports, and `u_versionToString` can print three or four parts. The test beside it already allows for that (`/^\d+\.\d+(\.\d+)*$/`). Two things go wrong then: - `Bun.semver.order` throws on a fourth part. - It reads a missing part as the highest, so a floor of `72.1` is `72.1.MAX`, and `72.1.0` is below it. The floors have two parts, so the comparison now takes the first two parts of the version. | The test's lines, run by themselves, floor `72.1` | Before | After | |---|---|---| | `78.1` (macOS 26.7.1) | pass | pass | | `74.2.0.1` | `Invalid SemVer: 74.2.0.1` | pass | | `72.1.0.1` | `Invalid SemVer: 72.1.0.1` | pass | | `72.1.0` | fail (-1) | pass | | `71.9.9.9` | `Invalid SemVer: 71.9.9.9` | fail (-1), as it should | No machine is known to report more than two parts, so nothing fails today.
… parser (oven-sh#47317) ### What does this PR do? 14 separate changes, one commit each, and a last commit with review follow-ups, in the css parser and printer, the bundler and the JS parser. They follow oven-sh#47291 and work the same way: signatures name the lifetime that the value already has, hand-written pointer code becomes plain borrows that the compiler checks, and what was never needed is deleted. Together: **74 fewer `unsafe` blocks and 2 fewer `unsafe fn`s**, +320/−929 over 54 files. There is no new `unsafe`, and no safe function became `unsafe`. **[Where the compiled code changes](#where-the-compiled-code-changes) lists every place where the program does different work.** | # | Change | `unsafe` | Diff | |---|---|---|---| | 1 | css: the parser's accessors return the lifetime their tokens already have | −22 blocks | +100/−279, 30 files | | 2 | css: `Printer::get_import_record_url` returns `&'a [u8]` | −3 blocks | +3/−14, 4 files | | 3 | css: `Parser::arena()` returns `&'a Bump` | −4 blocks | +11/−24, 2 files | | 4 | css: `ImportInfo` borrows the bundler's column as it is | −4 blocks, −1 `unsafe fn` | +13/−75, 5 files | | 5 | css: `ImportRule` embeds `ImportConditions` | −2 blocks | +36/−103, 4 files | | 6 | bundler: `CompileResultForSourceMap` borrows its source map chunk | −3 blocks, −1 `unsafe fn` | +8/−22, 5 files | | 7 | bundler: copy the `ParentRef` out of `ParseTask.ctx` instead of detaching the borrow | −5 blocks | +11/−16, 1 file | | 8 | bundler: `AstBuilder` holds its scope by value | −3 blocks | +14/−49, 1 file | | 9 | js_parser: copy `Copy` AST nodes with a plain copy | −7 blocks | +16/−62, 8 files | | 10 | bundler: `find_imported_files_in_css_order` takes `&LinkerContext` | −3 blocks | +11/−33, 3 files | | 11 | bundler: index `graph.build_graphs` directly so the map slot is a field borrow | −4 blocks | +21/−46, 1 file | | 12 | bundler: the cleanup guards in `on_load` and `on_resolve` own their `&mut` | −3 blocks | +20/−61, 1 file | | 13 | bundler: drop lifetime erasers in front of callees that take any lifetime | −6 blocks | +24/−51, 4 files | | 14 | bundler: read the metafile paths from the transpiler's options | −2 blocks | +6/−18, 2 files | | 15 | Review follow-ups to changes 7, 9 and 10 | −3 blocks | +31/−81, 10 files | The commits can be reviewed one at a time. Worth knowing before reading on: - **Change 1 does not fix the `'static` placeholder on css tokens.** It stops the parser's callers from re-creating it. ### Where the compiled code changes Nothing here is visible to a JS program or to someone using the CLI. These are the places where the machine is asked to do different work: an allocation, a copy, a lookup, a branch, a store, or their order. They are what the tests below are aimed at. | Change | What differs | Effect | |---|---|---| | 1 css | Four places (`:lang()`, two in `calc.rs`, `StepPosition::parse`) call a parser accessor instead of holding a copy of its body. | Same tokens consumed, same errors. | | 2 css | `Specifier::to_css` takes its 8-byte receiver by value, not by reference. | Same output. | | 3 css | `FontFamily::parse` reads the arena only when it needs it. | Same result. | | 5 css | `ImportRule`, `ImportConditions` and `Layer` are no longer `repr(C)`, so rustc picks the field order. `ImportRule::to_css` calls `ImportConditions::to_css` instead of holding a copy of its body. One of the two places that build an `ImportRule` for a chunk fills `conditions` in the literal, so no default value is built and dropped first. | `ImportRule` stays 136 bytes, `ImportConditions` 104 and the enclosing rule enum 160. Same CSS. | | 6 source maps | An append to the per-chunk list copies a reference instead of a whole `sourcemap::Chunk`, and the one loop that reads the list follows it. | Same source maps. | | 7 bundler | `run_from_thread_pool_impl` unwraps `ParseTask.ctx` once instead of twice. | Same value. | | 8 bundler | One arena allocation fewer per generated AST. The scope is moved out of the builder, which leaves a default `Scope` behind. `parts[1].scopes` is the static empty slice instead of an empty slice in the arena. | Same AST. | | 9 js_parser | Two element loops are `extend_from_slice`: one `reserve` and one copy, so the capacity can end up smaller. The HMR import dedupe concatenates with `split_at_mut` and `copy_from_slice`. | Same contents. | | 11 bundler | In `on_resolve`, the map slot and `out_source_index` are written before the pretty path is computed, not after. | Nothing in between reads them. | | 14 bundler | Two stores fewer in `BundleV2::init`, `LinkerOptions` is two slices smaller, and up to five reads per `Bun.build` go through `self.transpiler`. | Same paths. | | 15 js_parser | Two compactions test `a < b` instead of `a != b`, and assign instead of calling `copy_nonoverlapping`. A third swapped unconditionally and now has the same guard and assignment. Two places read a `Decl` whole instead of field by field. | Same contents. In the release assembly of `bun_js_parser`, two functions differ in their number of instructions or bounds checks: `visit_stmts` has one bounds check fewer and one instruction more, `Parser::_parse` one or two instructions fewer. In `ImportScanner::scan`, `to_ast` and `s_local`, the guard's jump tests another condition. | | 15 bundler | `find_imported_files_in_css_order` no longer tests `parse_graph` for null in a release build. As for every other reader, that is a `debug_assert`. `css_order_or_log` takes one parameter fewer. `postProcessJSChunk.rs` appends the cross-chunk exports with `extend_from_slice`. | Same order, same error, same items. | Changes 4, 10, 12 and 13 change only types, lifetimes, declarations and comments, and ask for the same operations in the same order. I have not compared binaries: a reference where there was a raw pointer can let the optimizer choose differently, and every moved line changes the panic locations. ### The changes in detail #### 1. css: the parser's accessors return the lifetime their tokens already have `Token` payloads are `&'static [u8]`. That `'static` is a placeholder for the arena's lifetime, and it is created in four places in the tokenizer. `Parser::expect_ident`, `expect_string`, `expect_function`, `expect_url` and the like copy such a payload out of a token, but their elided return type shrank it to the `&mut self` borrow. Callers that needed the parser again then widened it back with `unsafe` (`src_str`, `arena_str`, `&*ptr::from_ref(..)`), copied the accessor's body to get at the token, or used one of six `*_cloned` helpers that existed only for this. The accessors and `slice_from` now say `&'static [u8]`, which is what they return. The `*_cloned` helpers and the copied bodies are deleted, and the parameters that are only ever fed a token payload (`name` in `parse_prelude`, `parse_at_rule` and `parse_value`, `url` in `add_import_record`, `prefix` in `namespace_for_prefix`) take the payload's type. `add_symbol_for_name` stores its name in a `StoreStr`, which takes any lifetime, so the widening in front of it was never needed. This does not fix the placeholder: that needs a lifetime on `Token` and on the css AST. It stops re-creating it on what the accessors return. `src_str` is private to `css_parser.rs` now, with six calls left: the four in the tokenizer and two in `fill_property_bit_set`. The other uses of `arena_str` are not touched. #### 2. css: `Printer::get_import_record_url` returns `&'a [u8]` Everything it returns is `&'a`: a slice held by `import_info`, or a copy made in `self.arena: &'a Bump`. The elided return type tied the result to `&mut self`, and all three callers erased that with `unsafe` so that they could write to the printer. `import_record()` next to it already returns `&'a ImportRecord<'a>` from `&mut self`. With the raw pointer gone from its body, clippy asks `Specifier::to_css` to take its 8-byte `Copy` receiver by value, as `deep_clone`, `hash` and `eql` next to it do. #### 3. css: `Parser::arena()` returns `&'a Bump` The tokenizer holds a `&'a Bump`, and the accessor returned it as a borrow of `&self`, so two callers detached it only to use the parser again. The two rule parsers are only ever built with the `&'static Bump` that `parse_with` takes, stored it as `&'a` and erased it back for their `DeclarationList<'static>`. They keep the type they are given. `DeclarationBlock::parse` and `PageRule::parse` still detach `input.arena()`. They need `'static` for `DeclarationBlock<'static>`, which is the placeholder from change 1 and not something the accessor can give. #### 4. css: `ImportInfo` borrows the bundler's column as it is `ImportInfo::ast_unique_key_for_additional_file` was a `&[&[u8]]`, and the bundler's column is a `&[Box<[u8], AstAlloc>]`. Three callers bridged that with `bun_ptr::boxed_slices_as_borrowed`, which reinterprets one slice as the other and whose own doc says that it relies on a field order that "is not a language guarantee". The field now has the column's type, its one reader derefs the box, and the function is deleted. It was the only use of `allocator_api` in `bun_ptr`, so the crate's `#[feature]` goes with it. #### 5. css: `ImportRule` embeds `ImportConditions` `ImportRule` repeated the three fields of `ImportConditions`, and `conditions()` / `conditions_mut()` reinterpreted them in place by byte offset, guarded by a compile-time layout check and a `TODO: change this to be field on ImportRule`. It is a field now, and the three `#[repr(C)]` that the cast needed are gone. The size is the same. `ImportRule::to_css` calls `ImportConditions::to_css` instead of repeating it, and the three clones move into a new `ImportConditions::deep_clone`. #### 6. bundler: `CompileResultForSourceMap` borrows its source map chunk The list that `generate_source_map_for_chunk` reads held a bitwise copy of each `sourcemap::Chunk`, made with `unsafe fn Chunk::alias`. That was two owners of one buffer, and it only worked because a `MultiArrayList` does not drop its elements. The originals live in `chunk.compile_results_for_chunk`, which outlives the list, so the list holds `&Chunk`. `Chunk::alias` is deleted. An append now copies 8 bytes where it copied the whole `Chunk`, and the one loop that reads the list follows the reference. #### 7. bundler: copy the `ParentRef` out of `ParseTask.ctx` instead of detaching the borrow `unsafe fn ParseTask::ctx` returns `&'r BundleV2` with a lifetime the caller picks, so that the bundle can be read while the task is borrowed mutably. `ctx` is an `Option<ParentRef<BundleV2>>`, and a `ParentRef` is `Copy` and derefs. Five of the six callers only need the bundle inside their own function, and a local copy gives them that with the same `expect`. The sixth, `get_code_for_parse_task_without_plugins`, stores a path borrowed from the bundle in an out-parameter and keeps the call. #### 8. bundler: `AstBuilder` holds its scope by value `AstBuilder` allocated its module scope in the arena, kept a `*mut Scope` to it, dereferenced that for every symbol, and at the end copied the scope out of the arena with `ptr::read`. Nothing else ever points at the scope, so the builder holds it as a field and `to_bundled_ast` moves it out with `mem::take`, like the fields next to it. The `scopes` stack goes too. Nothing pushed to it (there is no `push_scope`), so `is_top_level` was always true and `parts[1].scopes` was always empty, which is `Part`'s default. #### 9. js_parser: copy `Copy` AST nodes with a plain copy Seven places copied a `StmtOrExpr`, a `G::Decl` or a `ClauseItem` with `ptr::read` or `copy_nonoverlapping`, most of them next to a comment that the type is not `Copy` or not `Clone`. `StmtOrExpr` and `Decl` derive it. `ClauseItem` did not, although all its fields are `Copy`, and derives it now. - The two element loops become `extend_from_slice`, which reserves once and copies. - The concatenation in the HMR import dedupe becomes `split_at_mut` and two `copy_from_slice`. Their length checks fold away, because both lengths come from `u32` fields: on a stand-alone copy of the function at `opt-level=3`, none of the three checks leaves a branch or a panic path. - Five comments elsewhere that said one of these types is not `Copy` or `Clone` are corrected. - The in-place compactions follow in change 15. #### 10. bundler: `find_imported_files_in_css_order` takes `&LinkerContext` It only reads the linker, but it asked for `&mut`. `compute_chunks` holds shared borrows of the graph's columns at both calls, so it made the `&mut` from a raw pointer, next to those borrows. With `&LinkerContext` the calls are plain, and `css_order_or_log` takes the reference too. The raw pointer had one use left, the step to the owning `BundleV2`. It is now taken where it is used, as at the other call sites of `bundle_v2_ptr`. #### 11. bundler: index `graph.build_graphs` directly so the map slot is a field borrow `on_resolve` and `process_resolve_queue` look a path up with `get_or_put` and fill the slot a few lines later, after reading other fields of the graph. `path_to_source_index_map(&mut self)` borrows the whole bundle (in `process_resolve_queue`, which calls `Graph`'s accessor of that name, the whole graph), so both kept the slot as a raw pointer. Inlined, the accessor is `&mut self.graph.build_graphs[target]`. Written out, as three other places in the file already do, it borrows one field, and the slot stays a reference. In `on_resolve` the slot is now filled before the pretty path is computed. That call borrows the whole bundle and reads only `graph.heap`, and its failure path is an `expect`, so nothing can see the order. `run_resolver`'s FileMap branch keeps its raw slot: what it stores comes back from `enqueue_parse_task(&mut self)`. #### 12. bundler: the cleanup guards in `on_load` and `on_resolve` own their `&mut` Three guards held a raw pointer to a `&mut` parameter, so that the function body could keep using the parameter, and dereferenced it in a hand-written `Drop`. `scopeguard::guard` takes the `&mut` itself and lends it back through `DerefMut`, as the `Worker::unget` guards in this crate do. The guards run in the same order, and the two early `drop(load_deinit)` calls stay. `ScanCounterGuard` and `decrement_scan_counter_on_drop` had one user and are deleted. #### 13. bundler: drop lifetime erasers in front of callees that take any lifetime `E::String::init` and `StoreStr::new` take `&[u8]` of any lifetime and keep a pointer, so widening the argument to `'static` first did nothing. Five such `unsafe` blocks go, and the `astr` closure that wrapped one of them. In `ParseTask`, `bump` is `&'static Bump`, so `bump.alloc(..)` is `'static` without `detach_ref`. What the callee's type does not express is unchanged: the strings have to live as long as the AST that points at them, and they do, in the same arenas as before. #### 14. bundler: read the metafile paths from the transpiler's options `LinkerOptions` kept `&'static [u8]` copies of `metafile_json_path` and `metafile_markdown_path`, erased from the `Box<[u8]>` in `transpiler.options`. Only `run_from_js_in_new_thread` reads them, it has the transpiler at hand, and nothing writes those options after `BundleV2::init`. The two fields are deleted. #### 15. Review follow-ups to changes 7, 9 and 10 **Three more `unsafe` blocks in the JS parser (change 9).** Two in-place compactions still copied with `copy_nonoverlapping`: the import items that stay in `scan_imports.rs` and the kept declarations in `visit_decls`, which also made its `&mut decls[i]` from a raw pointer. They are now `if a < b { items[a] = items[b]; }` and `&mut decls[i]`, and `visit_decls` counts with `for`. The old guard was `a != b`. Written as `a < b`, next to the loop's `b < len`, it proves both indexes in range, so no bounds check is left. In the release assembly of `bun_js_parser`, `ImportScanner::scan` and `s_local`, which `visit_decls` is inlined into, have as many instructions and as many bounds checks as before. **Code that worked around the missing `Copy` (change 9).** - The compaction in `visit_stmts` used `swap`. It is the same guarded assignment now, with one bounds check fewer. - `parse_entry.rs` and `postProcessJSChunk.rs` rebuilt a `Decl` or a `ClauseItem` field by field. They copy it. - The HMR export loop read a `Decl` with two indexings, one per field. It reads it once. Its three stores still build the value from the fields: storing the copy made `ImportScanner::scan`, which the loop is inlined into, up to 92 instructions longer. - Three `swap` compactions of `ClauseItem`s in `visit_stmt.rs` stay as they are. This PR does not touch that file. **`find_imported_files_in_css_order` (change 10).** Its visitor held the parse graph as a `BackRef` made from the raw field, under a comment that argued from the old `&mut` signature, as did a second comment further down. It borrows `this.parse_graph()`, and both comments are gone. `css_order_or_log` reads the parse graph from the linker it is given, so its second parameter is gone. **Dead code.** Change 9 removed the last callers of `StoreSlice::as_ptr`, which the `mordant` check reports. It is deleted. **Comments.** - The doc of `unsafe fn ParseTask::ctx` (change 7) named callers that are gone, sent writers to a field they do not use, called its `expect` a debug assertion, and said that `init()` sets `ctx`, which five other places also do. What is left is the contract and when it panics. - The `SAFETY` comment on the `ptr::read` of a `G::Fn` in `visit_stmts` said that the type only lacks `derive(Copy)`. It cannot have it: `return_ts_metadata` can own a `Vec`. The read is a move, and the comment now says why that is sound. - The comment on `Scope` told callers to `ptr::read` it. Both places that move one out use `mem::take`, so it is deleted. ### How did you verify your code works? - `cargo clippy --workspace`, `cargo fmt --check` and `bun test test/internal/source-lints/` pass. - `cargo check --workspace --release` passes for Linux, macOS and Windows. - No change is visible from outside, so there are no new tests. For the changes listed under "Where the compiled code changes" I ran the css, bundler and dev server test files below on a debug + ASAN build of all 15 commits and on a debug + ASAN build without them, and compared the names of the failing tests. - For change 15 I also compared the release assembly of `bun_js_parser` with and without it, function by function: instructions and bounds checks. The table above has the result. - Some edits to commit 15 came after the test run: `visit_decls` counts with `for` instead of `while`, the unused `StoreSlice::as_ptr` is deleted, and comments changed. The static checks above ran on the final commits, and with the `for` loop `s_local` has the same number of instructions and bounds checks in the release assembly. <details> <summary>46 test files: 5,159 pass and 74 fail, and the same 74 fail without this PR</summary> | Test file | Pass | Fail | Same without this PR | |---|---|---|---| | `test/bake/dev/bundle.test.ts` | 26 | 28 | yes | | `test/bake/dev/esm.test.ts` | 11 | 7 | yes | | `test/bake/dev/html.test.ts` | 6 | 9 | yes | | `test/bake/dev/sourcemap.test.ts` | 1 | 1 | yes | | `test/bundler/bun-build-api.test.ts` | 91 | 0 | no, see below | | `test/bundler/esbuild/default.test.ts` | 155 | 0 | yes | | `test/bundler/esbuild/css.test.ts` | 68 | 0 | yes | | `test/bundler/esbuild/splitting.test.ts` | 26 | 0 | yes | | `test/bundler/esbuild/loader.test.ts` | 10 | 0 | yes | | `test/bundler/bundler_loader.test.ts` | 62 | 0 | yes | | `test/bundler/css/css-modules.test.ts` | 46 | 0 | yes | | `test/bundler/bundler_plugin.test.ts` | 68 | 0 | yes | | `test/bundler/bundler_html.test.ts` | 26 | 0 | yes | | `test/bundler/esbuild/packagejson.test.ts` | 94 | 0 | yes | | `test/js/bun/css/css-fuzz.test.ts` | 39 | 0 | yes | | `test/js/bun/css/doesnt_crash.test.ts` | 61 | 0 | yes | | `test/js/bun/css/supports-condition-newline.test.ts` | 2 | 0 | yes | | `test/js/bun/css/attr-selector-namespace-star.test.ts` | 2 | 0 | yes | | `test/js/bun/css/color.test.ts` | 1146 | 0 | yes | | `test/bundler/css/wpt/background-computed.test.ts` | 24 | 0 | yes | | `test/bundler/css/wpt/color-computed-rgb.test.ts` | 94 | 0 | yes | | `test/bundler/css/wpt/color-computed.test.ts` | 14 | 0 | yes | | `test/bundler/css/wpt/relative_color_out_of_gamut.test.ts` | 27 | 0 | yes | | `test/bundler/bundler_npm.test.ts` | 2 | 0 | yes | | `test/bundler/bundler_splitting.test.ts` | 209 | 0 | yes | | `test/bundler/bundler_edgecase.test.ts` | 2 | 0 | yes | | `test/bundler/bun-build-compile-sourcemap.test.ts` | 9 | 0 | yes | | `test/bundler/native-plugin.test.ts` | 19 | 0 | yes | | `test/bundler/bundler_minify.test.ts` | 67 | 0 | yes | | `test/bundler/esbuild/ts.test.ts` | 59 | 0 | yes | | `test/bundler/transpiler/transpiler.test.js` | 258 | 0 | yes | | `test/bundler/bundler_plugin_chain.test.ts` | 13 | 0 | yes | | `test/bundler/bundler_defer.test.ts` | 12 | 0 | yes | | `test/bundler/plugin-requests-run-ahead.test.ts` | 2 | 0 | yes | | `test/bundler/plugin-error-nested-throw.test.ts` | 2 | 0 | yes | | `test/bundler/bundler_html_server.test.ts` | 4 | 0 | yes | | `test/bundler/metafile.test.ts` | 65 | 0 | yes | | `test/bundler/html-import-manifest.test.ts` | 10 | 0 | yes | | `test/bundler/esbuild/importstar.test.ts` | 99 | 0 | yes | | `test/js/bun/plugin/plugins.test.ts` | 0 | 1 | yes | | `test/js/bun/md/md-edge-cases.test.ts` | 184 | 0 | yes | | `test/bake/dev/hot.test.ts` | 0 | 12 | yes | | `test/bake/dev/production.test.ts` | 21 | 0 | yes | | `test/js/bun/css/css.test.ts` | 2018 | 0 | yes | | `test/bake/dev/plugins.test.ts` | 3 | 0 | yes | | `test/bake/dev/css.test.ts` | 2 | 16 | yes | | **Total** | **5159** | **74** | | The 74 failures are the same 74 tests on both builds: - 73 in `test/bake/dev` drive a browser page through the `happy-dom` package, which is not installed on the machine I used. - `plugins.test.ts` needs `svelte`, which is not installed there either. `Bun.build can be called thousands of times in one process without crashing`, in `bun-build-api.test.ts`, has a limit of 180 s, and on these builds it takes about that long. It stayed under the limit on the build with this PR and ran into it on the build without, in two runs. The whole file took 374 s with this PR and 381 s without. </details> --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ven-sh#47338) ### What does this PR do? 23 separate changes, one commit each, in `src/sys`, the shell and a few places around them. They follow oven-sh#47291 and oven-sh#47317 and work the same way: signatures name the type and the lifetime that the value already has, hand-written pointer code becomes plain ownership and borrows that the compiler checks, and what was never needed is deleted. Together: **118 fewer `unsafe` blocks, 31 fewer `unsafe fn`s and 19 fewer `unsafe impl`s**, +934/−1552 over 63 files. There is no new `unsafe`, no safe function became `unsafe`, and no wrapper type or helper was added. **[Where the compiled code changes](#where-the-compiled-code-changes) lists every place where the program does different work.** | # | Change | `unsafe` | Diff | |---|---|---|---| | 1 | sys: the syscall wrappers take slices and references instead of raw pointers | −40 blocks, −20 `unsafe fn` | +210/−412, 10 files | | 2 | sys: `c::fchmod` is the `safe fn` declaration that `bun_sys` already has | −5 blocks | +8/−15, 3 files | | 3 | Remove 16 `unsafe impl Send` / `Sync` that the compiler derives anyway | −16 `unsafe impl` | +31/−108, 13 files | | 4 | s3: the request setup borrows from a `&'static` task instead of erasing ten lifetimes | −10 blocks | +26/−52, 2 files | | 5 | Drop five lifetime erasers that change nothing | −5 blocks | +7/−24, 4 files | | 6 | event_loop: `MiniEventLoop::enqueue_task_concurrent` takes `&self` | −2 blocks | +11/−20, 5 files | | 7 | shell: `IOReader::set_interp` and `IOWriter::set_interp` take `&Interpreter` | −3 blocks, −1 `unsafe fn` | +17/−44, 7 files | | 8 | shell: `Script`, `Stmt` and `Expansion` take `&ast::X` like the other states | −3 blocks | +18/−37, 9 files | | 9 | shell: `ParseError` and `ParseFlagResult` borrow the option name | −1 block | +87/−87, 6 files | | 10 | shell: `Cmd::redirection_fd` is a plain `Fd` | −1 block | +39/−48, 4 files | | 11 | shell: `BuiltinIO::write_no_io_to` takes `&mut ShellExecEnv` | −2 blocks, −1 `unsafe fn` | +14/−21, 2 files | | 12 | shell: use `Base::shell()` / `shell_mut()` where the node is already borrowed | −3 blocks | +12/−21, 3 files | | 13 | shell: drop five `unsafe` blocks that have a safe spelling | −5 blocks | +10/−30, 3 files | | 14 | shell: `JsCell::with_mut` for `vm_args_utf8` and the recycled node slot | −2 blocks | +14/−20, 2 files | | 15 | shell: `ls` fills the subtask before leaking it and appends bytes as bytes | −3 blocks | +11/−13, 1 file | | 16 | shell: `ShellSubprocess::defuse_array_buffer_unpins` takes `&mut self` | −2 blocks, −1 `unsafe fn` | +6/−10, 2 files | | 17 | node:fs: `MkdirCtx::on_create_dir` takes `&mut self` | −1 block | +15/−20, 2 files | | 18 | shell: the brace lexer's char iterators borrow their input | −2 blocks | +29/−41, 1 file | | 19 | shell: `SmolList`'s heap list allocates through `&'a Bump` | −4 blocks, −3 `unsafe fn`, −3 `unsafe impl` | +24/−78, 3 files | | 20 | shell: `CmdHandle` holds a plain `ParentRef`, and its users call `as_cmd_mut` | −6 blocks, −1 `unsafe fn` | +22/−66, 3 files | | 21 | shell: `OutputTask` is passed around as the `Box` it is | −8 blocks, −3 `unsafe fn` | +169/−160, 6 files | | 22 | shell: the brace AST borrows its arena, parent links are `Cell`s | −4 blocks, −1 `unsafe fn` | +125/−167, 1 file | | 23 | shell: `ls`, `touch` and `cp` reclaim a finished pool task where it arrives | −6 blocks | +30/−59, 3 files | The commits can be reviewed one at a time. Worth knowing before reading on: - **Two leaks go away (changes 21 and 23).** A shell `OutputTask` that is still queued, and on Windows a `cp` task that is still parked after `EBUSY`, is now freed when its command is torn down. Both used to leak. The first has a new test that fails without this PR. The second has none. - **Some blocks are replaced by a safe accessor that already exists and has the `unsafe` inside.** That is all of change 12 (`Base::shell()`), three of the five sites of change 13 (`as_class_ref`, `worker_ref`, `loop_ctx`), change 14 (`JsCell::with_mut`) and one line of change 7 (`IOReader::reader()`). At those sites the code stops repeating what its neighbours already write, and in change 14 the borrow gets shorter, but the compiler checks nothing that it did not check before. No accessor was added or widened. - **Change 4 states one `'static` where there were ten.** The S3 task holds an `AsyncHTTP<'static>` that borrows from the task's own buffers, which no lifetime can express. The `'static` is still a promise, made at the one `unsafe` deref that was already there. - **Change 10 changes a test.** It relied on LeakSanitizer seeing a heap box that no longer exists. ### Where the compiled code changes I compared the release assembly (x86-64 Linux, `--emit=asm`) of all 18 crates that this PR touches, before and after, function by function, with names demangled and labels normalised. | Crates | Functions | Result | |---|---|---| | `bun_sys`, `bun_io`, `bun_watcher`, `bun_http`, `bun_core`, `bun_threading`, `bun_ptr`, `bun_event_loop`, `bun_spawn`, `bun_resolver`, `bun_bundler`, `bun_jsc`, `bun_js_parser_jsc`, `bun_standalone_graph`, `bun_install` | 9,415 | Every function has the same instructions. | | `bun_js_parser` | 1,289 | Same instructions. 12 instances of `OnceLock` code are named after `DefineData` instead of `SyncDefineData`, and `Define::for_identifier` calls them by that name. | | `bun_shell_parser` | 122 | 4 functions differ, and the `Vec` growth code is instantiated for `&Bump` instead of `SmolListAlloc` (changes 19 and 22). | | `bun_runtime` | 10,462 | 131 functions differ. 130 belong to the shell. The other one calls a `VecDeque::grow` that the compiler merged under another name. | So **changes 1 to 5 and 17 compile to the same code on Linux, and so does change 6 outside the shell**: the syscall wrappers and their callers, `fchmod`, the 16 impls, S3, the five erasers, `mkdir_recursive_impl` and `MiniEventLoop`. What the comparison does not cover is code for other targets: the macOS, FreeBSD and Windows arms of change 1, and the Windows half of change 23. Those are type-checked only. Of the 131 functions in `bun_runtime`, 77 keep their number of instructions, 45 get shorter and 9 get longer. Over all of them there are 55 fewer conditional branches, 56 fewer compares and 28 fewer calls. | Change | What differs | Effect | |---|---|---| | 10 | One `malloc` and one `free` fewer per `> file` redirect. `Cmd`'s layout changes, and with it how rustc tells `Node`'s variants apart: `Node` has other tag values and field offsets. It stays 176 bytes. | The fd is closed at the same statement. 75 of the 130 functions keep the same number of every kind of instruction and differ in constants, offsets and registers, most visibly in these tag values and offsets. | | 12 | In `Expansion::next`, `base.shell` is loaded where it is used instead of before the loop. | Nothing in the loop writes it. | | 13 | The `Box<MaybeUninit<Subprocess>>` becomes a raw pointer before the fields are written, not after. | Same allocation, same writes. | | 14 | The borrow of `vm_args_utf8` ends after one call instead of spanning the loop body. | Same calls. | | 15 | `ls -l` appends the file type, the permissions and the time with a push and two copies instead of three `fmt` arguments. `add_entry_long_from_stat` grows from 352 to 400 instructions, because four capacity checks that ran inside the formatter are now inline. In `enqueue`, two fields are stored before the box is leaked, not after. | The line has the same bytes, with fewer calls into the formatter. | | 19 | `RawVec::grow_one` has `grow_amortized` inlined (11 to 27 instructions, one function fewer), `SmolList::append` goes from 81 to 74 and `parse_if_body` from 255 to 251. `add_error` keeps its 89 in another order. | Same allocations from the same arena. | | 20 | `run_yield_with` loses a null test that was never taken. `SpawnArgs` is one pointer smaller. | Same calls. | | 21 | `write_err` and `write_out` hand the box back in their return value. `write_err` is 2 to 6 instructions longer (moves, and for `ls`, `mkdir` and `touch` the cold call for `unreachable!()`), `write_out` 1 to 4 shorter, with the same number of branches. `write_out` copies the output before it queues the task, not after. In a state other than `Exec`, `ls`, `mkdir` and `touch` used to skip the push and now panic. That state cannot be reached: `Exec` ends only when every task and every write has been counted as done. | Same output in the same order. | | 21 | The drop code of `Box<Builtin>` frees what is left in the queue: 147 to 198 instructions, and two new drop functions for the states of `ls` and `mkdir`. | **A queued task and its output are freed at teardown instead of leaking.** | | 22 | `expand_nested` goes from 378 to 387 instructions: 5 moves and 4 others. | No compare, branch or call more, so no new bounds check. Same expansions. | | 23 | For `ls` and `touch`, the finished pool task is dropped by `run_from_main_thread` right after the callback returns, instead of at the end of the callback. On Windows, `cp` uses `VecDeque::push_back` / `pop_front` instead of `Vec::push` and a cursor. | Same order. **On Windows, tasks still parked at teardown are freed instead of leaking.** | The other functions that got longer: `Cmd::init` by 2 moves, `as_cmd_mut` by 1 instruction, and `Interpreter::child_done` by 25, which now has `Vec::retain` and `as_expansion_mut` inlined (both are gone as separate functions). Changes 7, 8, 9, 11, 16 and 18 ask for the same operations in the same order, and show up, if at all, through the new layout of `Node` or as shorter code. ### The changes in detail #### 1. sys: the syscall wrappers take slices and references instead of raw pointers The private `sys_openat`, `sys_read`, `sys_write`, `sys_pread`, `sys_pwrite`, `sys_recv` and `sys_send`, and the public `linux::{read, sendfile, ppoll, epoll_ctl, copy_file_range, inotify_add_watch}`, `freebsd::copy_file_range` and `c::sendfile`, were `unsafe fn`s over `(ptr, len)` pairs and nullable pointers. Every caller has a slice, a `&ZStr` or a local, so they take `&[u8]` / `&mut [u8]`, `&ZStr`, `Option<&T>` and `Option<&mut T>`, are safe, and the callers drop their `unsafe`. The one FFI call in each keeps its `unsafe` block. The macOS arm and the other-POSIX arm of `openat`, `read`, `write`, `recv` and `send` were the same code twice and are one arm now. `c::sendfile` loses its `hdtr` parameter, which was always null. The FreeBSD arm of `copy_file` calls `freebsd::copy_file_range` instead of `libc`'s. #### 2. sys: `c::fchmod` is the `safe fn` declaration that `bun_sys` already has `bun_sys::c::fchmod` re-exported `libc::fchmod`, an `unsafe fn`, so its five callers wrapped it in `unsafe`. It takes two integers. `safe_libc` already declares it as a `safe fn` for the crate's own use, and `c` re-exports that one. #### 3. Remove 16 `unsafe impl Send` / `Sync` that the compiler derives anyway | Type | Removed | |---|---| | `ThreadCell`, `RacyCell`, `Once` (`bun_core`) | `Send` | | `DarwinImpl`, `RwLock`, `Semaphore` (`bun_threading`) | `Send` | | `DefineData`, `SyncDefineData` (`bun_js_parser`) | `Send` and `Sync` | | `fs::Entry` (`bun_resolver`) | `Send` | | `SourceMapDataTask` (`bun_bundler`) | `Send` | | `GenerateChunkCtx` (`bun_bundler`) | `Send` and `Sync` | | `CacheState` (`bun_jsc`), `WriteFile` (`bun_runtime`) | `Send` | Each of these is `Send` (or `Sync`) by its fields alone, under the same bounds that the hand-written impl had, so the impl only repeated what the auto trait says. I found them by replacing each impl with an assertion that the type has the trait, and type-checking on Linux, Windows and macOS. Without the impl, a new field that is not `Send` is a compile error wherever the type has to be `Send`, instead of a silent claim. `SyncDefineData` was a wrapper that existed to carry two of them, under a comment that `DefineData` is not `Send` or `Sync`. It is both, so the four `OnceLock`s in `defines_table.rs` hold the `DefineData` itself. `vm-thread-door.inventory.json` lists the `unsafe impl Send` of the VM crates and loses the two that were among the 16. #### 4. s3: the request setup borrows from a `&'static` task instead of erasing ten lifetimes The three functions that start an S3 request turn the fresh task pointer into a reference of a lifetime they pick, then borrowed `sign_result.url`, `headers.buf`, `body` and `proxy_url` from it and widened each borrow to `'static` with `detach_lifetime*`. The reference is `'static` itself, so the borrows need no widening and the compiler checks that they come from the task. The task lives until `on_response` drops it. The `unsafe` deref says what the `'static` rests on: `task` is not used once `task_ptr` is dereferenced, and only buffers that are never written again are borrowed. #### 5. Drop five lifetime erasers that change nothing I replaced every `unsafe { detach_lifetime*(x) }` and the like by `x` and kept what still compiles, then read each site for a pointer that is stored and outlives the borrow. These five have none. - `RequestContext::on_writable_bytes` and `render_bytes` use the blob's bytes only while they hold the context. - `RuntimeTranspilerStore`: `parse_options` does not need a `'static` source. It borrows the `fallback_source` slot, which is declared before it. - `jsc_hooks.rs` and `Macro.rs`: the value already comes from a reference of a lifetime that the function picks, so widening it again did nothing. The `# Safety` doc of `get_loader_and_virtual_source` now names `module_loader.eval_source`, which only the removed comment did. #### 6. event_loop: `MiniEventLoop::enqueue_task_concurrent` takes `&self` The body pushes onto the MPSC queue (`UnboundedQueue::push(&self)`) and reads the loop pointer, so it never needed `&mut self`. Callers had to write `unsafe { mini.get_mut() }` to call it, and `ConcurrentPoster` did that from pool threads while the loop's own thread was ticking. With `&self` they go through `BackRef`'s `Deref`. `enqueue_task_concurrent_with_extra_ctx` ends in the same push and wakeup and takes `&self` too. #### 7. shell: `IOReader::set_interp` and `IOWriter::set_interp` take `&Interpreter` Both were safe fns with a `# Safety` section: they took a `*mut Interpreter` and wrapped it with `ParentRef::from_nullable`. All nine callers hold a `&Interpreter` and turned it into a pointer only for this call, so the parameter is the reference and the field is filled with `ParentRef::new`. `from_nullable` had no other caller and is removed. #### 8. shell: `Script`, `Stmt` and `Expansion` take `&ast::X` like the other states Their `init` took a `*const ast::X` and built the `BackRef` with `unsafe { BackRef::from_raw(..) }`. The comment said that callers pass `&raw const` to escape borrowck, but every state fn takes `&Interpreter` and every caller has a plain `&ast::X` into the AST arena in hand. #### 9. shell: `ParseError` and `ParseFlagResult` borrow the option name Both carried the offending option as a `*const [u8]`, and `ParseError::opt()` turned it back into a slice with `unsafe { &**s }`. The slice is either a literal or a piece of the flag that `parse_flags` reads out of argv, so the two enums get a lifetime and hold `&'a [u8]`. `unsupported_flag` turned a literal into the raw pointer. There is nothing left for it to do, so its 26 callers pass the literal, which is most of the diff. #### 10. shell: `Cmd::redirection_fd` is a plain `Fd` `CowFd` was a heap-allocated fd with a refcount, handed around as a `*mut CowFd`. There is one `CowFd::init`, one `CowFd::deref` (a safe fn that dereferenced its raw argument) and nothing that ever takes a second reference, so the count only went from 1 to 0 in `Cmd::deinit`. The `Cmd` stores the `Fd` itself and `Cmd::deinit` closes it at the same place. The test "does not leak the redirection fd" only asserted that LeakSanitizer stays quiet, and what LeakSanitizer saw was the `CowFd` box. It now checks the fd: `out.txt` is open in the process while the command runs, and is closed after `worker.terminate()`. #### 11. shell: `BuiltinIO::write_no_io_to` takes `&mut ShellExecEnv` It was an `unsafe fn` whose contract was "`shell` must be the live `cmd.base.shell`". Both callers read that raw pointer out of the `Cmd` they had just borrowed, next to `&mut cmd_node.exec`. `base` and `exec` are disjoint fields, so they pass `cmd_node.base.shell_mut()` and the fn is safe. The one block that stays is the write through the `Bufio` pointer, which may belong to a parent env. #### 12. shell: use `Base::shell()` / `shell_mut()` where the node is already borrowed Three places copied the raw `base.shell` pointer out of a node and dereferenced it by hand. The comment in `Cmd` said that `Base::shell()` would block every later `interp.as_cmd_mut(..)`, which stopped being true when the accessors moved to `&self`. #### 13. shell: drop five `unsafe` blocks that have a safe spelling - `init_and_run_impl`: the `is_done` closure for `mini.tick` captured a raw pointer to the interpreter "so borrowck doesn't see an overlap". `interp` and `mini` are separate locals, so it borrows `interp.flags`. - `spawn_maybe_sync_impl`: the uninitialised `Box<Subprocess>` is turned into its raw pointer when it is allocated. It was kept as a `Box` next to a pointer into it, only to be `assume_init`ed and leaked after the write. - `as_class_ref::<Response>()` as for `Blob` a few lines above, `vm.worker_ref()`, and `global.bun_vm().loop_ctx()`. #### 14. shell: `JsCell::with_mut` for `vm_args_utf8` and the recycled node slot `Expansion::next` made a `&mut` to `interp.vm_args_utf8` at the top of its loop and kept it for the whole body, though only the call to `expand_simple_no_io` uses it. That call runs inside `with_mut`, so the borrow ends with it. `alloc_node` stored into a recycled slot through `unsafe { nodes_mut() }`, and the statement right below it already uses `nodes.with_mut`. #### 15. shell: `ls` fills the subtask before leaking it and appends bytes as bytes `ShellLsTask::enqueue` turned the new subtask into a raw pointer first and then set two fields through it. It keeps the `Box` until both are set, as `create` does. `add_entry_long_from_stat` went through `from_utf8_unchecked` twice, only to hand the permission and time bytes to `write!` into a `Vec<u8>`. Neither has a width, so they are appended with `extend_from_slice` around the `write!` of the four padded numbers. #### 16. shell: `ShellSubprocess::defuse_array_buffer_unpins` takes `&mut self` It was an `unsafe fn(this: *mut Self)` that projected `stdout` and `stderr` with `&raw mut` and dereferenced each projection again. Nothing in it can re-enter the subprocess, and `Cmd::deinit` calls `&mut self` methods on the same pointer. Only the write through the `Rc<PipeReader>` keeps its block. #### 17. node:fs: `MkdirCtx::on_create_dir` takes `&mut self` The hook took `&self`, so the shell's `mkdir -pv` context, which appends each created directory to a `Vec<u8>`, had to carry a `*mut Vec<u8>`. The context is only ever used by the one `mkdir_recursive_impl` call it is made for, so that call takes `&mut Ctx` and `MkdirVerboseVTable` holds `&mut Vec<u8>`. The other implementor is `()`. #### 18. shell: the brace lexer's char iterators borrow their input `SrcAscii` kept the input as a `*const [u8]`, and `SrcUnicode` widened it to `&'static [u8]` with `from_raw_parts`. The lexer is built and consumed inside `tokenize(src)` and its output owns its tokens, so the input's lifetime is simply named, like on the types of the same names in `parse.rs`. #### 19. shell: `SmolList`'s heap list allocates through `&'a Bump` `SmolListAlloc` was a `NonNull<Bump>` with hand-written `Send`, `Sync` and `Allocator` impls, each method forwarding to `&Bump`'s own `Allocator` impl: a `&'a Bump` with the lifetime erased. Every caller passes the parser's `&'bump Bump` and every type that holds a `SmolList` already has `'arena`, so the list is `SmolList<'a, T, N>` over `Vec<T, &'a Bump>` and the wrapper is gone. The interpreter's `ast` module aliases it to `'static` like the rest of the AST. #### 20. shell: `CmdHandle` holds a plain `ParentRef`, and its users call `as_cmd_mut` `CmdHandle::cmd_mut` was an `unsafe fn` that made a `&mut Interpreter` with `assume_mut()` and widened the result to `&'static mut Cmd`, all to call `Interpreter::as_cmd_mut`, which takes `&self`. The handle was built with `ParentRef::from_raw_mut(interp.as_ctx_ptr())`, a pointer that comes from a `&Interpreter` and never had the write provenance that constructor asks for. The handle holds a `ParentRef<Interpreter>` made with `ParentRef::new(interp)`. `cmd_mut` is gone: its three callers keep a local copy of the handle, call `handle.interp.as_cmd_mut(handle.id)` and only take a `Yield` out of the borrow. `SpawnArgs::interp` and `PipeReader::interp` were a second, raw copy of the same pointer. The first is gone and the second is the handle's `ParentRef`, so `run_yield_with` loses its null test and its deref. That pointer was never null: there is one constructor of each type, and it is always given the interpreter. #### 21. shell: `OutputTask` is passed around as the `Box` it is `OutputTask::start` leaked its `Box` to a `*mut Self`. `write_err` and `write_out` pushed that pointer onto the builtin's `output_queue`, the builtin's `on_io_writer_chunk` popped it into three `unsafe fn`s, and the last of them rebuilt the `Box` to free it. `write_out` also got `&mut me.output` next to the pointer it came from. The IOWriter never sees the pointer (its child is the `Cmd`'s node id), so the task always has one owner: the running frame or the queue. The queue holds `Box<OutputTask<P>>`. `write_err` and `write_out` take the box and return `ControlFlow::Break(yield)` when they queued it or `Continue(box)` when they wrote synchronously, and `next`, `on_io_writer_chunk` and `deinit` are safe fns that take it by value. `mkdir`, `ls` and `touch` have their queue in `State::Exec`, the only state in which a task writes, and say `unreachable!()` for the others, as `Ls::next` does. Before, they skipped the push in another state. #### 22. shell: the brace AST borrows its arena, parent links are `Cell`s `ast::Group` had no lifetime: `bubble_up: *mut Group`, `Many(*mut [Atom])` and `variants: *mut [Group]` all pointed into the caller's arena. `expand_nested` was an `unsafe fn` over raw pointers with a long aliasing audit, and the safe `ast_to_json` dereferenced the raw slices of a value that could outlive its arena. The only writes after parsing are the two link fields, so they become `Cell<Option<&'a Group<'a>>>` and `Cell<Option<u16>>`, the slices become `&'a [..]`, and `Parser::parse` returns `Group<'a>`. `expand_nested` takes `&'a Group<'a>` and is safe. `braces.rs` has no `unsafe` left. #### 23. shell: `ls`, `touch` and `cp` reclaim a finished pool task where it arrives `run_from_main_thread` read `cmd` through the raw task pointer and passed the pointer on to `on_shell_*_task_done`, which rebuilt the `Box` one call later. It rebuilds the `Box` itself, as `mkdir` already does. `ls` and `touch` lend it to the callback as `&mut`, and `cp` hands it over because it ends up in `print_shell_cp_task`. On Windows `cp` parked tasks that failed with `EBUSY` as raw pointers in a `Vec` with a cursor, and turned each one back into a `Box` when its turn came. They wait in a `VecDeque<Box<ShellCpTask>>` and `pop_front` replaces the cursor. ### How did you verify your code works? - **Static checks on the final commits:** `cargo fmt --check`, `cargo clippy --workspace` (0 warnings), the source lints (194 pass), and `cargo check --workspace --release` for x86-64 Linux, x86-64 Windows, aarch64 macOS, x86-64 FreeBSD and aarch64 Android. - **Assembly:** the comparison above. - **Baseline for the tests below:** a debug build of `377e942b`, two commits before this PR's base. Neither of the two touches the shell. - **The leak that change 21 fixes has a test.** `shell-worker-terminate-leak.test.ts` gets a case that terminates a worker while `ls` waits on a full pipe. On the baseline build, LeakSanitizer reports the 32-byte `OutputTask` and its 1.9 MB of output, allocated in `OutputTask::<Ls>::new`. With this PR it reports nothing. The file's other 8 tests, with the new fd check of change 10, pass on both builds. - **All 44 files in `test/js/bun/shell/`**, on a debug build with ASAN of these commits: 42 files pass completely, 1,012 tests. The two others give the same result on the baseline build: `shell-load.test.ts` (1 test) and 7 of the 42 tests in `leak.test.ts` (`memleak_*`, 500 iterations with three full collections each) run into their own time limit of 90 and 100 seconds, which a debug build is too slow for. - **Seven of those files again with leak detection on** (`ls`, `mkdir`, `touch`, `cp`, `cat`, `brace` and `file-io`, with `detect_leaks=1` and `BUN_DESTRUCT_VM_ON_EXIT=1`): LeakSanitizer reports nothing, before or after. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…oven-sh#47278) ### What does this PR do? `new TextDecoder().decode(bytes)` loses data or kills the process when the result has no room, if the bytes are all ASCII. With one non-ASCII byte in the input the same call already throws. | all-ASCII input | 1.4.4 | `main` | this PR | | --- | --- | --- | --- | | the string cannot be allocated | returns `""` | `abort()` | throws `ERR_MEMORY_ALLOCATION_FAILED` | | 2^31 bytes or more | returns `""` | returns `""` | throws `ERR_STRING_TOO_LONG` | A reader that joins decoded chunks loses a piece in silence. Repro (Linux, `sh -c 'ulimit -v 4194304; bun min.js'`): ```js const bytes = new Uint8Array(32 << 20).fill(97), dec = new TextDecoder(), keep = []; try { for (;;) { const s = dec.decode(bytes); if (s.length !== bytes.length) { console.log("WRONG: copy " + keep.length + " has " + s.length + " characters, not " + bytes.length); break; } keep.push(s); } } catch (e) { console.log("caught " + e.message); } ``` ``` 1.4.4 WRONG: copy 32 has 0 characters, not 33554432 main panic(main thread): abort() called (exit 134) this PR caught Failed to allocate memory ``` #### Root cause `JsHeap::create_latin1_string` is infallible by its type, and both ASCII roads of `TextDecoder` (utf-8 and latin1/windows-1252) use it. - For 2 characters or more it reaches `JSC__JSString__createUninitializedLatin1`, which had `RELEASE_ASSERT(text)` on a `tryCreateUninitialized` that can return null. - Over the maximum length it falls back to `EncodedSlice::to_js`, where a null `WTF::String` becomes `""`. #### The fix - `bindings.cpp`: the `RELEASE_ASSERT` is gone. The function, now `JSC__JSString__tryCreateUninitializedLatin1`, returns an empty value when the allocation is refused. - `bindgen.rs`: new `JsHeap::try_create_latin1_string`, whose error is `ERR_STRING_TOO_LONG` or `ERR_MEMORY_ALLOCATION_FAILED`. It shares one copy routine with `create_latin1_string`, which stays infallible for its other callers (out of memory there now goes through `bun_core::out_of_memory()` rather than an assert). - `TextDecoder.rs`: the two ASCII call sites use it. **These two lines are the fix for the report.** - Lengths 0 and 1 no longer allocate at all: they are the VM's empty and single-character strings. #### The same crash in the rest of `decode()` Three more allocations in the same function aborted on the same refusal (`memory allocation of N bytes failed`, on 1.4.4 too). They now throw `ERR_MEMORY_ALLOCATION_FAILED`: - the join with bytes carried over from a `{stream: true}` chunk, - windows-1252 with a non-ASCII byte, - UTF-16. #### The decoder after such an error These error paths are new, so what they leave behind is new too: - "do not flush" is still set before decoding, as the spec has it. A flush that throws ends the stream, and the next call starts a new one with nothing carried into it. - A `{stream: true}` chunk that throws keeps what UTF-8 and UTF-16 carried from earlier chunks (partial character, BOM seen), so the same chunk can be decoded again. - The `encoding_rs` arm is unchanged. #### `#[inline(never)]` on `decode_utf16` `decode_with_encoding_rs` already has it. Inlined, the UTF-16 loop and the UTF-8 arm are one function of about 17 KB, and while working on this, edits to one kept changing the speed of the other (measured without LTO): a change to the UTF-16 capacity arithmetic alone cost a 16-byte UTF-8 decode 9%, and the UTF-16 loop ran 11 to 28% slower per unit depending on the code around it. Out of line, both effects are gone. #### Not in this PR - `new TextDecoderStream()` (utf-8, not fatal) has its own C++ road, `streamingUTF8Decode`. It already throws in these cases, but for an over-long chunk or a failed join it is a plain `RangeError: Out of memory` with no `code`, where the other streams now give the two codes above. - `node:path` passes strings of any length to the infallible `create_latin1_string`, so it still ends the process when out of memory. - A `fatal` UTF-16 decoder that rejects a `{stream: true}` chunk goes on from that chunk's last odd byte or lead surrogate, so it can garble or reject a valid next chunk: `[00 dc 41]` throws, then `[42 00]` gives U+4241. UTF-8 and the `encoding_rs` arm drop what was pending. `main` and 1.4.4 do the same as this PR here. ### How did you verify your code works? **New tests.** All three fail on `main` and on 1.4.4, and pass with this PR on a debug+ASAN build and on a release build. - `test/js/node/buffer-oom.test.ts`, "TextDecoder reports an allocation that fails": every encoding arm, `TextDecoderStream`, and the decoder's state afterwards. What refuses the allocation depends on the build, so that each CI lane runs what it can: - release on Linux: `ulimit -v`. Address space is used up with pages that are never written, so the child takes no memory to speak of and well under a second. - ASAN: `max_allocation_size_mb`, for what Rust allocates. The input is a `WebAssembly.Memory`, which is mapped rather than malloc'ed, so it can be longer than the cap. - debug: also `BUN_JSC_maxSingleAllocationSize`, for what WTF allocates. - `test/js/web/encoding/text-decoder.test.js`, "throws for more characters than a string can hold…": the synthetic limit, in a child process. **Existing tests.** On both builds: `test/js/web/encoding`, `test/js/node/fs/fs-oom.test.ts`. Debug only, where it is not skipped: `test/internal/bindgen.test.ts`. Release only: `test/js/node/path`, `test/js/node/string_decoder`, `test/js/web/streams/streams.test.js`. **By hand** - The repro above, and the same loop with 16 KiB chunks, `{ stream: true }`, latin1 and UTF-16: all end in `catch`. 2^31 − 1 bytes still decode. - `BUN_JSC_validateExceptionChecks=1` is clean on the new error paths. - 503,107 `decode()` calls in 96,000 rounds (91,337 of them distinct) on random chunkings of BOMs, split surrogates and partial sequences (4 encodings × `fatal` × `ignoreBOM` × random `stream`) answer the same on `main`, 1.4.4 and this PR, round for round. The debug build asserts that the UTF-16 reservation is never outgrown, and it did not fire in that run. - 17 cases of ordinary streaming behaviour print the same as Node 24. **Performance.** `main` against this PR at the same commit, both built from source with ThinLTO (no PGO). ns per `decode()`, median of 10 runs in random order, pinned to one core, `useConcurrentGC=0` to take GC thread scheduling out of the measurement. | input | `main` | this PR | | | --- | ---: | ---: | ---: | | utf-8 1 B | 28.1 | 16.9 | −40.0% | | utf-8 2 B | 30.5 | 30.6 | +0.3% | | utf-8 16 B | 35.4 | 32.2 | −8.9% | | utf-8 64 B | 32.8 | 33.0 | +0.3% | | utf-8 256 B | 44.4 | 45.4 | +2.3% | | utf-8 1 KiB | 107.0 | 109.8 | +2.6% | | utf-8 64 KiB | 5842.9 | 5803.6 | −0.7% | | latin1 16 B | 30.4 | 30.9 | +1.8% | | latin1 1 KiB | 97.7 | 99.2 | +1.5% | | utf-16le 16 B | 48.0 | 57.0 | **+19.0%** | | utf-16le 256 B | 225.9 | 232.8 | +3.1% | | utf-16le 4 KiB | 3024.2 | 3129.2 | +3.5% | | utf-16le 64 KiB | 48807.5 | 49419.2 | +1.3% | | utf-16be 4 KiB | 3605.3 | 3093.0 | −14.2% | - **Known cost:** 9 to 13 ns on every UTF-16 call, which shows on short input (a second run had 2 B at 40.7 → 53.7 and 16 B at 47.8 → 59.6). What I tried: - It is not the fallible allocation: with `Vec::with_capacity` put back in the same structure the cost is the same, and `Vec::try_with_capacity` recovers 1.4 ns. Nor is it the size of the allocation. - Moving the whole UTF-16 arm out of line, with the loop inlined in it and working on a local copy of the carried state, halves the cost (2 B +16%), but with ThinLTO the UTF-16LE loop is then 37% slower from 4 KiB up (without LTO it is 4% faster, and the UTF-16BE loop 16% slower). The loop in a function of its own is the one shape that was steady in both. - UTF-8 and latin1 are within the run-to-run spread of `main`. - Full collections over 100,000 decodes of 64 KiB: 604, against 602 on `main`. - `bench/snippets/text-decoder.mjs` shows no consistent difference: two runs of the same binary differ by more than the two builds do. - `node:path` is unchanged, except that an answer of one character (`path.normalize("/a/..")`) is about 35% faster (measured without LTO). --- _Generated by [Claude Code](https://claude.ai/code)_ Co-authored-by: Claude <noreply@anthropic.com>
### What does this PR do? Adds the `windowsHandle` option to `fs.createReadStream` and `fs.createWriteStream`, so a raw Win32 `HANDLE` can back a file stream, as in Node 24.21 and 26.8 (nodejs/node#63851). Bun ignored the option. With an empty path, the stream resolved the current directory and failed with `EISDIR`. - `streams.ts` validates the option in Node's check order. - A small native binding wraps the handle with `uv_open_osfhandle`. The stream then owns the fd and closes it. - `bun-types` declares the option, because `@types/node` does not have it yet. It also fixes two related problems with fd 0: - Destroying a file stream skipped the close when its fd was 0. It now checks `fd == null`, as Node does since nodejs/node#66131, and Node's updated `test-fs-stream-fs-options.js` is ported. A `windowsHandle` can be wrapped as fd 0 if the process has closed its stdin. - Bun destroys `process.stdin` after `'end'`. With the change above, that would have closed the process's real stdin, so `process.stdin` now keeps fd 0 open when it is destroyed. A test in `process-stdin.test.ts` covers this. One deliberate difference: the handle is wrapped after `start`/`end` validation, so a failure there leaves the caller owning it. Node wraps first and leaks the fd in that case. An invalid `highWaterMark` or `signal` still leaks it, as in Node. No docs change: Bun's docs do not describe `node:fs` stream options. ### How did you verify your code works? - Buildkite build #133777 passed on every platform. Both new test files ran on Windows x64 (Server 2019) and Windows 11 arm64: `test/js/node/fs/fs-stream-windows-handle.test.ts` and the ported `test/js/node/test/parallel/test-fs-stream-windows-handle.js`. - On an unpatched Bun, the ported Node test fails at its first assertion and the `EISDIR` reproduces. - On an unpatched Bun, the ported fd 0 test fails, as expected. - The type fixture in `test/integration/bun-types/fixture/fs.ts` accepts a `bigint` and rejects a number. Not yet measured: a pending pipe read holds a libuv threadpool worker, which may matter with several streams open at once. Node has the same design. --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: Dylan Conway <dylan.conway567@gmail.com>
…47382) ### What does this PR do? 36 separate changes, one commit each, in the test runner (`bun test`) and the CLI commands. They follow oven-sh#47291, oven-sh#47317 and oven-sh#47338 and work the same way: signatures name the type and the lifetime that the value already has, hand-written pointer code becomes plain ownership and borrows that the compiler checks, and what was never needed is deleted. Together: **128 fewer `unsafe` blocks and 2 fewer `unsafe fn`s**, +585/−1093 over 40 files. There is no new `unsafe`, no safe function became `unsafe`, and no wrapper type was added. One method is new (change 13). **[Where the compiled code changes](#where-the-compiled-code-changes) lists every place where the program does different work.** | # | Change | `unsafe` | Diff | |---|---|---|---| | 1 | console, test: format `Response`, `Request`, `Blob` and `BuildArtifact` through `as_class_ref` | −11 blocks | +19/−52, 3 files | | 2 | test: `Expect::get_signature` caches `&'static str` | −2 blocks | +5/−9, 1 file | | 3 | test: `print_test_line` keeps `&DescribeScope` in its scope stack | −2 blocks | +8/−13, 1 file | | 4 | test: `Coordinator::abort_queued_files` iterates with `iter_mut` | −2 blocks | +4/−15, 1 file | | 5 | test: `Order::generate_order_test` takes `&mut ExecutionEntry` | −4 blocks | +9/−21, 1 file | | 6 | test: the two `expect` functions called from C++ take references | −5 blocks, −2 `unsafe fn` | +12/−26, 1 file | | 7 | test: `TestRunner::filter_regex` is a `&RegularExpression` | −1 block | +5/−13, 4 files | | 8 | test: move `handle_timeout` from `Execution` to `BunTest` | −1 block | +8/−21, 2 files | | 9 | test: set the `WorkerPipe` parent pointers without `unsafe`, and drop two dead stores | −2 blocks | +5/−16, 2 files | | 10 | test: `BunTest::advance` logs the new phase after the body, without a guard | −1 block | +5/−7, 1 file | | 11 | test: `TestCommand::run` borrows the reporter's fields instead of aliasing it | −2 blocks | +15/−23, 2 files | | 12 | resolver: `DirEntryIterator::next` takes `&mut self` | −2 blocks | +11/−21, 3 files | | 13 | jsc: add `VirtualMachine::run_with_api_lock_mut`, which lends the VM to the closure | −12 blocks | +47/−91, 3 files | | 14 | test: `Execution` hands out and accepts `&mut ExecutionSequence` | −7 blocks | +26/−55, 3 files | | 15 | test: `Snapshots` borrows the runner's file list from its callers | −3 blocks | +24/−64, 5 files | | 16 | test: `Worker` gets what it needs of the `Coordinator` from its caller | −5 blocks | +43/−30, 2 files | | 17 | test: pass the parent `DescribeScope` as a reference | −3 blocks | +22/−33, 2 files | | 18 | test: `run_as_coordinator` takes `&VirtualMachine` | −4 blocks | +9/−19, 2 files | | 19 | test: the worker's scratch `Frame` is a field of `WorkerCommands` | −3 blocks | +39/−52, 2 files | | 20 | cli: `Cli::start` keeps the `&mut Log` that `MaybeUninit::write` returns | −1 block | +2/−4, 1 file | | 21 | completions: leak the decompressed package list instead of erasing its lifetime | −1 block | +5/−18, 2 files | | 22 | compile: declare `bun_internal_modules_header` as a `safe static` | −1 block | +3/−3, 1 file | | 23 | init: borrow package.json's object through a copy of its `StoreRef` | −2 blocks | +3/−6, 1 file | | 24 | create: `Example::fetch` uses its local URL instead of the `URL_` static | −3 blocks | +3/−10, 1 file | | 25 | run: the guards that restore `PATH` hold the `&mut Loader` | −4 blocks | +12/−20, 2 files | | 26 | run: `completions` parks its result map in the arena before borrowing the keys | −2 blocks | +8/−32, 1 file | | 27 | repl: take the API lock with `get_api_lock` and borrow the `-e` script | −3 blocks | +10/−32, 1 file | | 28 | run: `boot_standalone` takes `&'static Graph` | −2 blocks | +6/−22, 3 files | | 29 | run: the prefix writers of `--parallel` take the `ScriptConfig` | −1 block | +20/−34, 1 file | | 30 | upgrade: reach the progress node as `refresher.root` | −10 blocks | +33/−48, 1 file | | 31 | create: reach the progress node as `progress.root`, and let the guards hold `&mut Progress` | −5 blocks | +53/−76, 1 file | | 32 | run: `configure_env_for_run` returns the `&mut Transpiler` it initialized | −5 blocks | +28/−36, 4 files | | 33 | upgrade: take the file and directory names from the executable's path itself | −6 blocks | +8/−51, 1 file | | 34 | cli: the profiler and `--fetch-preconnect` options keep argv's `&'static [u8]` | −5 blocks | +22/−37, 3 files | | 35 | cli: the `-e` script is a `&'static [u8]` | −1 block | +17/−25, 7 files | | 36 | run: `RemoteImageDownload` borrows its URL and its channel | −4 blocks | +36/−58, 1 file | The commits can be reviewed one at a time. Worth knowing before reading on: - **One change that users can see (change 33).** `bun upgrade` of an executable that sits directly in the root directory (`/bun`, `C:\bun.exe`) used to fail at its last step, after the download. It now works as in any other directory. There is no test for it, because it needs a writable root directory. - **Change 13 adds a method**, `VirtualMachine::run_with_api_lock_mut(&mut self, f)`. It is `run_with_api_lock` for callers that hold the `&mut VirtualMachine` and need it inside the closure: it passes `self` on to `f`. It has no `unsafe` of its own. Both methods go through the trampoline that was there, with its three blocks untouched. - **Change 1 uses a safe accessor that already exists and has the `unsafe` inside** (`as_class_ref`), and so do one line each of changes 7 (`opaque_ref`) and 28 (`Graph::get_ref`). In change 1 the type changes too: the sites made a `&mut` where the callee takes `&self` and can re-enter JS. - **Four changes say `leak` where the memory was never freed** (2, 21, 34 and 35). The owner was a `static` or the process-wide context, which are never dropped. What was leaked is still reachable from the same place. - **Change 27 takes the same lock another way.** The REPL no longer holds an extra reference to the VM while it runs. It ends the process itself, so nothing ever released that reference in an order that mattered. ### Where the compiled code changes I compared the release assembly (x86-64 Linux, `--emit=asm`) of the four crates that this PR touches, before and after, function by function, with names demangled and labels normalised. | Crates | Functions | Result | |---|---|---| | `bun_resolver`, `bun_options_types` | 513 | Every function has the same instructions. | | `bun_jsc` | 1,707 | 2 functions differ. `VirtualMachine::destroy` is 10 instructions shorter, because `interactive_eval_script` has nothing to free (change 35). `load_macro_entry_point` has the same 221 instructions with one test two places earlier, and names the trampoline `hold_api_lock::call` (change 13). | | `bun_runtime` | 10,461 | 87 functions differ. 17 of them are unrelated code that calls a function by another name: 16 call a `RawVec::grow_one` that the compiler merged with the one of change 2's map, and 1 calls a closure that got another number. The other 70 belong to `bun test` and the CLI commands. | Of the 87 functions, 33 keep their number of instructions, 30 get shorter and 24 get longer. Over everything that differs, with the functions that exist on one side only, there are 655 fewer instructions, 117 fewer conditional branches, 106 fewer compares and 64 fewer calls. No function calls a panic path that the source did not have. | Change | What differs | Effect | |---|---|---| | 2 | The cached values have no drop code. `get_signature` goes from 377 to 339 instructions and the map's `grow` from 261 to 216. | Same allocations. | | 5 | Three loads move in front of the list code. | That code does not write them. | | 8 | The dead error branch and the address arithmetic go. `BunTest::run` goes from 1,603 to 1,593. | Same pushes. | | 9 | Two stores fewer per worker in `run_as_coordinator`. `Worker::start` goes from 449 to 437 (with change 16). | Both stores were overwritten before anything read them. | | 13 | The trampoline's instances have other names, and the closures get the VM as an argument. `Context` in `run_all_tests` is built inside the closure. | Same lock, same calls. | | 15 | A pointer argument replaces the load of the global runner and its `expect`. `parse_file` loses a pooled path buffer and a concatenation. `Expect::snapshot` goes from 1,721 to 1,630. `write_inline_snapshots` goes from 1,259 to 1,284 with 3 compares and 3 branches fewer: the compiler duplicates code that builds error messages. | Same snapshot files. | | 16 | `SpawnContext`, seven words, is written to the stack once per spawned worker: `spawn_worker` +16 and `run_pending_reaps` +20, moves and address arithmetic. `dispatch` goes from 123 to 112 and `shutdown` from 62 to 59. `live_workers` gets one `+1` after `Ok`, instead of `+1` before `watch_or_reap` and `−1` on failure. | Nothing reads the counter in between: an exit that arrives during `start` is only recorded and is counted later, from `drive`. | | 17 | `Option<&mut DescribeScope>` is one word where `Option<*mut _>` was two. `ExecutionEntry::create` goes from 161 to 156 and `append_hook` from 56 to 53. `generic_hook_impl` goes from 350 to 368: two cold `throw` calls are duplicated, with no compare or branch more. | Same scopes. | | 18 | The branch for "at most one worker" moves from `run_as_coordinator` to `exec`. Together with changes 9 and 12 they go from 2,938 to 2,907 and from 4,145 to 4,125. | Same runs. | | 19 | The frame is a field instead of a static. `WorkerCommands::send` (110 instructions, with `Channel::send` inlined) is gone, and `Channel::send` (96) is called. `worker_flush_aggregates` +5, `on_channel_frame` +2. | Same bytes on the pipe. | | 3, 14, 19 | `advance_sequence`, which has `print_test_line` and `worker_emit_test_done` inlined, goes from 2,405 to 2,439: 35 moves more, the same number of conditional branches, and 3 call sites that the compiler duplicates. | Once per finished test. | | 21 | `Table` is three words smaller. `build_table` goes from 157 to 146. | Same table. | | 24 | One store to a static, two `unwrap`s and a clone of the `URL` fewer. `Example::fetch` goes from 743 to 604 (with change 31). | Same request. | | 25 | The env pointer is loaded once instead of three times. `run_scripts_with_filter` goes from 1,474 to 1,455. | `PATH` is restored at the same place. | | 26 | The arena allocation runs before the keys are collected and sorted, not after. One instance of `completions` has 6 moves more, two others keep their length, and `bun_getcompletes` is 24 longer, with one call site duplicated. | Same completions. | | 27 | No C++ frame, no indirect call and no extra reference to the VM. `ReplCommand::exec` goes from 458 to 408. The thunk's 439 instructions are `ReplRunner::start`. | Same REPL. | | 29 | `ScriptConfig` grows by a word and `ProcessHandle` shrinks by one. The search loop goes: `run` goes from 1,865 to 1,830, and `add_script_configs` is 15 longer for the four stores. The compiler inlines differently: `write_line_with_prefix` (96) is inlined in `on_read_chunk` (+50), and `flush_pipe_buffer` (109) is a function of its own, so `maybe_finish` goes from 349 to 318. **With that call in between, the `handle.process.unwrap()` that was already there is no longer folded into the test at the top of `maybe_finish`: one compare more per finished script.** | Same lines with the same colors. | | 30, 33 | Five `expect`s, the pooled buffer, the copy and the NUL stores go. `_exec` goes from 3,622 to 3,607. | **An executable in the root directory can be upgraded.** | | 31 | One argument fewer in four functions. `CreateCommand::exec` goes from 3,567 to 3,544, `fetch_from_github` from 660 to 666 (moves and address arithmetic). | The guards run at the same places. | | 32 | The callers use the pointer that comes back instead of the address of their own slot: `configure_env_for_run_impl` +2 and `pack` +43, all moves. `BunxCommand::exec` is 11 longer, because the compiler now inlines the drop of one `Option<File>` there. | Once per command. | | 34, 35 | One `malloc` and one copy fewer per option value, for the `-e` script and for the REPL bootstrap. `arguments::parse` goes from 6,293 to 6,080 and `boot_main` from 663 to 650. `exec_node_repl` (69) is inlined in its two callers (+52 and +10). `leak()` does not shrink the buffer of a `${pid}` name or of a script from stdin, so one `realloc` goes. | Same options, same scripts. | | 36 | One allocation and one copy fewer per remote URL. The channel is dropped after the downloads, not before. `render_markdown_file` goes from 1,034 to 1,006. | Same downloads. | Changes 1, 4, 6, 7, 10, 11, 12, 20, 22, 23 and 28 ask for the same operations in the same order. Where they show up at all, it is in registers and in the order of blocks: `abort_queued_files` goes from 147 to 139, `enqueue_describe_or_test_callback` from 526 to 520 and `boot_standalone` keeps its 1,048. `print_as` of the `expect` formatter has one call site duplicated (+1, twice), and `Expect_readFlagsAndProcessPromise`, `TestCommand::run`, `assign_work` and `Coordinator::bail_out` are 1, 1, 1 and 4 instructions longer. Both log lines of change 10 are compiled out. What the comparison does not cover is code for other targets, such as the Windows arms of changes 16, 31 and 33. ### The changes in detail #### 1. console, test: format `Response`, `Request`, `Blob` and `BuildArtifact` through `as_class_ref` The two formatters downcast with `as_` and turned the pointer into `&mut T` (`&T` for `BuildArtifact`) by hand. All four `write_format` methods take `&self`, and those of `Response` and `Request` call back into JS, which can reach the same cell, so the `&mut` was wrong as well as unneeded. They now use `as_class_ref`, like the other branches of the two formatters. `body_mixin_get_blob` did the same to call `get_blob_without_call_frame(&self)`, and `BuildArtifact::write_format` for its source map. Both use it too. #### 2. test: `Expect::get_signature` caches `&'static str` The cache held `Box<str>` values that are never removed, and both return paths stretched a borrow of the box to `'static` by hand. The rendered strings are now leaked when they are made and the cache holds the `&'static str`, which is copied out. Same allocations, still reachable from the static map. #### 3. test: `print_test_line` keeps `&DescribeScope` in its scope stack The walk up the parent chain pushed raw pointers, dereferenced each one to find the next parent, and dereferenced it again in the two loops that print the names. It now makes the reference once, in the walk, and stores that. #### 4. test: `Coordinator::abort_queued_files` iterates with `iter_mut` It walked the worker slots through raw pointers because a caller's `&mut Worker` was said to be live across the call. That is not so: it is reached only from `reap_worker`, which takes a slot index, derives its `&mut Worker` again after each of these calls, and runs only from `drive` between event loop ticks. #### 5. test: `Order::generate_order_test` takes `&mut ExecutionEntry` It was a safe function with a `# Safety` section and a `NonNull` parameter that its only caller made from a `&mut`. It now takes the reference, reads the plain fields it needs, and only then turns it into the pointer that the entry list links. The list code writes only `next` and `failure_skip_past`, so reading the three counters earlier gives the same values. #### 6. test: the two `expect` functions called from C++ take references `Expect_readFlagsAndProcessPromise` and `ExpectCustomAsymmetricMatcher__execute` were `unsafe extern "C" fn`s with raw pointer parameters, dereferenced one by one. Rust never calls them. C++ passes the addresses of three separate stack variables, all initialised, which nothing else touches during the call, and the payload that sits inside the matcher's own cell, which is only ever read. None is null. The parameters are now references (`&mut` for the three, `&` for the payload) and the functions are safe. The ABI is the same. #### 7. test: `TestRunner::filter_regex` is a `&RegularExpression` `RegularExpression` is an opaque zero-sized handle and the C++ function behind `matches` already takes `&RegularExpression`, so `matches` needs only `&self`. With that the test runner can hold the `--test-name-pattern` regex as a reference, made once with `opaque_ref`, instead of a `NonNull` that it turned into `&mut` for every test. #### 8. test: move `handle_timeout` from `Execution` to `BunTest` `Execution::handle_timeout` computed the address of the `BunTest` that holds it to push a result on the queue there. Its one caller has the `BunTest`, so the method now lives on it and uses the two fields directly. It never failed, so it returns nothing and the caller's error branch is gone. #### 9. test: set the `WorkerPipe` parent pointers without `unsafe`, and drop two dead stores `Worker::start` turned `&mut self.out` and `&mut self.err` into raw pointers and called `set_parent` through them. `set_parent` is a safe fn, so the calls go through the fields, as in `multi_run.rs`. `run_as_coordinator` stored each new worker's address in its two pipes through a raw pointer. Both callers of `Worker::start` replace the pipes, with a fresh back-pointer, right before the call, and only a started reader reads it, so the stores are gone. #### 10. test: `BunTest::advance` logs the new phase after the body, without a guard The debug log of the phase that `advance` ends in ran from a `defer!` that read `self.phase` through a raw pointer. The body is now `advance_inner`, and `advance` logs, calls it, logs `self.phase` and returns the result. Both log lines are compiled out of release builds. #### 11. test: `TestCommand::run` borrows the reporter's fields instead of aliasing it `CurrentFile::set` took the whole `&mut CommandLineReporter` although it is itself a field of it, so the call made two overlapping `&mut` from a raw pointer. It reads one flag and `reporters`, so it now takes those. The guard that restores `jest.only` wrote through another raw pointer. It now holds the `&mut` and lends it to the rest of the function. #### 12. resolver: `DirEntryIterator::next` takes `&mut self` The only implementor with state, the test file `Scanner`, has to change itself for each entry, so it hid a `*mut Scanner` in a newtype to get past `&self`. `readdir` calls the hook from one place and holds nothing else of it, so the hook can take `&mut self` and `Scanner` implements the trait itself. Also copies `StringOrTinyString`, which is `Copy`, instead of `ptr::read`. #### 13. jsc: add `VirtualMachine::run_with_api_lock_mut`, which lends the VM to the closure `run_with_api_lock(&self, f)` borrows the VM for the whole call, but reads only `jsc_vm` before `f` runs. Callers in `bun test` that hold a `&mut VirtualMachine` and need it inside `f` therefore made a raw pointer first and dereferenced it in the closure, writing to the VM while the `&self` argument was live. The new method takes `&mut self` and passes it on to `f`. Both methods share the existing trampoline, now the private `hold_api_lock`. With it `run_as_worker` and `WorkerLoop::begin` take `&mut VirtualMachine`, and `WorkerLoop` no longer stores a second pointer to the VM. #### 14. test: `Execution` hands out and accepts `&mut ExecutionSequence` `get_current_and_valid_execution_sequence` built a `&mut` to the sequence and to its group, erased both to `NonNull`, and every caller turned the first back into a reference on the next line. `advance_sequence` did the same with its parameter, and `step_group` and `step_sequence_one` with pointers they had just made. These now pass the references themselves: `groups` and `sequences` are separate fields, so the pair is an ordinary split borrow of `&mut self`. Whatever is held across `step_sequence` or `run_test_callback`, which run JS that re-enters the runner, is still a raw pointer, made just before the call. #### 15. test: `Snapshots` borrows the runner's file list from its callers `Snapshots` is the `snapshots` field of `TestRunner` and needs the `files` field next to it. Three methods reached it by reading the global runner pointer and dereferencing it. Every caller holds the `TestRunner`, so `get_or_put`, `get_snapshot_file` and `write_inline_snapshots` now take `&FileList`, and the callers pass `&runner.files`. The two callers that hold the reporter use `reporter.jest`, which is the runner, instead of the global pointer to it. `parse_file` used the list only to build the path of the snapshot file again. Its one caller has just built the same path, so it takes that instead. #### 16. test: `Worker` gets what it needs of the `Coordinator` from its caller `dispatch`, `shutdown` and `start` reached the `Coordinator` through the worker's raw back-pointer: for the scratch `frame`, for the spawn parameters and to count `live_workers`. All three are called only from `Coordinator` methods that hold `&mut self`, so those writes went behind the back of a live `&mut Coordinator`. `dispatch` and `shutdown` now take `&mut Frame`. `start` takes a `SpawnContext` with the VM, the working directory, argv and the environments, all of which live outside the `Coordinator` struct, and its two callers add to `live_workers` when it succeeds. Nothing reads the counter while `start` runs. #### 17. test: pass the parent `DescribeScope` as a reference `BaseScope::init` and `ExecutionEntry::create` took the parent as a raw pointer and dereferenced it to inherit `concurrent` and `mode`. `propagate` dereferenced the pointer it had just stored. Every caller is a `&mut self` method of the parent, or passes `None`, so they now take `&mut DescribeScope` and the raw back-pointer is made from it when it is stored. `Collection::init` was a safe function that dereferenced a `*mut BunTestRoot` only to take the address of its `hook_scope`. `enter_file` now takes that address itself and passes it down. #### 18. test: `run_as_coordinator` takes `&VirtualMachine` It was a safe function with a `*mut VirtualMachine` parameter that it dereferenced in six places. It needs `&mut` only to run the files sequentially when there is at most one effective worker. It now returns `false` in that case and `exec`, which holds the `&mut`, runs them. For everything else a shared reference is enough: `Coordinator::vm` already is one. #### 19. test: the worker's scratch `Frame` is a field of `WorkerCommands` The frame that a `--parallel` worker encodes its messages in was a mutable static, turned into `&mut` in three places. `WorkerLoop::begin` kept its `&mut` across the run of a test file, during which `worker_emit_test_done` made a second one. Everyone who sends a frame has the `WorkerCommands`, so the frame now lives there, and `emit` begins, fills and sends it in one call. #### 20. cli: `Cli::start` keeps the `&mut Log` that `MaybeUninit::write` returns `start` wrote the process-wide `Log` into its static slot, dropped the reference that `write` returns, and a few lines later took the same reference again with `assume_init_mut()` in a second `unsafe` block. It now binds the result of `write`. #### 21. completions: leak the decompressed package list instead of erasing its lifetime `build_table` made a `&'static [u8]` over the decompressed `Vec` with `from_raw_parts` and then stored the `Vec` in a field of the table only to keep it alive. The table lives in a `static` `Once`, which is written once and never dropped, so the buffer was never freed. `Vec::leak` says that and returns the `&'static` slice. The field is gone. The generator emits the same text. #### 22. compile: declare `bun_internal_modules_header` as a `safe static` The symbol is the header of a read-only section that is linked into every build, and its fields are plain integers and bytes. Reading it has no precondition, so the declaration says so and the read needs no `unsafe` block. #### 23. init: borrow package.json's object through a copy of its `StoreRef` Two functions turned `fields.object` into a `&mut E::Object` with `&mut *ptr.as_ptr()`, so that `fields` stayed free for other uses. `StoreRef` is `Copy` and implements `DerefMut`. A local copy gives the same `&mut` and leaves `fields` just as free. #### 24. create: `Example::fetch` uses its local URL instead of the `URL_` static `fetch` parsed the registry URL, stored it in a `static` with its lifetime erased, and read it back twice, each time in an `unsafe` block with an `unwrap`, the second time with a clone. Nothing else used the static. The URL borrows from a static buffer and `AsyncHTTP::init_sync` is generic in the lifetime, so the local is passed directly, as `fetch_from_github` already does. #### 25. run: the guards that restore `PATH` hold the `&mut Loader` `ProcessHandle::start` in `--filter` and in `--parallel`/`--sequential` runs dereferenced the raw env pointer three times: to set `PATH`, in the guard that restores it, and for `with_package_script`. The guard now owns the first `&mut Loader` and lends it to `with_package_script`, so one dereference is left. The guard still runs at the same place. #### 26. run: `completions` parks its result map in the arena before borrowing the keys The keys were turned into `&'static [u8]` with `from_raw_parts`, and only afterwards was the map that owns them moved into the process-lifetime arena. Moving it first gives a `&'static` map, whose keys can simply be borrowed. The descriptions come out of a map of `&'static [u8]`, so their `Vec` is declared with that type and needs no pointer cast. #### 27. repl: take the API lock with `get_api_lock` and borrow the `-e` script `boot_repl_vm` ran the REPL through the deprecated `hold_api_lock`, which needs an `extern "C"` thunk and a `*mut c_void` context. `get_api_lock` takes the same `JSLock` and lets the runner be called directly, as `Run::start` does. It does not add a reference to the VM, which made no difference here: the runner ends the process, and the teardown at exit releases every outstanding reference. Without the thunk, `ReplRunner::start` can say so: it returns `!`. The `-e` script was borrowed from `ctx` through a raw pointer. `ctx` is not used after the runner is built, so a plain borrow does. #### 28. run: `boot_standalone` takes `&'static Graph` Both `boot_standalone` functions held the embedded module graph as `&mut Graph` and only read it. To hand it to the VM as `&'static dyn StandaloneModuleGraph`, the second one erased the borrow through a raw pointer cast. The graph lives in a process-wide static, so the one place that dereferences the raw pointer now makes a `&'static Graph`, and the trait object is an ordinary coercion. There is also no `&mut Graph` left alive while `Graph::get_ref()` hands out shared references to the same object. `run_compiled::graph` made the same cast. It calls `Graph::get_ref()`. #### 29. run: the prefix writers of `--parallel` take the `ScriptConfig` `write_prefix`, `write_line_with_prefix` and `flush_pipe_buffer` took the whole `&ProcessHandle` and read `config` and `color_idx` from it. `maybe_finish` also had to pass `&mut handle.stdout_reader`, which is part of the same handle, so it made both references from a raw pointer. `color_idx` is the index of the script's group. It moves into `ScriptConfig`, where `add_script_configs` knows it, so the loop that searched the groups for each handle is gone and the three writers take `&ScriptConfig`. That leaves a plain field borrow in `maybe_finish`. `read_chunk`, whose `&mut PipeReader` is also part of the handle, copies `config` through the back pointer instead of making a `&ProcessHandle`. #### 30. upgrade: reach the progress node as `refresher.root` `Progress::start` returns `&mut self.root`. To keep that node and the `Progress` usable side by side, `_exec` held both as raw pointers and dereferenced them in ten `unsafe` blocks, and `get_latest_version` got two `&mut` parameters that overlap. `root` is a public field, so the node parameter is dropped, the leaked `Progress` is held as `&mut`, and the node is named `refresher.root` where it is needed. #### 31. create: reach the progress node as `progress.root`, and let the guards hold `&mut Progress` `exec` kept `&mut progress.root` alive next to `progress` itself by way of a raw pointer, and passed both to `fetch`, `fetch_from_github`, `file_copier_copy` and `exit_copy_failed` as two overlapping `&mut` parameters. Those take only the `Progress` now and name `root` where they need the node. The guard that refreshes on exit and the guards that count a copied file held raw pointers so as not to borrow. They are `scopeguard::guard`s that own the `&mut Progress` and lend it back, so `RefreshOnDrop` is gone. They run at the same places as before. #### 32. run: `configure_env_for_run` returns the `&mut Transpiler` it initialized The function fills a caller-provided `MaybeUninit<Transpiler>`. It threw away the `&mut Transpiler` that `MaybeUninit::write` returns and took it again with `assume_init_mut()`, and so did each caller after the call. It now keeps the reference and returns it next to the `DirInfo`, with the lifetime of the slot. The slot stays where it was, so the struct is not moved. `--filter` still takes the `Transpiler` out of its slot by value, so that it is dropped there as before. #### 33. upgrade: take the file and directory names from the executable's path itself `_exec` copied its own NUL-terminated path into a pooled buffer through a raw pointer and wrote a NUL over the separator, to get a NUL-terminated directory name and file name. The directory name is only ever used as bytes, so `dirname`'s slice does. The file name is the end of the path, which already ends in a NUL: it is the base name of the path with its NUL. The buffer, the copy and the writes into it, two of them only on Windows, are gone. For an executable directly in the root directory (`/bun`, `C:\bun.exe`) there is no separator after the directory name, and the NUL went over the first byte of the file name. The upgrade then failed at its last step, after the download. That no longer happens. #### 34. cli: the profiler and `--fetch-preconnect` options keep argv's `&'static [u8]` The argument parser returns option values as `&'static [u8]`. The names and directories of `--cpu-prof` and `--heap-prof` and the URLs of `--fetch-preconnect` were copied into `Box<[u8]>` fields, and `run_command` then erased the borrow of each box back to `'static` in an `unsafe` block. The fields now hold the `&'static [u8]`, so there is neither a copy nor an erasure. A `--cpu-prof-name` with `${pid}` in it still builds a new string. It is leaked. It used to sit in the process-wide context, which is never dropped. #### 35. cli: the `-e` script is a `&'static [u8]` `Eval::script` was a `Box<[u8]>`. `-e`, `-p` and `node --port` copied argv's `&'static [u8]` into it, `--interactive` copied the embedded REPL bootstrap into it, and `boot` then erased the borrow of the box back to `'static` for the VM's `Source`. The field, `Eval::interactive_script` and `ModuleLoader::interactive_eval_script` now hold `&'static [u8]`, so the copies and the `unsafe` block are gone. The two places that read the script from stdin leak the buffer they read. It used to sit in the process-wide context, which is never dropped. #### 36. run: `RemoteImageDownload` borrows its URL and its channel Each download owned its URL and held an `AsyncHTTP<'static>` that pointed into it, so the struct referred to itself. It had to be built field by field in an uninitialized `Box`, with the URL's lifetime erased and the `Box` cast at the end, in four `unsafe` blocks. The channel was a raw pointer. The URLs already sit in `collector.urls`, which outlives the downloads, and so does the channel once it is declared first. The struct borrows both, its `AsyncHTTP` has that lifetime, and it is written into the `Box` in one piece with `Box::write`. The address for the callback is still taken from the uninitialized `Box`. `remote_urls` no longer copies each URL. ### How did you verify your code works? - **Static checks on the final commits, on Linux:** `cargo fmt --check`, `cargo clippy --workspace` (0 warnings), the source lints (194 pass) and `cargo check --workspace --release`. - **Assembly:** the comparison above. - **Change 33, by hand.** I linked the debug build to `/bun-debug` and ran `bun upgrade --stable` against a release served locally, set up like the one in `bun-upgrade.test.ts`. On the base it downloads, prints `error: Failed to move new version of Bun to /bun-debug to ENOENT` and exits 1. With this PR it prints `Upgraded.`, exits 0, and `/bun-debug` is the new file. In an ordinary directory both builds upgrade. - **133 test files**, on a debug build with ASAN of these commits and, where something failed, on a debug build of the base: all of `test/cli/test/` except the `test-aot` files, all of `test/js/bun/test/`, and the files for what the CLI changes touch (`run_command`, `run-eval`, `multi-run`, `filter-workspace`, `cpu-prof`, `heap-prof`, `markdown-entrypoint`, `env`, `bun-pack`, `bunx`, `bun-create`, `bun-upgrade`, `init`, `create-jsx`, `repl`, `bundler_compile`, `bun-build-api`, `inspect`, `fetch-preconnect`, `bun.test.ts`, and `js-plugins` and `config-files` of `bun lint`, among others). - 125 files pass completely, 4,071 tests. `spyMatchers.test.ts` ran with `immutable` 5.1.4, not the 5.1.3 that `test/package.json` pins. - 6 files fail in the same tests on both builds: `bun-test.test.ts` (2 of 109), `create-jsx.test.ts` (6 of 36), `parallel.test.ts` (1 of 47, which gives a worker 200 ms to start), `pretty-format-overflow.test.ts` (1 of 1), `snapshot.test.ts` (1 of 65) and `bun-build-api.test.ts` (1 of 91, its time limit of 180 seconds). - `huge-failure-message.test.ts` has a time limit of 300 seconds. Its script takes 298.5 seconds on the base build and 298.6 with this PR, side by side, with the same output. - 7 tests of `bundler_compile.test.ts` need packages that I did not have. Its other 78 tests pass. - **Paths that no test reaches, by hand on both builds, with the same output:** `--cpu-prof-name 'x${pid}.cpuprofile'` with `--cpu-prof-dir`, `--heap-prof-name` with `--heap-prof-dir`, `bun getcompletes a reac` (change 21), `bun run -` with a script on stdin and `bun -p`. - **After the tests had run** I removed one `#[inline]` attribute that an earlier form of change 10 had added. The static checks ran again on the final commits. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
…ven-sh#47393) ### What does this PR do? 32 separate changes, one commit each: 20 in the dev server (`src/runtime/bake`) and 12 in the code behind `Bun.Transpiler`, `Bun.build`, `Bun.Image`, `Bun.Terminal` and a few other `Bun.*` APIs. Commit 33 is a follow-up to change 2, for a finding of the `mordant` check. They follow oven-sh#47291, oven-sh#47317, oven-sh#47338 and oven-sh#47382 and work the same way: signatures name the type and the lifetime that the value already has, hand-written pointer code becomes plain ownership and borrows that the compiler checks, and what was never needed is deleted. Together: **146 fewer `unsafe` blocks and 3 fewer `unsafe fn`s**, +833/−1426 over 45 files. There is no new `unsafe`, no safe function became `unsafe`, and nothing that is new has `unsafe` inside. **[Where the compiled code changes](#where-the-compiled-code-changes) lists every place where the program does different work.** | # | Change | `unsafe` | Diff | |---|---|---|---| | 1 | bake: `Entry::render_json` takes no `DevServer` | −2 blocks | +9/−58, 4 files | | 2 | bake: `start_async_bundle` keeps its `ASTMemoryAllocator` on the stack | −4 blocks | +6/−42, 2 files | | 3 | bake: `get_or_put_route_bundle` holds its index slot as `&Cell` | −2 blocks | +5/−10, 1 file | | 4 | bake: `on_file_update` borrows the watchlist's columns | −5 blocks | +6/−25, 2 files | | 5 | bake: `process_file_list` borrows the dependency across `resolve` | −1 block | +6/−12, 1 file | | 6 | bake: borrow the arena's `Framework` view instead of re-deriving it | −2 blocks | +6/−10, 2 files | | 7 | bake: `HmrSocket` reads the `DevServer` through its `BackRef` | −2 blocks | +8/−13, 1 file | | 8 | bake: `PerThread::init` takes the `BackRef` its caller holds | −1 block | +6/−11, 2 files | | 9 | bake: remove the two empty `emit_*visualizer_message_if_needed` functions | −2 blocks | +0/−14, 4 files | | 10 | bake: `finalize_bundle` runs its cleanup after the body, not from guards | −2 blocks | +24/−47, 3 files | | 11 | bake: `finalize_bundle_inner` borrows fields instead of going through `dev_ptr` | −6 blocks | +36/−56, 3 files | | 12 | bake: `BunFrontendDevServerAgent` wraps a reference to the debugger's slot | −1 block | +22/−30, 3 files | | 13 | bake: `RequestEnsureRouteBundledCtx` is passed `dev` and borrows the request | −6 blocks | +43/−75, 2 files | | 14 | bake: `take_js_bundle` is given `root` and the configuration hash key | −5 blocks | +55/−52, 4 files | | 15 | bake: trace a route and build its client bundle from `Copy` values | −16 blocks | +154/−174, 5 files | | 16 | bake: `compute_arguments_for_framework_request` takes `&mut self` | −20 blocks, −1 `unsafe fn` | +55/−144, 2 files | | 17 | bake: `server_transpiler` and `client_transpiler` are `ManuallyDrop`, not `MaybeUninit` | −17 blocks | +84/−167, 7 files | | 18 | bake: `generate_html_payload` takes the fields it uses | −6 blocks | +42/−51, 3 files | | 19 | bake: `init_transpiler` returns the transpiler it wrote | −5 blocks | +29/−39, 3 files | | 20 | bake: `Assets::replace_path` becomes `DevServer::replace_asset_path` | −1 block | +42/−40, 5 files | | 21 | bundler: `create_and_configure_transpiler` reborrows the transpiler instead of going through a raw pointer | −2 blocks | +2/−9, 1 file | | 22 | api: pass the `*mut JSObject` straight to `JSPropertyIterator::init` | −4 blocks | +4/−12, 2 files | | 23 | transpiler: `Bun.Transpiler` stops swapping `Transpiler.arena` on every call | −4 blocks | +24/−53, 1 file | | 24 | bundler: `JSBundleCompletionTask` binds what it matched instead of matching again with `unreachable_unchecked` | −4 blocks | +27/−51, 1 file | | 25 | image: build the encoders' `NonNull<[u8]>` from the null test that is already there | −6 blocks | +27/−27, 3 files | | 26 | image: `codecs::encode` takes the ICC profile as `Option<&[u8]>` | −1 block | +16/−23, 2 files | | 27 | terminal: `Terminal::open` owns the first ref as a `RefPtr` from the start | −2 blocks | +7/−14, 1 file | | 28 | transpiler: `Bun.Transpiler`'s guards borrow the transpiler, and `Transpiler::print` takes `&self` | −3 blocks | +37/−71, 2 files | | 29 | transpiler: `JSTranspiler`'s constructor and `Drop` use the exclusive access they have | −3 blocks | +12/−19, 2 files | | 30 | api: keep a `NonNull` a `NonNull` instead of going raw and back with `new_unchecked` | −3 blocks | +17/−27, 4 files | | 31 | api: four functions exported to C++ are typed like the ones next to them | −4 blocks, −2 `unsafe fn` | +10/−29, 3 files | | 32 | runtime: drop four small `unsafe` blocks that a safe form replaces | −4 blocks | +24/−33, 4 files | | 33 | bake: `start_async_bundle` makes the arena's `&'static` reference once | 0 | +4/−4, 1 file | The commits can be reviewed one at a time. Worth knowing before reading on: - **Nothing that users can see changes.** There is no bug fix in this PR, so it adds no test. - **Most of the dev server's blocks had one cause.** A function took `&mut self` (or went from a field back to the `DevServer` that contains it) and was also handed a reference to something inside `self`, so one of the two became a raw pointer. The changes give such a function what it reads: the fields it works on (14, 17, 18, 20), a `Copy` index instead of a reference (15, 16), or `dev` as a parameter instead of a stored pointer (13). - **What is new**, all of it without `unsafe`: - `JsCell::as_mut(&mut self)` (change 29), which is `UnsafeCell::get_mut`. - `TraceRoot` (15), a `Copy` enum of two indices. - `DevRouteScan` (17), which borrows three fields of the dev server for the router's scan, like `RouteScan` of the production build. - Four functions that hold the body of a method that needed all of `self`: `relative_path(root, ..)` (14), `BunFrontendDevServerAgent::of(vm)` (11), `GraphTraceState::new` (18) and `finalize_bundle_inner` (10). The first three methods forward to them. - **Two changes replace guards by code after the body** (4 and 10). `panic = "abort"` is set for every profile, so "after the body" covers the same exits as a guard, as long as the body has no early return past it. In change 10 the body is a function of its own for that reason. - **Change 17 keeps the point where the two transpilers are dropped**, which is why they are `ManuallyDrop` and not plain fields. - **Change 32 declares `HMAC_CTX_init` as a `safe fn`** that takes `&mut MaybeUninit<HMAC_CTX>`. It only writes to its argument. ### Where the compiled code changes I compared the release assembly (x86-64 Linux, `--emit=asm`) of the six crates that this PR touches and of `bun_jsc`, which calls a function whose signature changes, before and after, function by function, with names demangled and labels normalised. Commit 33 came after this comparison; it hands `BundleV2::init` the same two addresses as before. | Crates | Functions | Result | |---|---|---| | `bun_ast`, `bun_boringssl_sys`, `bun_bundler`, `bun_ptr`, `bun_watcher` | 2,422 | Every function has the same instructions. `bun_ast` loses `release_ast_state` (change 2). | | `bun_jsc` (not edited, it calls `Transpiler::print_with_source_map`) | 1,707 | Every function has the same instructions. | | `bun_runtime` | 10,462 | 138 functions differ. 80 of them have the same instructions with other constants: the fields of `DevServer` have other offsets (change 17), and `EncodeOptions` is 10 bytes instead of 32 (change 26). | Of the other 58 functions, 10 keep their number of instructions, 31 get shorter and 17 get longer. 13 functions exist only before and 10 only after: they were renamed, moved, inlined or removed. Over everything that differs, with the functions that exist on one side only, there are 316 fewer instructions, 22 fewer conditional branches, 14 fewer compares and 23 fewer calls. **Checks.** In these functions the call sites of panic functions go from 858 to 856. Three functions have more of them than before: `on_framework_request_with_bundle` (+2), `ensure_route_is_bundled` (+1) and `serve_client_script` (+1). In all three the number of checks on a path stays the same or goes down. They looked a route bundle up through `route_bundle_ptr`, whose one panic location the compiler could share between the inlined copies. Now each lookup is an index expression on a line of its own, with a panic location of its own. | Change | What differs | Effect | |---|---|---| | 1 | The stores of `ref_count` and the guard go. `Drop for DevServer` goes from 358 to 322 (with change 17). | Nothing read the stores. | | 2 | One arena allocation fewer per bundle. The two calls of `release_ast_state` are the drop of the local. `start_async_bundle` goes from 691 to 660 (with change 17). | The state is released at the same exits. | | 4 | `on_file_update` goes from 308 to 302. | `flush_evictions` runs at the same point. | | 10 | `finalize_bundle` (5,951) and the drop code of its two guards (180 and 52) are now `finalize_bundle_inner` (5,908, with changes 1 and 11) and a wrapper, which the compiler inlines in the dispatch thunk (8 to 197). | The cleanup runs after every exit of the body. | | 11 | `BunFrontendDevServerAgent::of` is a function of 19 instructions. `make_array_for_server_components_patch` is 1 longer. | Same messages. | | 13 | No store of `dev` in the context: `on_defer` goes from 424 to 417, and `on_request` and `html_bundle::Route::on_any_request` are 2 shorter. `defer_request` picks the list right before it links the node, not before it saves the request. | Same list. | | 14 | `root` arrives in two registers instead of being loaded from `self`. `relative_path` goes from 287 to 294, and its callers get longer: `take_js_bundle_to_list` from 924 to 929 and its closure from 79 to 83, `ErrorReportRequest::on_body` +4, `insert_failure` (twice) and `report_pages_hidden_by_server_routes` +1. In `take_js_bundle_to_list` the compiler duplicates the capacity test of one `Vec` append. One `malloc`, one copy and one `free` per client bundle go, those of the entry point's path. | Same bundle text. | | 15 | The test "framework route or HTML route" moves from `trace_all_route_imports` (369 to 357) to its callers. `ensure_route_is_bundled` goes from 602 to 622: it has that test, and the call for an HTML route. `serve_client_script` goes from 979 to 966 (with change 14), with 7 conditional branches fewer. | A request for a cached client bundle makes one bounds check less. | | 16 | `on_framework_request_with_bundle`, which has `compute_arguments_for_framework_request` inlined, goes from 1,572 to 1,599. A request whose arguments are all cached makes 1 bounds check on `route_bundles` instead of 2. One that has to build the list of styles makes 2 instead of 3, with 2 tests of the route's kind as before. The function gets longer for two reasons: the second lookup has its own two panic calls, and **the test in front of the drop of one `Strong`, which the 8 error exits shared, is now duplicated at each of them (+7 compares, +7 branches in the listing, one per path as before).** | Same arguments. | | 17 | Apart from the offsets: `scan_initial_routes` goes from 81 to 91, because it writes three references to the stack. The three handler functions go from 35, 116 and 14 to 33, 119 and 14. `init` goes from 1,712 to 1,711. | Once at startup. | | 18 | `on_html_request_with_bundle` goes from 693 to 696 and calls `trace_imports` itself (change 15). | Same page. | | 19 | `init_transpiler` goes from 595 to 591. `build_with_vm` goes from 2,539 to 2,544, with 2 compares and 2 branches fewer (with changes 6 and 8). | Once per production build. | | 20 | `Assets::replace_path` (514) is `DevServer::replace_asset_path` (516). Its dispatch thunk goes from 75 to 74. | Same assets. | | 22 | `Config::from_js` +2 and the `Bun.Transpiler` constructor +1 (with change 29), moves. | Same options. | | 23, 28 | The stores that swapped the arena and put it back go. `scan` goes from 528 to 509, `transformSync` from 607 to 585, the guard's `drop` from 32 to 26. `scanImports` goes from 418 to 419 and `TransformTask::run` keeps its 361. In `transformSync` the compiler duplicates two tests and one call site of `mi_heap_destroy`. | Same output. | | 24 | The `Vec<OutputFile>` is not moved out of the result and back, and the code that dropped it on the way goes. `on_complete` goes from 2,611 to 2,599. | Same result objects. | | 25, 26 | `PipelineTask::run` goes from 6,783 to 6,753, `Image::set_format` from 286 to 258, `schedule` from 573 to 564 and `write` from 356 to 310. Seven more functions that copy the options are 2 to 4 shorter, and five keep their length. **In `write`, the compiler no longer folds "can this format be written" into the lookup of the file extension. It used to drop `bmp`, `gif`, `tif` and `tiff` from the lookup, because they end like an unknown extension. Now it looks up all 11 extensions and tests the format afterwards: up to 4 compares more per `write()` that takes its format from the path.** The source of that code is not touched. | Same files. | | 27 | The three failure paths share one call of `RefCount::deref`, where each had the decrement and the destructor inline. `fail_reader_start` (12) is inlined. `Terminal::open` goes from 427 to 411. On the failure paths the error value is freed before the ref is released, not after. | On the path of a terminal that opens, only the names of two registers differ. | | 29 | `Drop for JSTranspiler` goes from 15 to 8 and the drop of its `Box` from 43 to 39. | The messages are freed by the drop of `config`. | | 30 | One closure in the completion vtable has another signature, and so another name. | Same 9 instructions. | | 32 | The `abort` hook calls `std::process::abort`. `metadata` goes from 236 to 235. | The hook is only reachable from `bun:internal-for-testing`. | Changes 3, 5, 6, 7, 8, 9, 12, 21 and 31 ask for the same operations in the same order. Where they show up at all, it is in registers and in the order of instructions: `process_file_list` goes from 1,213 to 1,212, `index_failures` from 524 to 525, and `HotReloadEvent::run`, `on_js_request` and `start_next_bundle_if_present` keep their length. The comparison covers x86-64 Linux only. ### The changes in detail #### 1. bake: `Entry::render_json` takes no `DevServer` `render_json` took `&mut DevServer` and never used it. To pass it, the `.js.map` handler turned its borrow of the entry into a raw pointer and back. Without the parameter the handler borrows the entry. `finalize_bundle` renders a temporary `Entry`, a local, and had a guard that reached it through a raw pointer to clear `files` and `paths` at the end of the block. The local is dropped at the same brace, which frees the same two fields, so the guard is gone, and with it `Entry::deinit`, which nothing else called. `deinit` asserted that `ref_count` is 0. Two writes of that field were only there for the assert: the 1 in the temporary entry, and the zeroing of every entry in `Drop for DevServer`. Nothing reads either, so both are gone. The loop in `drop` keeps its own assert. #### 2. bake: `start_async_bundle` keeps its `ASTMemoryAllocator` on the stack The allocator was allocated in the bundle's arena and kept as a raw pointer. Because an arena value never runs `Drop`, a hand-written guard released its `AstAllocState` through that pointer, and entering the scope and taking the state went through it too. Nothing needs the allocator after the function returns: the nodes live in the arena and in the `AstAllocState`, and leaving the scope restores the thread-local pointer. It is a local now, like at the other callers of `borrowing`, and its own `Drop` releases the state on the error paths. `release_ast_state` has no caller left and is removed. `BundleV2::init`'s `alloc` parameter takes any lifetime, so it gets `&heap`. #### 3. bake: `get_or_put_route_bundle` holds its index slot as `&Cell` The slot that caches a route's bundle index is either a plain field of the framework route or the `Cell` in the `HTMLBundleRoute`. To give both one type, the function turned them into `*mut Option<Index>` and read and wrote through that. `Cell::from_mut` gives the plain field the type the other one already has. The borrow of `self.router` lasts until the final store, and the code in between only touches `graph_safety_lock`, `client_graph` and `route_bundles`. #### 4. bake: `on_file_update` borrows the watchlist's columns The three columns came from one `Slice`, whose `items_mut` borrows all of it, so each column was turned into a raw slice pointer and back. The two columns that are only read now come from `watchlist` itself, which leaves the `Slice` free for the `count` column. `flush_evictions` ran in a `defer!` that reached `self` through a raw pointer. Nothing after it leaves the function early, so it is the last statement, still before the guard that submits the event. #### 5. bake: `process_file_list` borrows the dependency across `resolve` The loop copied the dependency's `specifier` out as a raw pointer so that no borrow of `dev` was held during the call to the resolver. The dependency lives in `dev.directory_watchers` and the resolver in `dev.server_transpiler`, two different fields, so a plain borrow of the dependency can stay alive for the call. #### 6. bake: borrow the arena's `Framework` view instead of re-deriving it `arena.alloc` returns `&'a mut Framework`. Both callers turned it into a raw pointer at once, because `Drop for DevServer` needs one, and then dereferenced that pointer to get the `&'a Framework` for the transpiler options. They now keep the reference, take the raw pointer from it and let the reference coerce to the shared one. #### 7. bake: `HmrSocket` reads the `DevServer` through its `BackRef` `on_open` and the console-log message only read the dev server, but got it from `dev()`, the `unsafe fn` that hands out `&mut DevServer`. The field is a `BackRef`, which derefs to `&DevServer`. A copy of it in a local leaves `self` free for the assignments in between. #### 8. bake: `PerThread::init` takes the `BackRef` its caller holds The caller unwrapped `pt.vm` to a raw pointer, and `init` wrapped that pointer in a `BackRef` again with `from_raw_mut`. It now takes the `BackRef`. #### 9. bake: remove the two empty `emit_*visualizer_message_if_needed` functions The body of `emit_memory_visualizer_message_if_needed` is `{}`. Its only two callers, both in `sweep_weak_refs`, went from the `SourceMapStore` to the `DevServer` that contains it, and so formed a `&mut DevServer` over the live `&mut SourceMapStore`, to call it. `emit_visualizer_message_if_needed` is just as empty, so it goes too, with its two calls. #### 10. bake: `finalize_bundle` runs its cleanup after the body, not from guards Two `defer!`s at the top of `finalize_bundle` drained the bundle's requests and ran `finalize_bundle_cleanup` on every exit. Each had to reach `dev`, `bv2` and the current bundle through raw pointers, because a closure that borrowed them would have locked them for the whole body. The body is now `finalize_bundle_inner`, and `finalize_bundle` calls it, then the cleanup, then returns its result. The cleanup already matches on `dev.current_bundle`, so it drains the requests in the `Some` arm, before `bv2.deinit_without_freeing_arena()` as before. #### 11. bake: `finalize_bundle_inner` borrows fields instead of going through `dev_ptr` Four places reached a second field of the dev server through a raw copy of `dev`, because the first borrow covered all of it. Two more kept the inspector agent as a raw pointer: - `route_bundle_ptr` borrows all of `dev`. Indexing `dev.route_bundles` leaves `dev.source_maps` free for `invalidate_client_bundle`. - The `Entry` from `put_or_increment_ref_count` borrows only `dev.source_maps`, so `dev.client_graph.take_source_map(entry)` compiles as it is. - `make_array_for_server_components_patch` and `send_serialized_failures` took `&mut self` but only read. With `&self` they can be given slices of `dev.incremental_result` and `dev.bundling_failures`. - `inspector()` ties the agent's lifetime to `&self`, although the agent lives in the VM's debugger. `BunFrontendDevServerAgent::of(vm)` holds the same body and `inspector()` forwards to it. Here the agent is borrowed from a copy of `dev.vm`, so it can be kept across the loop that mutates `dev`. The guard that takes `graph_safety_lock` again before the function returns is the only user of `dev_ptr` left, so the pointer is taken right above it. #### 12. bake: `BunFrontendDevServerAgent` wraps a reference to the debugger's slot The agent was a `#[repr(transparent)]` struct around `ErasedAgentSlot`, and `from_slot` cast `&ErasedAgentSlot` to `&Self`. It is only ever a view of the slot, so it now holds `&'a ErasedAgentSlot`, is `Copy`, and its methods take `self`. `Option<BunFrontendDevServerAgent<'_>>` is one nullable pointer, as `Option<&BunFrontendDevServerAgent>` was. #### 13. bake: `RequestEnsureRouteBundledCtx` is passed `dev` and borrows the request The context kept `dev` as a raw pointer, and `ensure_route_is_bundled`, which holds `&mut DevServer` itself, called methods on the context that made a second `&mut DevServer` from it. `on_defer` and `on_loaded` now take `dev` as a parameter. `on_defer` also picked the request list inside `dev` and handed it to `defer_request` next to `&mut self`. `defer_request` now takes the `BundleQueueType` and picks the list itself, right before it links the node. `ReqOrSaved::Req` held `*mut Request`, although both places that build a context have `&mut Request`. It holds that reference now, which `SavedRequestUnion::Stack` and `prepare_and_save_js_request_context` take as it is. #### 14. bake: `take_js_bundle` is given `root` and the configuration hash key To print an initial response, `take_js_bundle_to_list` went from the client graph to the `DevServer` that contains it and read `root`, through `relative_path`, and `configuration_hash_key` from there. The step to the container needs `&mut self`, which is the only reason that both functions took it, and because of it `generate_client_bundle` had to copy the entry point's path out of the graph before the call. Both values are now fields of `TakeJSBundleOptionsClient`, the functions take `&self`, and the entry point's path is borrowed from the graph. The body of `DevServer::relative_path` moves to a free function that takes `root`, and the method forwards to it. #### 15. bake: trace a route and build its client bundle from `Copy` values `trace_all_route_imports` and `generate_client_bundle` took `&mut self` and a `&RouteBundle` that lives inside `self`, so every caller turned one of the two into a raw pointer. All that they read from the route bundle is `Copy`: the route index or the HTML file's index, and the source map id. `trace_all_route_imports` now takes the `RouteIndex` of a framework route. For an HTML route it only forwarded to `client_graph.trace_imports`, which the callers now call themselves, in the `match` on the route's data that used to be inside. `generate_client_bundle` takes a `TraceRoot`, a `Copy` enum of the two indices, and the source map id. `on_js_request_with_bundle` is folded into `serve_client_script`, its only caller, which has already looked the route bundle up. The `TraceRoot` is built only where the bundle has to be built, so a request for a cached bundle does what it did before, with one bounds check less. In `generate_client_bundle` the guard that resets the client graph owns `self`, and the one that released the source map on failure is an `if let Err`. #### 16. bake: `compute_arguments_for_framework_request` takes `&mut self` It was an `unsafe fn` over `*mut Self` that made its accesses in 17 `unsafe` blocks, and was also handed a `&mut Framework` that points into `self.route_bundles`. Its last arm read that same route bundle again while the `&mut` was live. What it touches are separate fields: `route_bundles`, `router`, `server_graph` and `root`. With `&mut self` each is borrowed on its own. The one call that needs all of `self`, `generate_css_js_array`, now takes the route index, so no borrow is alive across it, and the result is stored through a fresh lookup. `client_file` is read before the borrow of the router type ends, and `global` comes from the caller, which has it. The function looks the route bundle up itself, so `on_framework_request_with_bundle` no longer does. `Data::framework` has no caller left. #### 17. bake: `server_transpiler` and `client_transpiler` are `ManuallyDrop`, not `MaybeUninit` The three transpiler fields were `MaybeUninit`, so every use after `init()` needed `assume_init_*`. But `init()` builds the whole struct under `Box<MaybeUninit<DevServer>>` and only calls `assume_init()` on it after the transpilers are written, so no `DevServer` with an unwritten server or client transpiler ever exists. Those two are `ManuallyDrop` and not plain fields to keep the point where they are dropped: inside `Drop for DevServer`, before the `bundler_framework_views` that their options borrow are dropped in place. Plain fields would be dropped after `drop` returns. The order in `drop` is unchanged. Their four accessors are gone, and the callers use the fields. `ssr_transpiler` stays `MaybeUninit`. Without a separate SSR graph it is a bitwise copy of `server_transpiler` and holds no value of its own. Its accessor, its uses under `separate_ssr_graph` and the check in `drop` are as they were. `start_async_bundle` gives the bundler a pointer to the slot with `NonNull::from(..).cast()`, which claims nothing about its contents. With that, the rest of `init()` can borrow fields through the `Box` instead of through `dev_ptr`: `framework.resolve` with the two resolvers, and the loop over the router types. `scan_initial_routes` passed `&mut *self_ptr` as the insertion handler next to `&mut self.router`. The handler only needs `root`, `server_graph` and `route_lookup`, so `DevRouteScan` borrows those three, like `RouteScan` of the production build and `JSFrameworkRouterScanCtx`. #### 18. bake: `generate_html_payload` takes the fields it uses It took `&mut self` and a `&RouteBundle` inside `self`, so `on_html_request_with_bundle` kept both `self` and the route bundle as raw pointers and dereferenced them for each step. The function only uses the graph lock and the two graphs, none of which overlaps `route_bundles`. It takes those three, in the way `invalidate_client_bundle` takes the `SourceMapStore`, and the caller holds one `&mut RouteBundle` from the lookup to the store of the response. `GraphTraceState::new` holds the body of `init_graph_trace_state`, which needs all of `&self`, and that method forwards to it. #### 19. bake: `init_transpiler` returns the transpiler it wrote `init_transpiler` fills a `MaybeUninit` slot. `MaybeUninit::write` gives it a `&mut Transpiler` to the result, which it dropped at the end, so the production build called `assume_init_mut` on each slot afterwards, for the SSR slot at every use. It now returns that reference. The production build keeps the three references. The SSR one is an `Option`, which replaces the tests of `separate_ssr_graph` at its uses. The bundler wants `&'a mut Transpiler<'a>`, which borrows the slot for as long as the arena. That is fine here, because nothing uses the slots after that call, so the reference is passed as it is instead of through a raw pointer, and the other two become `NonNull::from(..)` without a null check. #### 20. bake: `Assets::replace_path` becomes `DevServer::replace_asset_path` `replace_path` went from the `Assets` to the `DevServer` that contains it, to read `magic` and `server` and to register the path in `client_graph`. Both callers hold `&mut DevServer` with nothing of it borrowed, so the function is a method of `DevServer` that starts by destructuring the three fields it works on. It takes the same arguments as before, and `Assets` does not recover its parent from its own address any more. #### 21. bundler: `create_and_configure_transpiler` reborrows the transpiler instead of going through a raw pointer `configure_bundler` takes `&mut Transpiler<'a>` with a borrow of its own, so passing the `&'a mut Transpiler<'a>` from `bump.alloc` is an implicit reborrow that ends when the call returns. The transpiler lives in `bump`, not in `self`, so nothing conflicts with `&mut self`. The raw pointer and the two `unsafe { &mut *tp }` are gone; the reference is passed and then returned. #### 22. api: pass the `*mut JSObject` straight to `JSPropertyIterator::init` `init` takes `impl IntoIterObject`, which is implemented for `*mut JSObject`, and stores the raw pointer. Four callers in `JSTranspiler` and `JSBundler` turned the pointer from `get_object()` / `get_own_object()` into a `&JSObject` with `unsafe { &*obj }` only to hand it to `init`, which turned it back. They pass the pointer as the other callers do. #### 23. transpiler: `Bun.Transpiler` stops swapping `Transpiler.arena` on every call `scan`, `transformSync` and `scanImports` pointed the long-lived `Transpiler` at their per-call arena, and a guard put the old one back. The `transform` task did the same to its own bitwise copy of the `Transpiler`, with nothing to put back. Each took a `detach_lifetime_ref` to `'static`. Nothing they call reads that field: its readers are `configure_defines`, `build_with_resolve_result_eager` and `build_css_output`, and `parse`, `print`, `MacroContext::init` and `cache::JavaScript::scan` reach none of them. `parse` and `print` take the arena as an argument with a lifetime of its own. The swap, the saved `prev_arena` and the four lifetime erasures are gone, and `&arena` is passed as it is. #### 24. bundler: `JSBundleCompletionTask` binds what it matched instead of matching again with `unreachable_unchecked` Four places tested an enum with `matches!` and then matched it a second time with `_ => unsafe { unreachable_unchecked() }` to get at the payload. - The sourcemap loop in `do_compilation` binds `bytes` in the condition. - `do_compilation` took `&mut self`, so `on_complete` had to move `output_files` out of `self.result` and put it back. It only uses `config`, `env` and `log`, so it takes those, and the caller lends it `&mut build.output_files` next to them: four disjoint fields. - `on_complete` handles `Pending` and `Err` as arms of the one match. No loan of `this.result` is live in the `Err` arm, so `to_js_error` can borrow `this`. #### 25. image: build the encoders' `NonNull<[u8]>` from the null test that is already there The PNG and WebP encoders tested the codec's buffer for null and then made `Encoded.bytes` with `NonNull::new_unchecked`, or with a `&mut [u8]` from `from_raw_parts_mut` that was only used for its pointer and length. `NonNull::new` is that same null test, and `NonNull::slice_from_raw_parts` (as in `Encoded::from_owned`) adds the length without claiming a reference. `WebPChunkIterator` is written out as a literal of zeroes instead of `MaybeUninit::zeroed().assume_init()`; it has no padding. The `Blob` arm of `PipelineTask::then` drops the `Encoded`, as the base64 arm does, instead of calling its `free` by hand: `Encoded::drop` makes the same call. #### 26. image: `codecs::encode` takes the ICC profile as `Option<&[u8]>` `EncodeOptions.icc_profile` was an `Option<NonNull<[u8]>>` standing in for a borrow that only has to last for one `encode()` call. It had one writer, right before that call, and one reader, `unsafe { p.as_ref() }` at the top of `encode()`; no other code set it, so the `is_none()` test in front of the write was always true, and both system backends ignore it. The field is gone and the one caller passes `decoded.icc_profile.as_deref()` as an argument. #### 27. terminal: `Terminal::open` owns the first ref as a `RefPtr` from the start `open` boxed the `Terminal`, leaked it with `heap::into_raw`, worked through `unsafe { &*terminal }`, called `deref_()` by hand on each failure path and wrapped the pointer with `unsafe { RefPtr::from_raw(..) }` at the end. It returns a `RefPtr<Terminal>` anyway, so it makes one with `RefPtr::new`, which is the same allocation. Every method it calls takes `&self`, which the `RefPtr` derefs to. Returning early drops the `RefPtr`, which releases the initial ref where the explicit `deref_()` did, so the one in the writer path and the one in `fail_reader_start` (called only from `open`) are gone. The reader's own ref is still released by hand. #### 28. transpiler: `Bun.Transpiler`'s guards borrow the transpiler, and `Transpiler::print` takes `&self` - `TransformTask::run` kept a raw pointer to `self.transpiler.macro_context` in a scopeguard while it went on using `self.transpiler`. The guard holds `&mut *self.transpiler` and the body works through the guard; the task's other fields are disjoint places. - `TranspilerStateGuard` kept `JsCell::as_ptr()` and turned it into a `&mut` in its `Drop`. It borrows the `JsCell` for its own lifetime and restores the state with `with_mut`, the way the callers set that state up. - The six `print*` functions of `Transpiler` only read `self.options`, so they take `&self`, and `transformSync` prints through `JsCell::get()` instead of the unsafe `transpiler_mut()`. #### 29. transpiler: `JSTranspiler`'s constructor and `Drop` use the exclusive access they have The constructor owns a fresh `Box<JSTranspiler>` and still went through the unsafe `JsCell::get_mut(&self)` for `config` and `transpiler`. `JsCell` gets `as_mut(&mut self)`, which is `UnsafeCell::get_mut` and needs no `unsafe`, and the constructor borrows the two fields with it. `Drop` called `clear_and_free()` through the raw `transpiler.log`. That pointer is `&self.config.log` whenever a `JSTranspiler` is dropped: the constructor sets it right after boxing, before anything can return, and the guard in `scan`, `transformSync` and `scanImports` puts it back on every exit. Dropping the `config` field frees the same messages, so the call is gone. #### 30. api: keep a `NonNull` a `NonNull` instead of going raw and back with `new_unchecked` Three callers had a non-null pointer by type, passed it on as `*mut T`, and the receiver rebuilt the `NonNull` with `unsafe { NonNull::new_unchecked(..) }`. - `CompletionDispatch::enqueue_task_concurrent` takes `NonNull<ConcurrentTask>`: the bundler holds one and `Ticket::post` wants one. - `Subprocess::on_process_exit` takes `NonNull<Process>`, made from the `&mut Process` that the exit interface hands the thunk. Its only use is `VirtualMachine::on_subprocess_exit`, which takes a `NonNull`. - `CronJob::tick_settled` gets the `NonNull` inside its `ThisPtr` through the existing `BackRef` conversions, as `html_rewriter` does for `deref_nn`. #### 31. api: four functions exported to C++ are typed like the ones next to them - `Bun__CryptoHasherExtern__destroy` takes `Box<CryptoHasher>`, the type that `getByName` and `getFromOther` hand out, instead of a raw pointer it fed to `Box::from_raw`. `Box<T>` has the ABI of a non-null pointer, and no C++ caller passes null: `~JSHash` and `digest` test `m_zigHasher` first, and a `unique_ptr` only runs its deleter on a non-null pointer. - `JSBundlerPlugin__onDefer` takes `&mut Load` and `&JSGlobalObject`, like `onLoadAsync` below it. C++ calls it only for a request that `holdsRequest` found. - `JSBundlerPlugin__addError` takes `&mut Plugin`. It stays `unsafe` for its `*mut c_void` context. - The `BunObject_lazyPropCb_*` thunks take `&JSGlobalObject` and `&JSObject`, which is what they pass on. `jsc_host_abi!` still wants `unsafe fn`. The first two are no longer `unsafe fn`. None has a Rust caller. #### 32. runtime: drop four small `unsafe` blocks that a safe form replaces - `Image::metadata`: `js_thread_bytes` returned a slice of a temporary `ArrayBuffer` view with the lifetime stretched to `&self`. Its one caller keeps the view in its own frame, so the slice borrows the view. - `HMAC_CTX_init` is a write-only initialiser that reads nothing from its argument, so it is declared `safe fn (&mut MaybeUninit<HMAC_CTX>)`, like `tcgetattr` in `bun_sys`, and `HMAC::init` and `HMAC::copy` call it without a block. The SHA `*_Init` declarations in the same file stay `*mut`; their comment says that what it objects to is `&mut CTX`. - The `abort` crash-test hook calls `std::process::abort()`, which is `libc::abort()` on Unix. #### 33. bake: `start_async_bundle` makes the arena's `&'static` reference once `BundleV2::init` takes the arena twice, as `alloc` and as `heap`. `start_async_bundle` passed `&heap` for the first and `unsafe { &*heap_ptr }` for the second. The lint `arg_named_like_other_param` reads a variable named `heap` in the `alloc` position as an argument in the wrong place. The `&'static` reference is made once, in the block that was already there, and both parameters get it. It is the address that both got before. ### How did you verify your code works? - **Static checks on the final commits:** `cargo fmt --check`, `cargo clippy --workspace` (0 warnings), the source lints and `cargo check --workspace --release` on Linux. - **Assembly:** the comparison above. - **84 test files**, on a debug build with ASAN of these commits and, where something failed, on a debug build of the base: all 26 files of `test/bake/`, all of `test/js/bun/image/`, `test/js/bun/terminal/` and `test/js/bun/cron/`, 20 files of `test/js/bun/transpiler/` and `test/bundler/transpiler/`, 14 files of `test/js/bun/spawn/`, and the files for the rest (`bun-build-api`, three of the bundler's plugin tests, `plugins`, `resolve`, `import-meta`, `bun-cryptohasher`, two `node:crypto` files and `run-crash-handler`). - 78 files pass completely, 2,162 tests. 25 of them are dev-server files, with 269 tests. - 2 of the 78, `terminal.test.ts` and `spawn-maxbuf.test.ts`, failed in 2 and 1 tests the first time, with eight files running at once. Alone they pass on both builds. Because change 27 touches `Terminal::open`, I then ran `terminal.test.ts` six more times on each build, all twelve at once: 101 of 101 every time. - 3 files fail in the same tests on both builds: `spawn-pipe-leak.test.ts` (3 of 3, their time limit of 30 seconds), `bun-build-api.test.ts` (1 of 91, its time limit of 180 seconds) and `run-crash-handler.test.ts` (1 of 31, which expects a frame name that a debug build prints differently). - 3 files need packages that I did not have: `plugins.test.ts` and `test/bake/dev/ecosystem.test.ts` (`svelte`), and 1 of the 108 tests of `resolve.test.ts` (`tsyringe`). - The dev-server tests load `happy-dom`. I ran them with version 20.0.11 and its script evaluation switched on, not with the version that `test/package.json` pins. - **After the tests had run** I removed three `#[inline]` attributes that an earlier form of changes 11, 14 and 18 had added, and added commit 33. The static checks ran again on the final commits. --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
Fixes oven-sh#44697 ### Problem - Bun's JavaScriptCore is 360 upstream commits behind WebKit/WebKit@e8f704815e94. 59 touch JavaScriptCore, WTF or bmalloc. - `RegExp.lastMatch` after a `test()` that matched can end the process: `panic(main thread): Segmentation fault at address 0x4`. - Two results are wrong: `Math.sumPrecise([-1])` is `-1.0000000000000002`, and an async function that awaits itself through `Promise.all` repeats its frame in `Error.stack`. ### Fix - `WEBKIT_VERSION` is a7b59e61de2e, the merge commit of oven-sh/WebKit#923: the engine of `main` plus the upstream merge. - Upstream deleted one builtin: in `test/aot/engine-tests`, 2,648 builtin numbers are one lower and 19 instance offsets eight bytes lower. Loop bytecode changes, so `bundler_bytecode_portable.test.ts` has new snapshots. - **Two test bounds go up, which needs a maintainer's yes**: the `--trim` size in `compile-aot-size.test.ts` and the function count of one program (Notes). - Verified, local debug build: `test/js/bun/jsc/webkit-upgrade-e8f704815e.test.ts` (new, 17 cases, Bun 1.4.3-canary fails 4) and `test/js/bun/jsc/regexp-legacy-static-properties.test.ts` (new, the panic on Bun 1.4.3-canary). `test/aot/engine-tests` passes with a release `testaot`. ### Background - `WEBKIT_VERSION` (`scripts/build/deps/webkit.ts`) names the oven-sh/WebKit release whose JavaScriptCore the build downloads. - The ahead-of-time compiler numbers the engine's JavaScript builtins by name order, and the instance of a compiled program has one word per link-time constant. ### Downsides - Size: hello world with `--aot --trim` maps 167,808 bytes more. A compiled program that calls `Array.from` holds 22 functions, was 6. - A thread that breaks the collector's stop-the-world protocol now crashes a release build (Notes). - The own keys of the `Proxy.revocable` revoke function now start with `length` and `name`. <details><summary>Notes</summary> #### The WebKit side oven-sh/WebKit#923 merges upstream `e8f704815e94` into the fork. Its description lists the 16 conflicts and how each one is resolved. The first CI run of that pull request built every lane and published a preview release. Its test jobs failed for three reasons. None of them is in the libraries that Bun links: - Upstream now links its programs with `-z pack-relative-relocs` (WebKit/WebKit@c47b7c073c). The `jsc` and `testFFI` programs of the Linux lanes then need glibc 2.36. The arm64 test runner has glibc 2.35. - One new upstream test (`stress/regexp-legacy-static-properties-rematch-failure.js`) cannot pass in two test modes that only the fork has. - Upstream deleted one JavaScript builtin. The tests of the ahead-of-time compiler name builtins by number, and the numbers behind the deleted one moved. That pull request has a commit for each of them. After that, the fork's `main` got oven-sh/WebKit#914, oven-sh#886 and oven-sh#926, and Bun's `main` got the ahead-of-time compiler (oven-sh#45631) with that WebKit (oven-sh#47125). Both are merged in, over there and here. A review of the merged head found one defect of the merge itself: with the object loop of `JSON.parse` that upstream rewrote, `Bun.JSONL` could give an object a Structure that a collection had freed. `bcd362b31e` over there repairs it and adds a stress test. With the `jsc` of the build before it, that test ends in `SIGSEGV` in 6 runs of 6. The fork's `main` then moved three more times, and each time its Bun side had to be on Bun's `main` before this branch could pin a build with it: | oven-sh/WebKit `main` | Bun side | merged into oven-sh/WebKit#923 at | |---|---|---| | oven-sh#927 | oven-sh#47129 | `be93e08e36` | | oven-sh#930 | oven-sh#47129 | `35b64c4afb` | | oven-sh#919, oven-sh#936 (ICU 79.1), oven-sh#937 | oven-sh#47281 | `b67a0b1a3a` | - `780dc90982` over there turns upstream's fast path for `{ ...fn }` off in builds with the ahead-of-time compiler (see "What Bun's CI showed"). - oven-sh/WebKit#923 is merged as `a7b59e61de`, and `WEBKIT_VERSION` is that commit. It has the same tree as `b67a0b1a3a`, the head whose preview build (`autobuild-preview-pr-923-b67a0b1a`) this branch pinned before. That is `8c6c963e3c`, which `main` pins since oven-sh#47281, plus the upstream merge. The CI of oven-sh/WebKit#923 was green at that head: 84 jobs, with `testFFI`, `TestWTF`, `testaot` and `run-javascriptcore-tests` on Linux x64 and arm64. - A build of this branch against an engine whose Bun side is not on `main` does not work. With the preview of `780dc90982` and `main` before oven-sh#47129, the build stopped at `src/jsc/bindings/BunCompiledModules.cpp:756` (`too few arguments to function call`, `JSC::AOT::tryReservePagesForNewThread`), and with that call changed every `bun build --compile --aot` ended with `IdOutOfRange`. The release zip sizes of an earlier version of this description compared builds 133970 and 133646: between -0.15% and +0.12% over the 12 release zips (`bun-linux-x64.zip` -38,546 bytes). #### What Bun's CI showed The head is `e22cf6a525` now. On top of `3688162c4c` it has the pin of the merge commit, the test for oven-sh#44697 and one corrected comment. The engine source is the same as in build 135015. **Build 135015** ran `3688162c4c`, the head with the answers below. 184 of its 188 jobs passed and none failed: every build, and every test job on Linux, Windows and macOS on ARM64. The four macOS x64 test jobs had not started when this was written. No agent of that queue was free, and three attempts of each job expired after an hour of waiting. Four tests failed once and passed on a retry or alone (`bun-serve-html.test.ts`, `no-leak-records-at-exit.test.ts`, `hot.test.ts`, `process-stdin.test.ts`). **Build 134977** ran the head before that, with `main` up to oven-sh#47129 and the preview of `35b64c4afb`. Three things failed that do not pass on a retry. `3688162c4c` answers each: | Test | Failure | Answer | |---|---|---| | `test/aot/engine/ir-1`, `ir-3`, `ir-5`, `ir-6` (nine `.ir` files, Linux x64) | `lower-to-b3: output line 7 does not match: Int64 b@22 = Load(b@10, offset = 8264, ..); nearest: .. offset = 8256 ..` | The offsets are written anew (see "The engine tests") | | `test/aot/parallel/arrays/searches-folds-and-flat-maps-of-a-list-bring-in-only-the-functions-that-do-them.js` | `ImageFunctionsEmitted <= 40, but the build counted 42` | The bound is 48. **A maintainer has to agree** | | `test/bundler/compile-aot-size.test.ts` | hello world with `--trim` maps 25,777,292 bytes, and the bound is 25,700,000 | The bound is 25,880,000. **A maintainer has to agree** | Eight more tests failed and passed on a retry or alone (`plugin-requests-run-ahead.test.ts`, `no-leak-records-at-exit.test.ts`, `compile-aot-run-time.test.ts`, `43831.test.ts`, `filesink.test.ts`, `process-stdin.test.ts`, `body.test.ts`, `test-cluster-shared-leak.js`). **The function count.** The program calls `Array.from({ length: 16 })` and six array methods. Its executable holds 24 functions with the canary of `main` and 42 here. - Upstream's `Array.from` (WebKit/WebKit@37aa2b5bd94d) steps an iterator by a call of its `next` and closes it by a call of its `return`. The old text was a `for-of` loop. - One function of the library runs for every call of `Array.from`, so the compiler keeps what that call of `next` can reach by name. The 18 more functions are the two helpers `iteratorGenericNext` and `iteratorGenericClose`, one part of `Array.from` that is moved out of line, and the `next`, `return` and `throw` of array iterators, generators, iterator helpers and wrapped iterators. - A one-line `console.log(Array.from([1, 2, 3], x => x + 1).join())` holds 22 functions where `main` holds 6. Its bound is 24. - oven-sh#47316 expands the two helpers in line in the front end (its engine half is oven-sh/WebKit#934). A release build of it keeps 25 functions for this program and 7 for the one-line program, so the bound of 40 holds again once that pull request is on `main`. The bound of 48 is needed only until then. **The size.** `test/bundler/compile-aot-size.test.ts`, Linux x64 with glibc. Build 135015 is the commit of the canary plus this upgrade: | hello world | canary of `main` (`081ace731d`) | build 135015 | bound | |---|---|---|---| | `--aot` | 68,138,180 | 68,305,988 | 68,440,000 | | `--aot --trim` | 25,584,836 | 25,752,644 | 25,700,000, now 25,880,000 | Both grow by 167,808 bytes. `.text` of `jsc` grows by 143,376 bytes with the 360 upstream commits. The test says that its bounds only go down, so this is a maintainer's call. The new bound has about the slack the old one had (127,356 bytes, it was 115,164). The comment beside it says 183,000 bytes, which was measured on an older commit of `main`. The next push corrects it to 168,000. **Earlier runs** (builds 134164 and 134232, before oven-sh#47129, with the test names of that time) showed two more causes. Both are answered on the WebKit side: - A stack frame after `{ ...fn }`. Bun names a frame by the own property `name` of the callee first, and by `displayName` after it. A copy that looks every key up makes `name` a property of a function. Upstream WebKit/WebKit@e04a6f9faecf copies from a function by its Structure and leaves `name` alone, and the run time of compiled programs still looks the keys up. Three programs printed another frame compiled than plain. A maintainer chose to turn upstream's fast path off in builds with the ahead-of-time compiler (`780dc90982` of oven-sh/WebKit#923). With a local debug build of this branch, `errors/frames-functions-display-name-statement.js`, `errors/frames-functions-given-display-name-far.mjs` and `errors/names-frames-new-function-objects-names.mjs` print the same compiled and plain. The cost: `{ ...fn }` does not get upstream's speedup (160 ms against 50 ms on `JSTests/microbenchmarks/object-spread-function.js`). - `Array.from` in a compiled program held 441 functions. The flow analysis took the call of `next` inside the helper `iteratorGenericNext` for a call of a value nobody knows. oven-sh/WebKit#930 changed that rule, and 22 functions are left (above). Build 134164 also showed a crash under collector stress (`SIGSEGV` at address 0 in `WTF::findAtomBeforeAdding`). Its cause was on `main` too: `AOT::nameOfBuiltin()` makes an atom while a collection ends, where `Heap::pruneDeadReferences()` has set the atom table of the thread to null. #### The engine tests `test/aot/engine-tests` has the IR text tests of the engine that need entries of Bun's table of builtins (1,601 files since oven-sh#47129). - A test names a builtin that is written in JavaScript by its runtime function number. Upstream WebKit/WebKit@37aa2b5bd94d deleted `defaultAsyncFromAsyncIteratorWithNext`, which was number 23. Every number from 24 on is one lower: 2,648 numbers in 371 files. `every` is 30 (was 31), `find` 34, `from` 46, `map` 64 and `some` 88. The numbers of Bun's own functions start at 0x10000 and do not move. No file names a number that the engine's list does not have. - The instance of a compiled program holds one word for every link-time constant of the engine, and the deleted builtin was one. Every field behind that list is eight bytes lower. Nine tests check the B3 of a function line by line and had such offsets in 19 lines (8040, 8168, 8192, 8216, 8264). `test/aot/tools/update-ir-tests.ts` wrote them anew with a release `testaot` on Linux x64, and nothing else in those files changed. - With a release `testaot` and `jsc` built from this branch, `testaot run test/aot/engine-tests` passes: 1,599 files, 2 todo. - oven-sh/WebKit#923 makes the same change of numbers in the engine's own tests. #### The new test file `test/js/bun/jsc/webkit-upgrade-e8f704815e.test.ts` has 17 cases. Four of them fail on Bun 1.4.3-canary, because the engine changes there: | Case | Old engine | New engine | |---|---|---| | `Math.sumPrecise([-1])` | `-1.0000000000000002` | `-1` | | Own keys of the revoke function of `Proxy.revocable`, after `length` is deleted and defined again, against those of an arrow function after the same steps | another order (`name`, `length`) | the same order (`length`, `name`) | | `Error.stack` of an async function that awaits its own promise through `Promise.all` | the frame repeats | one frame | | `[x, y].sort(boundComparator)` when the comparator leaves optimized code (the program of oven-sh/WebKit#885) | the caller gets `Array.prototype.sort` itself | the caller gets the sorted array | The other 13 cases pass on both engines. They cover the code that the merge rewrote by hand: spread and rest of a function, the iterator protocol of `Array.from`, `Object.groupBy`, `Map.groupBy` and the iterator helpers, `Array.fromAsync`, `Array.prototype.join`, `JSON.parse` of objects with the same keys (also `-0`, invalid input and a reviver), `Bun.JSONL.parseChunk` cut at every position, loops that end in a null test or a negated comparison, and an anchored RegExp and a `switch` on a substring. #### The bytecode snapshots `test/bundler/bundler_bytecode_portable.test.ts` compares a hash of the serialized bytecode. 22 entries come from a local debug build with the merged engine. The entry `bun build --bytecode libraries.js` comes from CI: the bundle of that entry differs on the machine the other values come from. All ten test platforms of build 133970 reported the same value for it, and accepted the other 22. #### Bun's CI on the first preview build Build 133970 ran this branch with `autobuild-preview-pr-923-b6e34894`. One test failed on every platform: the snapshot entry above, which had its old value then. Eight other tests failed and passed on a retry (`bake.test.ts`, `hot.test.ts`, `no-orphans.test.ts`, `spawn_waiter_thread.test.ts`, `test-zlib-invalid-input-memory.js`, `nth-anplusb-ident.test.ts`, `process.test.js`, `bun.test.ts`). The zip sizes in "The WebKit side" compare the release zips of that build with those of build 133646, which is `main` at the commit this branch started from. The range is over the 12 release zips. The unpacked binaries were not measured: the artifact store of CI is not reachable from the machine this was written on. #### The crash of oven-sh#44697 `test()` records only where the match is. A read of `RegExp.lastMatch`, `RegExp.$1` or another legacy static property runs the match again for the captures. That second run can need more stack than the first, and when it failed, `RegExpGlobalData::getBackref` read through a null pointer. Upstream WebKit/WebKit@191803bb2304, which is in this range, throws a `RangeError` there. ```sh BUN_JSC_maxRegExpStackSize=65536 bun -e ' const re = new RegExp("^" + "(z)?".repeat(8) + "(?:a|b)+$"); console.log(re.test(Buffer.alloc(100000, "a").toString())); console.log(RegExp.lastMatch.length);' ``` - Bun 1.4.3-canary: `true`, then `panic(main thread): Segmentation fault at address 0x4`. - A debug build of this branch: `true`, then `RangeError: Out of memory`, also for `RegExp.$1`, `RegExp.$9`, `RegExp["$&"]` and `RegExp.lastParen`. `test/js/bun/jsc/regexp-legacy-static-properties.test.ts` runs the program of upstream's stress test in a child with `BUN_JSC_maxRegExpStackSize=65536` and compares every line that the child prints: `matched: true` for the first `test()`, a `RangeError` for each read and for an assignment to `RegExp.input`, and the values of the properties after the next match. `USE_SYSTEM_BUN=1 bun test` fails it with the panic, and `bun bd test` passes it. The test comes from oven-sh#47160, which had the upstream commit alone (oven-sh/WebKit#929, closed now). #### What ran locally - With the pin of the merge commit, a debug build with ASAN: `webkit-upgrade-e8f704815e.test.ts` passes 17 of 17, and `regexp-legacy-static-properties.test.ts` passes. - `bun run build:local` against the merged WebKit (Debug, no ASAN). `-p '42'` prints `42`. - `test/js/bun/jsc`: 215 pass. Two `domjit.test.ts` cases pass their 20 s limit in this Debug build. The same two take 42 s on a debug build of `main`. - `test/js/bun/jsonl`, the worker tests, the GC tests and the heap snapshot tests pass. - `JSType.h`, `OptionsList.h`, the WebCore bindings scripts and the structured clone sources have no upstream change in this range. `src/jsc/JSType.rs` needs no change. #### The three assertion commits WebKit/WebKit@0bf94cf0c1, WebKit/WebKit@4bc9ad4d7f and WebKit/WebKit@cf89f1571e check the bits of `Heap::m_worldState` with `RELEASE_ASSERT`, in the functions through which a thread takes or gives up its access to the heap. Bun gives up heap access while the event loop waits (`Bun__JSC_onBeforeWait`) and takes it again after the wait. That order is the one the assertions check. #### Upstream changes Range: [`f59345e04a8d`](WebKit/WebKit@f59345e) (2026-10-22) .. [`e8f704815e94`](WebKit/WebKit@e8f7048) (2026-10-27). **59** of the 360 upstream commits touch `Source/JavaScriptCore`, `Source/WTF` or `Source/bmalloc`. 20 of the 59 change nothing that a JSCOnly build compiles: they touch a preferences yaml, a sandbox profile, the Xcode project, or code for Swift, GLib or Cocoa only. ### Highlights - **`JSON.parse` reads the keys it already knows in a loop of their own** ([`6f5f583b55`](WebKit/WebKit@6f5f583b55)). A key found in `JSONCache`'s name table, and a value that is a cached string or an integer of at most nine digits, are read through a local cursor. The lexer is only used for the other values. Upstream measures 1.13x on its JSON benchmarks (geometric mean), up to 1.32x. - **Object spread and rest of a function take the fast path** ([`e04a6f9fae`](WebKit/WebKit@e04a6f9fae)): `{ ...fn }` and `let { a, ...rest } = fn` are 5x faster. PostCSS does this once per `LazyResult`. - **`Array.from`, `%TypedArray%.from`, `Object.groupBy`, `Map.groupBy` and `Iterator.prototype.flatMap/some/every/find` drive the iterator themselves** ([`37aa2b5bd9`](WebKit/WebKit@37aa2b5bd9)). They no longer build a wrapper object for a `for-of` loop: 1.1x to 1.6x on the `Array.from` microbenchmarks. - **`Proxy.revocable` is 2.9x faster** ([`fad04e2527`](WebKit/WebKit@fad04e2527)). The revoke function is a `JSFunctionWithFields` with a cached executable, and the result object has a cached Structure. The class `ProxyRevoke` is gone. - **Loop conditions are one instruction more often** ([`bd50166362`](WebKit/WebKit@bd50166362)). A null test or a negated compare that feeds a backward conditional jump is fused into the jump (`jneq_null`, `jnless`, ...). No opcode is new, but the bytecode of most loops changes. - **The RegExp first-character filter reads substring ropes in place** ([`b92d6a6bea`](WebKit/WebKit@b92d6a6bea)), and `String#match` with a constant anchored RegExp uses the filter in the DFG. The same commit makes DFG/FTL `switch` on a substring rope read the width of the base string, not the bit cached in the rope. - **Two DFG/FTL and Wasm miscompile fixes**: an OSR exit in a bound `Array#sort` comparator returned past the `sort` call ([`f274ef4c03`](WebKit/WebKit@f274ef4c03)), and a cross-instance Wasm tail call copied its frame from one `restoreFrameSize` too low ([`0927910cf9`](WebKit/WebKit@0927910cf9)). - **The GC asserts its stop-the-world protocol in release builds** ([`0bf94cf0c1`](WebKit/WebKit@0bf94cf0c1), [`4bc9ad4d7f`](WebKit/WebKit@4bc9ad4d7f), [`cf89f1571e`](WebKit/WebKit@cf89f1571e)). A mutator that takes or gives up heap access in a state the protocol does not allow crashes now. - **`Math.sumPrecise` is correct for negative sums** ([`8ee560d748`](WebKit/WebKit@8ee560d748)): `Math.sumPrecise([-1])` was `-1.0000000000000002`. ### Runtime & builtins - [`e04a6f9fae`](WebKit/WebKit@e04a6f9fae) ([328286](https://bugs.webkit.org/show_bug.cgi?id=328286)): `@copyDataProperties` and `@cloneObject` accept a `JSFunction` source on the path that reads the Structure, through the new `JSFunction::canPerformFastEnumerablePropertyEnumeration(VM&)`. It needs an unmodified `name` and `length`, no accessors and no own `__proto__`. object-spread-function 4.99x, object-rest-function 5.54x. - [`37aa2b5bd9`](WebKit/WebKit@37aa2b5bd9) ([328563](https://bugs.webkit.org/show_bug.cgi?id=328563)): Five builtin files call `@iteratorGenericNext` in a `for (;;)` loop and close the iterator with `@ifAbruptCloseIterator`. The link-time constant `defaultAsyncFromAsyncIteratorWithNext` is removed, and `defaultAsyncFromAsyncIterator` has its body now. - [`fad04e2527`](WebKit/WebKit@fad04e2527) ([328635](https://bugs.webkit.org/show_bug.cgi?id=328635)): `Proxy.revocable`: see Highlights. Removed: `ProxyRevoke.{h,cpp}`, `ProxyRevokeInlines.h`, `proxyRevokeSpace`, `JSGlobalObject::proxyRevokeStructure()`. Added: `proxyRevocableResultStructure()`, `VM::proxyRevokeExecutable()`, `JSFunctionWithFields::Field::RevocableProxy`, the identifiers `proxy` and `revoke`. Observable: the own keys of the revoke function list `length` and `name` first, like any function. - [`e81441a5ec`](WebKit/WebKit@e81441a5ec) ([328769](https://bugs.webkit.org/show_bug.cgi?id=328769)): `JSOnlyStringsAndInt32sJoiner` is renamed `JSExactLengthJoiner`. `Array#join` keeps its exact-length path for arrays that also hold booleans, `undefined` or `null`. - [`8ee560d748`](WebKit/WebKit@8ee560d748) ([315450](https://bugs.webkit.org/show_bug.cgi?id=315450)): `WTF::Xsum::XsumSmall::compute()` rounded a negative sum away from zero when it must not. `Math.sumPrecise([-Number.MAX_VALUE])` was `-Infinity`. - [`071ce01fcb`](WebKit/WebKit@071ce01fcb) ([328147](https://bugs.webkit.org/show_bug.cgi?id=328147)): `Interpreter::getAsyncStackTrace` stops at the second visit of a promise combinator. An async function that awaits its own promise through `Promise.all` no longer repeats its frame up to `Error.stackTraceLimit`, or hangs. - [`be583048f3`](WebKit/WebKit@be583048f3) ([328788](https://bugs.webkit.org/show_bug.cgi?id=328788)): `Identifier::add` no longer does one atomic ref and one deref of the `AtomStringImpl` it returns. - [`ca48e0119c`](WebKit/WebKit@ca48e0119c) ([328786](https://bugs.webkit.org/show_bug.cgi?id=328786)): The last two `USE(BIGINT32)` blocks are deleted. Dead code. ### Parser / bytecode generator - [`bd50166362`](WebKit/WebKit@bd50166362) ([328623](https://bugs.webkit.org/show_bug.cgi?id=328623)): `emitJumpIfTrue` and `emitJumpIfFalse` no longer require a forward target for their fusions. `while (node != null)` and `do {} while (!(a < b))` end in one fused jump. The subject names only the null tests. The diff also covers the ten negated compares. - [`4e5e850636`](WebKit/WebKit@4e5e850636) ([328770](https://bugs.webkit.org/show_bug.cgi?id=328770)): `BytecodeGeneratorBase<Traits>` is folded into `BytecodeGenerator`. `GenericLabel` and `GenericBoundLabel` are the plain classes `Label` and `BoundLabel`, and `Label::bind` takes the instruction writer. `BytecodeGeneratorBase.h` is deleted and `BytecodeGeneratorBaseInlines.h` is renamed `BytecodeGeneratorInlines.h`. `BytecodeList.rb` loses the unused type `:WasmBoundLabel`, and no opcode. ### DFG / FTL - [`f274ef4c03`](WebKit/WebKit@f274ef4c03) ([327983](https://bugs.webkit.org/show_bug.cgi?id=327983)): An inlined bound function keeps the call kind of its caller. `InlineCallFrame::BoundFunctionCall` and `BoundFunctionTailCall` are removed. A bound comparator inlined into `Array#sort` is an `ArraySortComparatorCall`, so an OSR exit inside it restarts the `sort`. A recursive tail call through a bound function can become a jump in the FTL. - [`b92d6a6bea`](WebKit/WebKit@b92d6a6bea) ([328389](https://bugs.webkit.org/show_bug.cgi?id=328389)): See Highlights. `compileRegExpMatchFast` (DFG) returns `null` without a call when the first byte cannot start a match. ### RegExp - [`59e81f3e12`](WebKit/WebKit@59e81f3e12) ([328828](https://bugs.webkit.org/show_bug.cgi?id=328828)): A match array gets `index`, `input`, `groups` and `indices` written into its butterfly before the array cell exists. `tryCreateUninitializedRegExpMatchesArray` has a fifth parameter, and `JSArray::tryCreateUninitializedRestrictedButterfly` is new. - [`1984003fd9`](WebKit/WebKit@1984003fd9) ([328895](https://bugs.webkit.org/show_bug.cgi?id=328895)): On x86_64 the Yarr JIT copies fewer than 16 `ParenContext` frame slots with straight-line code, and eight per loop iteration above that. regexp-parencontext-frame-slots 1.80x. - [`191803bb23`](WebKit/WebKit@191803bb23) ([328417](https://bugs.webkit.org/show_bug.cgi?id=328417)): `RegExp.lastMatch`, `RegExp.$1` and the other legacy properties throw a `RangeError` when the lazy re-match fails. Before, they read a null array. ### JSON - [`6f5f583b55`](WebKit/WebKit@6f5f583b55) ([328615](https://bugs.webkit.org/show_bug.cgi?id=328615)): See Highlights. The object branch of `LiteralParser::parseRecursively` is one loop with the lambdas `parseValue`, `putWithTransition` and `finishObject`. `Lexer::skipWhitespaceBeforeKey()` is removed, `Lexer::setPtr()` and `end()` are new, and `numberOfDigitsForSafeInt32` moves to `ParseInt.h`. - [`2dff3f1dbf`](WebKit/WebKit@2dff3f1dbf) ([329046](https://bugs.webkit.org/show_bug.cgi?id=329046)): Compile fix for the commit above with clang 20: three lambdas capture `vm` by name. ### GC / heap - [`0bf94cf0c1`](WebKit/WebKit@0bf94cf0c1) ([328987](https://bugs.webkit.org/show_bug.cgi?id=328987)): `Heap::stopTheMutator` asserts that no mutator holds the conn, where it had a branch that returned false. - [`4bc9ad4d7f`](WebKit/WebKit@4bc9ad4d7f) ([329005](https://bugs.webkit.org/show_bug.cgi?id=329005)): `Heap::resumeTheMutator` asserts that the mutator is stopped, has no heap access and no conn. It returned early when the mutator was not stopped. - [`cf89f1571e`](WebKit/WebKit@cf89f1571e) ([329038](https://bugs.webkit.org/show_bug.cgi?id=329038)): Release asserts on `Heap::m_worldState` in `acquireAccessSlow`, `releaseAccessSlow`, `setNeedCollectionEpilogue`, `setMutatorWaiting`, `clearMutatorWaiting`, `Collector::beginCollectionInEachHeap`, `finishChangingPhase` and `stopThePeriphery`. `Collector::requestCollection` gets two debug asserts. `Heap::needCollectionEpilogue()` is new. ### WebAssembly - [`0927910cf9`](WebKit/WebKit@0927910cf9) ([329002](https://bugs.webkit.org/show_bug.cgi?id=329002)): `Wasm::emitRestoreInstanceFrameIfNeeded` starts its copy at the stack pointer from before the adjustment. ### WTF - [`45f193ae80`](WebKit/WebKit@45f193ae80) ([328940](https://bugs.webkit.org/show_bug.cgi?id=328940)): `Thread::tryCreate()` returns null when the thread cannot be created. `Thread::create()` calls it and still crashes. `$.agent.start` in the `jsc` shell throws. - [`e8f704815e`](WebKit/WebKit@e8f704815e) ([329066](https://bugs.webkit.org/show_bug.cgi?id=329066)): The defaults of `LOG_DISABLED`, `ASSERT_MSG_DISABLED`, `ASSERT_ARG_DISABLED`, `BACKTRACE_DISABLED`, `FATAL_DISABLED` and `ERROR_DISABLED` move from `Assertions.h` to `PlatformEnable.h`. - [`bbfbc41dc9`](WebKit/WebKit@bbfbc41dc9) ([327419](https://bugs.webkit.org/show_bug.cgi?id=327419)): The `RELEASE_LOG*` macros are defined once, on top of a `WTF_RELEASE_LOG_SEND` per backend. `LOGF`, `SD_JOURNAL_SEND` and `LOG_ANDROID_SEND` are removed. - [`3757158274`](WebKit/WebKit@3757158274) ([328861](https://bugs.webkit.org/show_bug.cgi?id=328861)): `allowCompactPointers<T>()` looks at the `AllowCompactPointers<T*>` specialization first. `WTF_ALLOW_COMPACT_POINTERS_TO_INCOMPLETE_TYPE(JSC::JSCell)` moves from `MicrotaskQueue.h` to `JSCJSValue.h`. - [`2b85a5e547`](WebKit/WebKit@2b85a5e547) ([328859](https://bugs.webkit.org/show_bug.cgi?id=328859)): `printInternal(PrintStream&, String&)` is defined in `PrintStream.cpp`, not inline. - [`38418ba096`](WebKit/WebKit@38418ba096) ([328643](https://bugs.webkit.org/show_bug.cgi?id=328643)): `NeverDestroyed` and `LazyNeverDestroyed` read their storage through `std::launder`. - [`701620c269`](WebKit/WebKit@701620c269) ([328791](https://bugs.webkit.org/show_bug.cgi?id=328791)): `Vector::appendRange(Range&&)` for a sized range. The rest of the commit is WebCore. - [`272972fe8c`](WebKit/WebKit@272972fe8c) ([328652](https://bugs.webkit.org/show_bug.cgi?id=328652)): `WTF::initializeObjectIdentifierDomain()`. Only WebKit's GPU process calls it. - Fixes in headers that nothing in JavaScriptCore uses: [`961981e9e8`](WebKit/WebKit@961981e9e8) (`MediaTime` ordering), [`2c813011e9`](WebKit/WebKit@2c813011e9) (`WeakListHashSet` const iterator), [`3e90cc907c`](WebKit/WebKit@3e90cc907c) (`LoggingHashMap`), [`474bb3fa62`](WebKit/WebKit@474bb3fa62) (`BitSet::setEachNthBit` asserts `n`). ### bmalloc / libpas Bun's JavaScriptCore allocates through mimalloc, and the ASAN builds through the system allocator, so none of these changes what a shipped `bun` does. - [`c8ce68fbaa`](WebKit/WebKit@c8ce68fbaa) ([328247](https://bugs.webkit.org/show_bug.cgi?id=328247)): Each page config of a heap has its own minimum of objects per page. `PAS_MIN_OBJECTS_PER_PAGE` is renamed `PAS_DEFAULT_MIN_OBJECTS_PER_PAGE`, and `PAS_MAX_OBJECT_SIZE` takes the minimum as an argument. - [`37e9021372`](WebKit/WebKit@37e9021372) ([328674](https://bugs.webkit.org/show_bug.cgi?id=328674)): Medium segregated pages take objects up to 18432 bytes (from 11776). - [`ad3ad2c6ea`](WebKit/WebKit@ad3ad2c6ea) ([328277](https://bugs.webkit.org/show_bug.cgi?id=328277)): Free small segregated pages stay committed 450 ms longer (900 ms on macOS). - [`1ae8574624`](WebKit/WebKit@1ae8574624) ([328282](https://bugs.webkit.org/show_bug.cgi?id=328282)): Four more size classes in the designated intrinsic heap: 112, 144, 272 and 384 bytes. ### Build system - [`9c5e8c5fc1`](WebKit/WebKit@9c5e8c5fc1) ([328481](https://bugs.webkit.org/show_bug.cgi?id=328481)): CMake Debug builds compile bmalloc and `FastMalloc.cpp` with `-O3`, except with MSVC and clang-cl. - [`3c41f3fb2f`](WebKit/WebKit@3c41f3fb2f) ([329015](https://bugs.webkit.org/show_bug.cgi?id=329015)): Symbols that only offlineasm output references get `REFERENCED_FROM_ASM`, for GCC LTO. ### Nothing for a JSCOnly build (20) `2fe794df22`, `a46f812bfd`, `0ae4c9d112`, `da8e478d08`, `1d91ff5512`, `ac53330cbd`, `319a9b6c84`, `8655634bf3`, `e381818e3a`, `280384b995`, `933708cada`, `3b48f12459`, `17d262face`, `4e2a07134b`, `d71f3d702b`, `500be61de4`, `ad490a9bef`, `58ebe7bde3`, `8bd119f86f`, `48fd938fe4`: web preferences, sandbox profiles, the Xcode project, Swift bridging, GLib and GTK/WPE build files. </details> <!-- robobun:evidence:begin --> --- **no test proof** · iteration 7 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/bundler/compile-aot-size.test.ts, test/bundler/bundler_bytecode_portable.test.ts <!-- robobun:evidence:end -->
…4 more sites (oven-sh#47498) ### What does this PR do? Two changes to Bun's C++ that make the binary smaller. One commit each. No behaviour changes. **`defaultGlobalObject(JSGlobalObject*)`.** It is inlined at 1,431 places in a linux-x64 release build, 129,954 B in total. Each copy carries the loop over the class chain from `dynamicDowncast` and the thread-local read of the fallback. Now the inline part only compares the class with `Zig::GlobalObject::info()`. A subclass, another kind of global object or null goes to `defaultGlobalObjectSlow`, which has the old body. Estimated saving from the sizes of the inlined copies: 76 KB. **`Bun::putDirectNamed`.** The helper already has 409 callers. 114 more sites that wrote `object->putDirect(vm, Identifier::fromString(vm, "name"_s), value)` now call it. Each of those sites created and released an `Identifier` inline. The declaration moves from `ZigGlobalObject.h` to `root.h` so that every caller sees it. Not sized on its own. ### How did you verify your code works? Both changes were part of a stripped linux-x64 release build at 69f4077 together with other changes. On that build a sample of 58 test files had the same results as without them. The counts and sizes above are from that commit. This branch itself has not been built locally.
…n once (oven-sh#47490) ### What does this PR do? Four pieces of Rust code were compiled into every caller. Each is now compiled once and called. One commit per change. No behaviour changes. | Change | Where | Size | |---|---|---| | A lint `Report` is a `Box` around its fields, so the builder calls in a rule move 8 bytes. A message is passed as `&'static Message`, and the rules declare `const NAME: &Message = &Message::new(id, text)`. `Report::drop`, the owned copy in `Fixer`, borrowed `data` values and the `Options` / `Object` getters are no longer inlined into the rules | `src/lint` (nearly all of the diff is the two `&` in each declaration) | -385,024 B | | The first-use path of `File::parents`, `owners` and `known_kinds`, the part of `Checker::filter` and `map_type_ex` that does not depend on the closure, a few other miss paths of the type checker, and `CursorRegion::enter` / `exit` in the formatter | `src/sema`, `src/format/cursor.rs` | -118,784 B | | `sys::Error` to `SystemError` to JS error, and `ErrorCode::throw` | `src/jsc/SystemError.rs`, `src/sys_jsc/lib.rs`, `src/jsc/ErrorCode.rs` | about -88 KB of functions | | The loop in `walk::<R>` that submits what a markdown rule reported does not depend on the rule | `src/lint/markdown/listen.rs` | about -18 KB of functions | The first change costs one heap allocation per reported lint problem. The others add a call on a path that runs once per file, on a cache miss or when an error is thrown. ### How did you verify your code works? The sizes are from stripped linux-x64 release builds at 69f4077, with the changes added one after the other. On that build a sample of 58 test files had the same results as without the changes, and instruction counts of `bun lint`, `bun check`, `bun fmt`, `bun build` and `bun test` runs stayed within run-to-run noise. Those builds had `Message` as a wrapper around the reference instead of the plain reference. This branch has only been type-checked locally (`cargo check` on the lint crates).
… first (oven-sh#47512) ### What does this PR do? On x86-64 a field within 127 bytes of the start of a struct is reached with a one-byte offset, a field further away with four bytes. `Checker`, `Files`, `OwnStore`, `hir::FileIn`, `Lexer`, `Parser`, `Formatter` and `Tracker` are large and are accessed all over their crates. They are laid out as declared (`#[repr(C)]`), with the small fields that most accesses touch first. No field is added or removed. Linux-x64 release binary, stripped: -155,648 B. `Parser` grows by 8 bytes. On arm64 the encoding is the same either way, so nothing is saved there. The order now has to be kept by hand: a new field goes at the end unless it is small and hot. ### How did you verify your code works? The change was part of a stripped linux-x64 release build at 69f4077. On that build a sample of 58 test files had the same results as without it, and instruction counts of `bun lint`, `bun check`, `bun fmt`, `bun build` and `bun test` runs stayed within run-to-run noise. This branch itself has not been built locally.
### What does this PR do? `BunGlobWalker` is a generic walker with its parameters filled in. Its methods are instantiated in each crate that uses it (the bundler, the linter, the runtime), because `-Zshare-generics` only reuses an instantiation from an upstream crate and these crates do not depend on each other. `bun_glob::walk_bun_glob` calls `walk()` inside `bun_glob`, so the instantiation exists upstream of all of them and each links to that copy. Nothing needs to call the new function. Function sizes of the walker in a linux-x64 release build: -23,750 B. ### How did you verify your code works? The change was part of a stripped linux-x64 release build at 69f4077. On that build a sample of 58 test files had the same results as without it, and instruction counts of `bun lint`, `bun check`, `bun fmt`, `bun build` and `bun test` runs stayed within run-to-run noise. This branch itself has not been built locally.
* commit '7b573c05d0c52dffc67d4d25daf108bed5930cf2': fix(parser): keep decorated private assignment targets as references (#217)
|
Hi! I'm the It looks like you correctly set up a CI job that uses the autofix.ci GitHub Action, but the autofix.ci GitHub App has not been installed for this repository. This means that autofix.ci unfortunately does not have the permissions to fix this pull request. If you are the repository owner, please install the app and then restart the CI workflow! 😃 |
Known-bounds copies can inline under _FORTIFY_SOURCE=2, leaving no __memcpy_chk import. Keep exact stack-protector and __snprintf_chk requirements as hardening witnesses without requiring the optimizer to preserve an out-of-line memory-copy check. Exercise the verifier CLI with accepted and missing-import controls. The existing hardened arm64 compiler passes without any binary change; the ordinary compiler still fails the checked-formatting requirement.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change
Merge upstream Bun into the OpenClaw fork while retaining upstream and fork history. Resolve all 84 merge conflicts and preserve the fork's runtime, transport, module-hook, worker, path, and verified engine-artifact contracts across upstream's AOT and typed-binding changes.
Frozen inputs:
34cbc70fa1ddb580aa34c2ca624d19ea23f9bbce191755ec115ece3dd642eba3cb0143cecb5e5d7a, followed by landed decorator fix fix(parser): keep decorated private assignment targets as references #217 at7b573c05d0c52dffc67d4d25daf108bed5930cf2bace183eb495961bd69ac59d1a45a5f78b32e564; landed-fix merge:b691e0e7f014bdc39d7279acc8040c8618b8cb26Both frozen inputs are ancestors of this branch. Land with a merge commit to retain that history. The retained automation branch follows the documented sync-recovery route.
Reconciliation includes the UTF-16/JsScope node:path port, AOT HTTP response/transport ownership, compiled module-hook admission, worker and async-context contracts, and the Windows regression selector. Source-only type/generator jobs now check out the exact fork engine commit from the manifest; native builds retain complete SHA-256-verified fork archives with no upstream fallback.
Carry Peter Steinberger's hardened GNU verifier fix from the overlapping AOT stack in #208, preserving his original authorship and message from 62fdec7. Fortified copies can inline without
__memcpy_chk; exact stack-protector and checked-formatting requirements remain.Validation
Prior-head evidence at
2f741a3dac3f907396e4121f878fbf64304555ab:git diff --check; preserved upstream whitespace findings in 67 imported files were verified byte-identical to the frozen upstream.At current head
dd5eed75f82e5972e3a3928032bbede7eafd2795, the three verifier controls are separate test cases, preserving all assertions and the default five-second limit. This fixes the combined case's native ASan timeout. Independent review of that test change found no P0–P2 issues.The upstream port is open at oven-sh#47550. A native Linux x64 debug/ASan build at frozen upstream
34cbc70franbun bd -j4 test test/internal/build-hardened-flags.test.ts: original import policy 13 passed / 1 failed at the accepted-copy case; fixed policy 14 passed / 0 failed / 151 assertions. Prettier and build-script TypeScript checking passed. Both changed files exactly match this sync's files. That result qualifies the upstream verifier port; it does not qualify a native build of this fork or a hardened release binary.At this same head, formatting and source lints pass. Lint passes JavaScript lint and build/CI TypeScript checking before failing on missing engine AOT generator sources; bun-types reaches the same generator gap. Native builds and Rust setup stop on missing AOT archive checksums before compilation. These are setup failures, not runtime test results.
The fork’s SQLite diagnostics from #212 are also submitted upstream at oven-sh#47557, with native/AOT proof recorded there. That port’s loader, native-hook classification, and Node disposal corrections are not yet included in this pushed head.
Remaining qualification
This PR remains draft. The committed engine manifest still pins
b82bc917e99110b61c3a21202d6def7457aa194cand contains no AOT archives or generators. Linux/Darwin native builds and Rust setup stop before Bun compilation; generator/type jobs stop on missing engine AOT sources. These failures do not qualify the reconciled runtime.The producer selection at engine
42e4245f4180941634b482a7e8ef7fca47467de5still builds non-AOT twins. The sync requires eight AOT build/release variants, plus the Linux x64 debug-ASan AOT archive for the Rust setup as currently implemented.BUN_WEBKIT_GENERATORS_ONLYis exported but has no build owner at this source head. The existing canonical alternative is a verifiedbun-webkit-aot-sourcespayload plus a repaired generator-only consumer. The exact required labels, optional musl targets and producer/consumer evidence are recorded in openclaw/WebKit#34 (comment). This selection gap must be resolved before the combined-head qualification can pass.Paired engine work is tracked in openclaw/WebKit#34. Qualify the required engine variants and update the source pin/checksums together, then rerun combined-head native builds, affected runtime/AOT regressions, Rust and generated-type gates, and the six-platform Bun dry run. Retained-run/Darwin changes and the qualification harness in the separate AOT stack remain under review; this sync does not claim they have all migrated.
Consumer integration follows qualification: update and validate the shared six-platform runtime in
openclaw/openclawand the Darwin ARM64/Linux x64 toolchain inopenclaw/birdclaw, including their documented Node/Bun replay, installer, package and runtime acceptance gates. No consumer pin or Bun release has been changed by this PR.Reviewed port follow-ups prepared locally
The pushed sync head above is unchanged. Four signed local checkpoints retain the SQLite loader/hook/disposal corrections, optional-trace guards and published types, the hardening fixture repair, and the generator-only WebKit consumer repair. Upstream
34cbc70fa1ddb580aa34c2ca624d19ea23f9bbceand landed fork commit7b573c05remain ancestors. The local sync head isc8530468f98bf51f38094aaf9ee60c93d08e5a4f; these follow-ups are not yet part of this PR's pushed head.Local commit
c8530468f98bf51f38094aaf9ee60c93d08e5a4ffreezes the existingBUN_WEBKIT_GENERATORS_ONLYflag through ninja regeneration, selects the checksum-pinnedbun-webkit-aot-sources.tar.gzwithout native outputs or configure-time archive acquisition, enables clone-only Rust setup, and rejects native/default/runtime targets in that setup. The existing source-only--mode=codegencontract and Rust output paths are preserved. Fresh independent static review completed with no actionable P0–P3 findings, andgit diff --checkpasses. Added regressions and the real Rust setup replay have not run: the source payload and native engine qualification remain pending. This checkpoint is not pushed and does not satisfy the native test gate.b204527216a209b1f73ad62dd773772a56d64577: fresh native Debug/ASan verifier tests pass 14/14 with 151 assertions, including the path-independent shell fixture.f9a7311c0f0ab356038cc3f96cc2a9c5b0a82aa2: native SQLite/diagnostics tests pass 171/171 with 900 assertions; exception checks, generated/source/packed types, regular/AOT executables and the real lazy-loaderSQLITE_OMIT_TRACEquery/close/dispose smoke pass. The native implementation, published overrides and public type fixture in the local sync match that reviewed port.Those upstream-port results qualify their pinned upstream engine context. Full fork qualification and pushing the prepared integration still require the missing AOT archives/generator outputs from the existing WebKit producer handoff. Keep the eventual sync merge history-preserving; no squash/rebase merge.