Skip to content

[JSC] A collection hint that the heap can turn down - #1085

Draft
robobun wants to merge 1 commit into
mainfrom
robobun/768fe9fd/collect-async-if-allocated
Draft

robobun wants to merge 1 commit into
mainfrom
robobun/768fe9fd/collect-async-if-allocated

Conversation

@robobun

@robobun robobun commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • Bun's GC timer calls Heap::collectAsync() every second, and the heap runs every request. setTimeout(() => {}, 12000) gets 10 Eden collections that free nothing.
  • Bun cannot decide it with a byte count. A cell is counted when its allocator has used up a free list, and a small object can own memory that nothing counts. Only the heap knows of the Full collection that an Eden collection left owing (m_shouldDoFullCollection).

Fix

  • Heap::collectAsyncIfAllocated() calls collectAsync() only if a cell was allocated since the last collection ended, or if a Full collection is owed. It returns whether it asked.
  • The heap already keeps the first fact. MarkedSpace::isEdenEmpty() reads the eden bits of the block directories and the nursery of the precise allocations. No allocation path changes.
  • Correct because an Eden collection sweeps exactly those blocks and allocations, so any other request would sweep nothing.
  • Verified: two JSTests in five configurations. Each of the four clauses fails a test when it is removed (Notes). Bun's GC tests run on this change (GC timer: collect at a tick only when JSC has something to collect bun#49398).

Background

  • A request without a scope lets the collector choose Eden or Full. A Full collection is owed after an Eden one that left little room, and after any collection but a Full one while memory use is over criticalGCMemoryThreshold.
  • A block gets its eden bit when an allocator sweeps it to a free list. A precise allocation is appended behind m_preciseAllocationsNurseryOffset. prepareForAllocation() resets both at the end of every collection.
  • Considered a flag that every allocation slow path sets (the earlier version of this PR): one more store per slow path in every program, and a second record of what the eden bits already say.

Downsides

  • A hint that the heap turns down walks the block directories: 102 to 126 ns in Bun's release build, once per tick of Bun's timer.
  • In a heap that collects only Full (no generational collector, a mini VM), a hint with nothing allocated would have run a Full collection, which can free an old object. The heap turns that hint down too.
Notes
  • Heap::collect(), the allocators and the code of every other request are as before. The change in heap/ is 44 added lines: the entry point, MarkedSpace::isEdenEmpty(), and Heap::wantsFullCollection(), which Collector::decideCollectionScope() now shares with the hint.
  • What an Eden collection sweeps: BlockDirectory::snapshotUnsweptForEdenCollection() adds the eden bits to the unswept bits, and the precise allocations are swept from m_preciseAllocationsNurseryOffsetForSweep. MarkedBlock::Handle::specializedSweep() sets a block's eden bit when it sweeps to a free list, which is the only way an allocator gets a block. BlockDirectory::prepareForAllocation() clears the bits.
  • A cell that the mutator allocates while a collection runs is that collection's: it is marked when the collection ends, or it is free. So the boundary is the end of the last collection, which is where the heap resets its own record.
  • Who may read the bits: the thread with heap access (BlockDirectory::assertMayAccessBitsUnlocked()). The hint runs on the mutator thread.
  • Cost of a hint, Bun release build, 1,000,000 calls in a loop after a Full collection: 102 to 109 ns each when the heap turns it down, 119 to 126 ns after node:http, node:crypto, node:zlib, bun:sqlite and a server were loaded. 17 ns each when a collection is already requested. With only this change the binary's text has the same size as before (155,785,633 bytes): Heap::didAllocate() and LocalAllocator::allocateSlowCase() are byte for byte as long as they were.
  • Each clause, removed one at a time in a local build:
removed what fails
the nursery test in isEdenEmpty() "a hint after a precise allocation", in the 3 runs of the first test
the eden bits in isEdenEmpty() "a hint after one object", in all 5 runs
!lastWasFull "a hint after a collection", in the run over the critical threshold
the owed Full collection "a hint after an Eden collection that left a Full one owing", in the 3 runs of the first test
  • JSTests/stress/bun-collect-async-if-allocated.js has three runs. Two use --minEdenToOldGenerationRatio=2, so that every Eden collection leaves a Full one owing: one with the concurrent collector and one without. The third is over the critical memory threshold (--criticalGCMemoryThreshold=0.000001). The precise allocation in it is the butterfly of an array that grows to 50,000 elements. Nothing else is allocated then.
  • JSTests/stress/bun-collect-async-if-allocated-full-only.js runs without the generational collector and in a mini VM. A heap that collects only Full owes no collection for that alone. It also has the cell whose slow path runs the collection that the hint queued: the cell is allocated after that collection, so the next hint is a yes.
  • 100 of 100 runs of each of the 5 configurations pass on a release build of the jsc shell. Both tests are listed in JSTests/BUN-TEST-DIFFERENCES.md.
  • The jsc shell gets collectAsyncIfAllocated() for the tests, next to idleFullGC().
  • Not used: m_hasAllocatedSinceLastCollection. Outside compiled images it is set again after every collection, and in compiled images it also turns on the shortcuts of an Eden collection with an empty Eden.
  • Neighbours. [JSC] Cells get an allocation limit of their own, which extra memory raises by no more than the cells do #1072 adds a second reason for m_shouldDoFullCollection, and the hint runs that collection too. [JSC] The marking helper threads join a collection once it has shown them enough work #800 and [JSC] A marking thread pays for the wake-up of another one with marking, and an idle helper leaves the collection #838 make a collection wake fewer helper threads. They lower the cost of each collection, and this PR removes the collections that have nothing to do.
  • History. The first version put the test inside Heap::collect(), gave the hint a byte floor, and kept a flag that a collection cleared as it began. A pass over the free lists kept that flag exact while a collection ran. The second version moved the pass to MarkedSpace::stopAllocating(). With the first version one test of Bun failed on Windows x64 only (bun-serve-html-build-holds-server.test.ts, 5 of 5 runs). With the byte floor, Bun made no collection when that test's server started to listen. The cause on Windows is not known, and Bun asks at listen as before since the second version.

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown

Preview build of cc0d0fd: autobuild-preview-pr-1085-cc0d0fd2

@robobun
robobun force-pushed the robobun/768fe9fd/collect-async-if-allocated branch 2 times, most recently from db66f73 to fbd9946 Compare November 21, 2026 05:27
@robobun robobun changed the title [JSC] A collection hint that the heap can turn down, and a notice when a quiet heap grows [JSC] A collection hint that the heap can turn down, and a notice when a quiet heap allocates again Nov 21, 2026
@robobun
robobun force-pushed the robobun/768fe9fd/collect-async-if-allocated branch from fbd9946 to bf97c96 Compare November 21, 2026 12:09
@robobun robobun changed the title [JSC] A collection hint that the heap can turn down, and a notice when a quiet heap allocates again [JSC] A collection hint that the heap can turn down Nov 21, 2026
@robobun
robobun force-pushed the robobun/768fe9fd/collect-async-if-allocated branch 2 times, most recently from 3ba7fe9 to 1a8b7a6 Compare November 21, 2026 13:48
@robobun
robobun force-pushed the robobun/768fe9fd/collect-async-if-allocated branch from 1a8b7a6 to 80e1b16 Compare November 21, 2026 20:13
robobun added a commit to oven-sh/bun that referenced this pull request Nov 21, 2026
The preview adds Heap::collectAsyncIfAllocated() to 499846f18f75. The
pin moves to the merged commit when that pull request lands.
Bun's GC timer asks for a collection once a second, and once every 30 s
in a program that has gone quiet, in case the program has stopped
allocating. It asks with Heap::collectAsync(), which runs every request.
A program that allocates nothing gets one Eden collection per request:
it frees nothing and wakes the marking threads.

Heap::collectAsyncIfAllocated() is collectAsync() for such a hint. It
asks for the collection only if the heap has something to do:

- A cell was allocated since the last collection ended. The heap already
  keeps that: a block that an allocator takes has its eden bit set, a
  precise allocation is appended to the nursery, and
  prepareForAllocation() resets both at the end of a collection. An Eden
  collection sweeps exactly those blocks and that part of the list. So
  MarkedSpace::isEdenEmpty() reads them, and no allocation path changes.
- Or a Full collection is owed. After an Eden collection that left
  little room the heap makes the next collection a Full one
  (m_shouldDoFullCollection), and over the critical memory threshold
  every collection that has no scope is a Full one. Only the next
  request runs it, also with nothing allocated. After a Full collection
  none is owed.

Any other request would run as an Eden collection with nothing to sweep.
It returns whether it asked.

A byte count cannot say that no cell was allocated: a cell is counted
when its allocator has used up a free list, and a small object can own
memory that no count has.

Heap::collect() and the code of every other request do not change.

The jsc shell gets collectAsyncIfAllocated() for the tests.

Tests: JSTests/stress/bun-collect-async-if-allocated.js
       JSTests/stress/bun-collect-async-if-allocated-full-only.js
@robobun
robobun force-pushed the robobun/768fe9fd/collect-async-if-allocated branch from 80e1b16 to cc0d0fd Compare November 21, 2026 23:41
robobun added a commit to oven-sh/bun that referenced this pull request Nov 22, 2026
The preview adds Heap::collectAsyncIfAllocated() to 499846f18f75. This
commit is here so that CI can run. When that pull request has merged,
the pin comes from main and this commit goes away.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant