Repository navigation
bazel: sweep only generations that changed - #42
Merged
Merged
Conversation
The collector swept every generation each minute: full listings of its index, objects, receipts, sessions, and markers, a read of each entry, and a usage upsert, then a listing of every Bazel object. Almost every pass recomputed what the last one had found, and with forty generations that was the largest periodic cost in the service. Each store operation that writes a generation's objects now marks it as the operation ends, and wakes the collector. A pass sweeps only the marked generations, no sooner than a minute after the last pass, so a build's writes share sweeps as before while a lone write is observed at once. Management rereads the targets after it creates a Bazel cache or changes a capacity or an organization's limit, and only generations left above their target are swept. With nothing marked, the collector issues no storage request and no statement. A pass over everything remains, one minute after start and then hourly. It covers what this process never saw: another instance's writes, manual changes, and targets moved by other products' usage. It is also the timer for the two-hour graces and for retired generations, which are minimum ages rather than deadlines. Access markers mark nothing: a swept generation is within its target, so a marker alone evicts nothing and changes no usage.
The collector read retention targets and recorded usage through runtime query functions, so a schema change could only fail them when a sweep ran. They now go through the query macros and are described in the offline metadata with the rest of the server's statements. The retention view's columns are marked non-null by hand because Postgres reports every view column as nullable.
Uzaaft
force-pushed
the
perf/bazel-sweep
branch
from
November 18, 2026 16:24
a99c7a0 to
87fbd59
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.