Repository navigation
Free-list save: stop drawing after 8 rounds, derive the round budget (ADR-0044) - #175
Conversation
…(ADR-0044) The save loop was limited to 64 rounds. The termination proof (SPEC 05 GC-13) bounds the rounds by the pool sizes and the committed GC pages, not by a constant, so a valid state could in principle panic the commit, and a retry of the same batch would likely panic again. After 8 rounds the save now allocates from loose pages or by extending the file, and no longer from the drain pool or the annex pool. Draws are the only way one round makes the previous round's records stale, so the save then ends within a bounded number of rounds (GC-13a), with a file a few pages larger. The remaining guard is computed at entry from the proof's quantities and is never below the old limit. `just save-no-draw`, also a CI job, builds with the threshold at 0 and at 1 and runs the engine's tests and the crash smoke on that path. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
`run` returns `ExitCode`, which is itself `#[must_use]`. Clippy in Rust 1.100 reports the attribute as `double_must_use`, and CI runs clippy with `-D warnings` on the current stable toolchain. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 4 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Once the save has stopped drawing (GC-13a), an allocation the pools could serve extended the file instead. With the file at the map's end that returned `MapFull` and failed a commit that succeeded before. In that case the save now draws from the pools as it does in its first rounds; it then ends by GC-13 and the round budget. `save_at_map_limit.rs` covers it: under `just save-no-draw` it fails without the fallback. Also corrects comments that still described the save's allocation mode as fixed for the whole procedure. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Follow-up commit, after two independent checks of the first one. Independent review. A second agent, given the code and the three claims but not my reasoning, tried to break them (termination after the switch, the budget as a true bound, no leak or double listing across the switch). It could not, and verified each step against the code. It found one real gap and several stale comments:
Fuzz. An agent's review is not a substitute for a human who knows LMDB's free list. The parts that most need one are the GC-13 budget paragraph and GC-13a in Re-run locally after this commit: fmt, clippy, |
Problem
freelist_save(commit step C1) loops until the free-page records stop changing. Release builds limited it to 64 rounds with anassert!. The termination proof (SPEC 05 GC-13) bounds the rounds by the pool sizes and the number of committed GC pages, which can be thousands, so 64 did not follow from it. A valid state could in principle panic the commit. No data would be lost (nothing is written before C2), but a retry of the same batch would likely panic again: for Meilisearch, an index that cannot be written.This has never been observed. With a temporary counter,
cargo test --workspace, the crash harness (200 and 50 multi-process cycles) and the reader stress tests needed at most 3 rounds. The fuzz targets were not measured.Change
ADR-0044, Option C with Option B's budget:
G + 3rounds (G= committed GC pages). The argument is the new rule GC-13a. The ids left in the pools stay listed and are reusable by later transactions; the cost is a file a few pages larger.3·(P + D + A) + G + 64rounds, computed at entry (GC-13). It is never below the old limit. It is still a panic.Tests
No test reaches round 9 at the default threshold, so the new path gets its own run:
just save-no-draw, also a CI job. It builds with--cfg zerodb_save_draw_rounds="0"and="1"(no round draws; only the first round draws) and runs the tests ofzerodb-coreandzerodband 200 crash cycles. Those tests check contents and page accounting (every page reachable or listed free, never both).Two tests are skipped in that run, by name in the recipe, because they assert a file size and page numbers that hold only when the save reuses pool pages:
over_cap_freed_set_spills_whole_to_gc_treeanddelete_rebalance_logical_images_match_golden. They run unchanged in the normal suite.Checks run locally (macOS arm64)
cargo fmt --all -- --check,cargo clippy --workspace --all-targets -- -D warnings: cleancargo test --workspace: all passjust crash-test-quick,just loom,just save-no-draw: passjust stress,just bench-ab. The allocator's hot path gains one empty match arm; I make no performance claim.For the reviewer
🤖 Generated with Claude Code