From 7fe8dee4847f5b3a0ca72883956730779e1b3a59 Mon Sep 17 00:00:00 2001 From: Quentin de Quelen Date: Thu, 12 Nov 2026 20:25:16 +0100 Subject: [PATCH 1/3] Free-list save: stop drawing after 8 rounds, derive the round budget (ADR-0044) The save loop was limited to 64 rounds. The termination proof (SPEC 05 GC-13) bounds the rounds by the pool sizes and the committed GC pages, not by a constant, so a valid state could in principle panic the commit, and a retry of the same batch would likely panic again. After 8 rounds the save now allocates from loose pages or by extending the file, and no longer from the drain pool or the annex pool. Draws are the only way one round makes the previous round's records stale, so the save then ends within a bounded number of rounds (GC-13a), with a file a few pages larger. The remaining guard is computed at entry from the proof's quantities and is never below the old limit. `just save-no-draw`, also a CI job, builds with the threshold at 0 and at 1 and runs the engine's tests and the crash smoke on that path. Co-Authored-By: Claude Fable 5.2 --- .github/workflows/ci.yml | 19 ++- CHANGELOG.md | 7 + CONTRIBUTING.md | 1 + crates/zerodb-core/Cargo.toml | 8 +- crates/zerodb-core/src/rwtxn/alloc.rs | 8 +- crates/zerodb-core/src/rwtxn/freelist.rs | 60 ++++++-- docs/DECISIONS.md | 1 + docs/SPEC/05-gc.md | 51 ++++++- docs/adr/0044-freelist-save-round-bound.md | 159 +++++++++++++++++++++ justfile | 16 +++ 10 files changed, 308 insertions(+), 22 deletions(-) create mode 100644 docs/adr/0044-freelist-save-round-bound.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 28f87668..4026f7c7 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -1,7 +1,8 @@ # ZeroDB CI — the CONTRIBUTING.md check battery, automated. # # Per push / PR: fmt, clippy, tests (x86-64 and aarch64), an MSRV check, miri -# on the core, loom on the reader table, the fuzz gate and the crash smoke. +# on the core, loom on the reader table, the free-list save without pool +# draws, the fuzz gate and the crash smoke. # Nightly: the long fuzz, the full crash run and the reader stress. # The loom, fuzz, crash and stress jobs run the justfile recipes of the same # name, so `just ` locally runs exactly what CI runs. @@ -115,6 +116,22 @@ jobs: - run: cargo install just --version 1.58.0 --locked - run: just loom + save-no-draw: + name: free-list save without pool draws + if: github.event_name != 'schedule' + runs-on: ubuntu-24.04 + timeout-minutes: 60 + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false + - uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # master (2026-10-01) + with: + toolchain: stable + - uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2.9.3 + - run: cargo install just --version 1.58.0 --locked + - run: just save-no-draw + fuzz-quick: name: fuzz gate (${{ matrix.os }}) if: github.event_name != 'schedule' diff --git a/CHANGELOG.md b/CHANGELOG.md index 95e2f556..67e9d678 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -183,6 +183,13 @@ Ratios are ZeroDB time ÷ LMDB time on Graviton4 unless stated; details in ### Fixed +- **A commit no longer fails when the free-list save converges slowly** + (ADR-0044). The save loop was limited to 64 rounds, a number its + termination proof did not give; past it the commit panicked, and a retry + of the same batch was likely to panic again. After 8 rounds the save now + stops reusing free pages and extends the file by a few pages, which ends + it. The remaining guard is computed from the proof (SPEC 05 GC-13, + GC-13a). No test, crash or stress run has needed more than 3 rounds. - `LeafRef::new` and `BranchRef::new` reject a buffer shorter than the page size, and `LeafMut::lookup` bounds-checks its reads. Both could read out of bounds from safe code when called directly; the engine always passed whole diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 62ecc449..a1c6fd60 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -131,6 +131,7 @@ cargo +nightly miri test -p zerodb-core # non-mmap logic must pass miri just fuzz-quick # differential fuzz against LMDB just crash-test-quick # crash-consistency smoke just loom # reader-table model check, when touching readers or nested txns +just save-no-draw # free-list save without pool draws, when touching the allocator or the save just stress # 180 s reader/writer stress, before an integration gate ``` diff --git a/crates/zerodb-core/Cargo.toml b/crates/zerodb-core/Cargo.toml index d4efc835..ca155002 100644 --- a/crates/zerodb-core/Cargo.toml +++ b/crates/zerodb-core/Cargo.toml @@ -29,5 +29,9 @@ proptest.workspace = true zerodb-io = { path = "../zerodb-io", features = ["test-backing"] } [lints.rust] -# `cfg(loom)` is set externally by the `just loom` recipe (ADR-0006). -unexpected_cfgs = { level = "warn", check-cfg = ["cfg(loom)"] } +# `cfg(loom)` is set externally by the `just loom` recipe (ADR-0006), and +# `cfg(zerodb_save_draw_rounds)` by the `just save-no-draw` recipe (ADR-0044). +unexpected_cfgs = { level = "warn", check-cfg = [ + "cfg(loom)", + 'cfg(zerodb_save_draw_rounds, values("0", "1"))', +] } diff --git a/crates/zerodb-core/src/rwtxn/alloc.rs b/crates/zerodb-core/src/rwtxn/alloc.rs index ba831362..62dc197e 100644 --- a/crates/zerodb-core/src/rwtxn/alloc.rs +++ b/crates/zerodb-core/src/rwtxn/alloc.rs @@ -223,9 +223,12 @@ pub(super) enum AllocMode { /// loose pages, the already-loaded drain pool, the carried annex pool or /// extend only (GC-12). Every request is a single page: GC records are /// always inline (GC-5), so the save never allocates an overflow run. - /// Set once at `freelist_save` entry and never cleared: the commit - /// consumes the txn. + /// Set at `freelist_save` entry; the commit consumes the txn. GcSave, + /// Inside `freelist_save` once its drawing rounds are spent (GC-13a): + /// loose pages or extend only. Neither pool is drawn from, so no record + /// already written can come to list a handed-out page. + GcSaveExtend, } impl<'env> RwTxn<'env> { @@ -330,6 +333,7 @@ impl<'env> RwTxn<'env> { return Ok(start); } } + AllocMode::GcSaveExtend => {} } let mp = map_pages(self.env.inner().map_size(), self.psize); if is_map_full(self.next_pgno, n, mp) { diff --git a/crates/zerodb-core/src/rwtxn/freelist.rs b/crates/zerodb-core/src/rwtxn/freelist.rs index 2b78d9f4..6ae9cc77 100644 --- a/crates/zerodb-core/src/rwtxn/freelist.rs +++ b/crates/zerodb-core/src/rwtxn/freelist.rs @@ -9,6 +9,13 @@ use super::*; struct SaveLoop { /// GC records awaiting their GC-20 rewrite. pending: BTreeSet, + /// Rounds of the outer loop so far, and their bound, which release + /// builds enforce too (GC-13 guard). Every round that does not exit + /// consumes a pool id, a committed GC page, or a loose fold; folds follow + /// record deletes, which follow pool draws or the drained records. The + /// budget counts each of those quantities generously. + outer_iters: usize, + outer_budget: usize, /// Rewrites done so far, and their bound, which release builds enforce /// too (GC-13 guard): /// each rewrite consumes one pending entry, and an entry only re-enters @@ -56,9 +63,22 @@ pub(super) fn merge_sorted_unique_into(a: &[u64], b: &[u64], out: &mut Vec) debug_assert!(out.windows(2).all(|w| w[0] < w[1]), "merge not ascending"); } -/// GC-13 guard: `freelist_save`'s loop provably terminates; a bug that -/// breaks the fixed point should panic, not hang. -pub(super) const FREELIST_SAVE_MAX_ITERS: usize = 64; +/// Rounds of [`RwTxn::freelist_save`] whose allocations may draw from the +/// drain pool and the annex pool (GC-12). Later rounds take loose pages or +/// extend the file, which removes the only way one round makes the records +/// of the previous one stale, so a save that converges slowly ends with a +/// slightly larger file instead of failing the commit (GC-13a, ADR-0044). +/// +/// `--cfg zerodb_save_draw_rounds="0"` or `="1"` (the `just save-no-draw` +/// recipe) lowers it for a test run, so that every save, or every save of +/// two rounds or more, takes the path that is otherwise almost never +/// reached. +#[cfg(not(any(zerodb_save_draw_rounds = "0", zerodb_save_draw_rounds = "1")))] +pub(super) const FREELIST_SAVE_DRAW_ROUNDS: usize = 8; +#[cfg(zerodb_save_draw_rounds = "0")] +pub(super) const FREELIST_SAVE_DRAW_ROUNDS: usize = 0; +#[cfg(zerodb_save_draw_rounds = "1")] +pub(super) const FREELIST_SAVE_DRAW_ROUNDS: usize = 1; impl<'env> RwTxn<'env> { /// GC-10 trailing shrink: loose pages that are the highest-numbered pages @@ -125,7 +145,9 @@ impl<'env> RwTxn<'env> { /// references it, TXN-62), so without them every commit would extend /// the file. An entry touched by a pool draw goes back on the `pending` /// rewrite set, and the loop only exits when no rewrite is pending, so - /// the final tree state never lists a handed-out page. + /// the final tree state never lists a handed-out page. After + /// [`FREELIST_SAVE_DRAW_ROUNDS`] rounds `alloc_mode` becomes + /// `GcSaveExtend` and the pools are no longer drawn from (GC-13a). /// /// **Termination (GC-13):** each iteration that does not exit strictly /// shrinks a finite pool (a drain-pool or annex-pool draw), grows `freed` @@ -135,8 +157,8 @@ impl<'env> RwTxn<'env> { /// after a record is deleted, which needs a record emptied by pool draws: /// so loose folds are bounded by the pool draws too. Nothing else frees /// a page: a put into a dirty leaf rewrites it in place or splits it - /// (one fresh page, nothing freed). `FREELIST_SAVE_MAX_ITERS` turns a - /// violation into a panic. + /// (one fresh page, nothing freed). The round budget computed at entry + /// from those quantities turns a violation into a panic. pub(super) fn freelist_save(&mut self) -> Result<()> { debug_assert!(self.alloc_mode == AllocMode::Normal); self.alloc_mode = AllocMode::GcSave; @@ -162,10 +184,20 @@ impl<'env> RwTxn<'env> { // Every entry the txn's operations drew from needs its GC-20 // rewrite at least once. let pending: BTreeSet = work!(self).drains.keys().copied().collect(); + let drain_ids = work!(self).drains.values().map(Drain::len).sum::(); + let committed_gc_pages = usize::try_from( + self.free_db + .branch_pages + .saturating_add(self.free_db.leaf_pages), + ) + .unwrap_or(usize::MAX); let mut st = SaveLoop { - inner_budget: pending.len() - + work!(self).drains.values().map(Drain::len).sum::() - + 16, + outer_iters: 0, + outer_budget: (pending.len() + drain_ids + work!(self).annex.len()) + .saturating_mul(3) + .saturating_add(committed_gc_pages) + .saturating_add(64), + inner_budget: pending.len() + drain_ids + 16, pending, inner_iters: 0, own: BTreeMap::new(), @@ -179,13 +211,15 @@ impl<'env> RwTxn<'env> { self.release_trailing_loose(); let work = work_mut!(self); work.freed.extend(std::mem::take(&mut work.loose)); - let mut iters = 0usize; loop { - iters += 1; + st.outer_iters += 1; assert!( - iters <= FREELIST_SAVE_MAX_ITERS, + st.outer_iters <= st.outer_budget, "freelist_save failed to reach a fixed point (GC-13)" ); + if st.outer_iters > FREELIST_SAVE_DRAW_ROUNDS { + self.alloc_mode = AllocMode::GcSaveExtend; + } // (a) apply_drains (GC-11a/GC-20). self.rewrite_pending_drains(&mut st)?; // (c) this txn's own list: the meta annex when it fits (GC-32), @@ -436,6 +470,8 @@ mod tests { let cap = gc_chunk_cap(PS) as u64; let mut st = SaveLoop { pending: BTreeSet::new(), + outer_iters: 0, + outer_budget: 0, inner_iters: 0, inner_budget: 0, own: BTreeMap::new(), diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index 30603657..43c8953b 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -41,6 +41,7 @@ Approved or accepted, and in `main`. | 0031 | [Hardware CRC32C for the meta checksum](adr/0031-hardware-crc32c.md) | Accepted (Option B), implemented; amends the `unsafe` policy | Performance (commit), `unsafe` policy | | 0034 | [Trust page contents by default, as LMDB does](adr/0034-trusted-pages-by-default.md) | Accepted | Page validation, LMDB parity, `unsafe` policy | | 0037 | [Bounded free-page records](adr/0037-bounded-free-page-records.md) | Accepted (Option C), implemented | On-disk format, page reclamation | +| 0044 | [The round limit of the free-list save has no derivation](adr/0044-freelist-save-round-bound.md) | Accepted (Option C), implemented | Page reclamation, commit | ## Shipped, approval not recorded diff --git a/docs/SPEC/05-gc.md b/docs/SPEC/05-gc.md index 44b1c7fd..6eeaacad 100644 --- a/docs/SPEC/05-gc.md +++ b/docs/SPEC/05-gc.md @@ -269,8 +269,12 @@ in-loop allocator so the loop cannot leak GC entries. release_trailing_loose(txn) freed_pgs += take(loose_pgs) - loop: # bounded: GC-13 + a hard - # iteration guard in release + round = 0 + loop: # bounded: GC-13 + the round + # budget, also in release + round += 1 + if round > DRAW_ROUNDS: # GC-13a: from here on, + alloc_mode = GC_SAVE_EXTEND # loose pages or extend only # (a) apply_drains, INSIDE the loop (GC-20): rewrite the remainder # of every pending entry / delete the fully-drained ones. # These puts/deletes may themselves (i) draw from the drain @@ -360,7 +364,8 @@ in-loop allocator so the loop cannot leak GC entries. txn's **already-loaded drain pool** — the in-memory remainders of entries drained during §6 reclamation — provided every entry so touched is **re-rewritten** (GC-20) before the loop's fixed point is declared, or - **(3)** file extend (`next_pgno`, GC-15). The anti-leak property is carried + **(3)** file extend (`next_pgno`, GC-15). After `DRAW_ROUNDS` rounds source + (2) is closed (GC-13a). The anti-leak property is carried by the fixed point, not by a draw ban: the loop may not exit while any touched entry awaits its rewrite, so the final tree state never lists a handed-out page (INV-22) and never drops an undrained one. Pool draws @@ -428,8 +433,44 @@ in-loop allocator so the loop cannot leak GC entries. a fixed point — no pending record rewrite, `freed_pgs` and the annex pool stable across the step-(c) writes, no unlisted loose page — in finitely many iterations. (Mirrors the fork's `mdb_freelist_save` retry loop with - `me_pghead` consumption.) A hard iteration guard turns a regression into a - panic. + `me_pghead` consumption.) + + **Round budget [ADR-0044].** The guard that turns a regression into a + panic is computed at entry from the quantities above, not a constant. With + `P` the drained records, `D` the ids in the drain pool, `A` the ids in the + annex pool and `G` the pages of the committed GC tree: a round that does + not exit either draws from a pool (at most `D + A` rounds), obsoletes a + committed GC page (at most `G`), or folds loose pages; a fold follows a + record delete, and there are at most `P` deletes of drained records and + `2·A` of own records (one per record emptied by annex draws, one per + re-key). The budget is `3·(P + D + A) + G + 64`, which is above that sum. + LMDB's loop has no guard. + +- **GC-13a — the save stops drawing after `DRAW_ROUNDS` rounds [ADR-0044].** + `DRAW_ROUNDS = 8`. From round `DRAW_ROUNDS + 1` on, `allocate()` inside the + save serves a request from loose pages or by extending the file, and never + from the drain pool or the annex pool (GC-12 sources 2 and the GC-30 pool + are closed). The ids left in the pools stay listed: a drained record keeps + its remainder (GC-20), and the annex remainder is carried into this txn's + own entry (GC-30). They are reusable by later transactions; the cost is a + file a few pages larger than it could have been. + + *Why the save then ends.* Without draws no record is re-marked pending, + and the annex pool no longer shrinks, so quantities (i) and (ii) are out + of the loop. Records are deleted only in the first round after the switch: + the drained records already empty, and the own records emptied by draws + made before it (with the re-key of record 0). From the second round on, + the ids of this txn's own records only grow, so there is no delete, hence + no merge and no new loose page; the only event that keeps the loop going + is the COW of a committed GC page during a put, and each one consumes + quantity (iii). The save ends at most `G + 3` rounds after the switch. + + The anti-leak and gate arguments of GC-12 are unchanged: the rule closes + sources and opens none, and extending is already an allowed source. + Reaching round `DRAW_ROUNDS + 1` has not been observed: the test suite, + the crash harness and the reader stress tests need at most 3 rounds. + `just save-no-draw` runs the engine's tests and the crash smoke with + `DRAW_ROUNDS` at 0 and at 1. **(v) Own-entry size guard [added 2026-10-19, issue #29; retired by ADR-0037].** It existed because a single own entry on an overflow run could diff --git a/docs/adr/0044-freelist-save-round-bound.md b/docs/adr/0044-freelist-save-round-bound.md new file mode 100644 index 00000000..1b3a33d6 --- /dev/null +++ b/docs/adr/0044-freelist-save-round-bound.md @@ -0,0 +1,159 @@ +# ADR-0044: The round limit of the free-list save has no derivation + +- Status: Accepted (Option C with Option B's budget, maintainer, 2026-11-12), implemented +- Related issue: none +- Date: 2026-11-12 + +## Context + +Commit step C1, `RwTxn::freelist_save` (`crates/zerodb-core/src/rwtxn/freelist.rs`), +writes the free-page database in a loop that runs until nothing changes: +rewriting a record can allocate a page, the allocation can draw a page from a +pool the records list, and the records must then be rewritten +([SPEC 05](../SPEC/05-gc.md) GC-11 to GC-13). + +GC-13 proves the loop ends. Every round that does not exit strictly reduces +one of four finite quantities: the drain pool, the annex pool, the committed +free-tree pages not yet copied, and the loose pages not yet folded in. + +The code holds two guards in release builds: + +| Guard | Limit | Where the limit comes from | +|---|---|---| +| Inner loop (`rewrite_pending_drains`) | pending records + ids in the drain pool + 16 | The quantities of the proof | +| Outer loop | `FREELIST_SAVE_MAX_ITERS = 64` | Nothing. The proof bounds the rounds by the four quantities, which can be thousands | + +The outer limit is therefore not implied by GC-13. A valid state could in +principle need more than 64 rounds, and the commit would panic. + +**What a panic costs.** Nothing has reached the disk before step C2, so no +data is lost and the previous snapshot stays intact (GC-14). But the save is +a function of the base snapshot and the transaction's operations: an +application that retries the same batch is likely to hit the same panic +again. For Meilisearch that is an index that cannot be written until someone +intervenes. + +**Prior art.** LMDB's `mdb_freelist_save` loops with no limit at all. An +earlier ZeroDB design did cycle for many rounds in one case (an own record on +an overflow run at a page-count boundary, issue #29); ADR-0037 removed that +case by making records inline and bounded. So the guard has caught a real +defect once, and a hang there is not hypothetical. + +**Measured.** With a temporary counter in `freelist_save` (not committed), at +commit `9115969` on macOS arm64: + +| Run | Largest number of rounds | Saves that took 3 rounds | +|---|---:|---:| +| `cargo test --workspace` | 3 | 43 | +| Crash harness, 200 cycles | 2 or fewer | 0 | +| Crash harness, multi-process, 50 cycles | 2 or fewer | 0 | +| `reader_stress` and `multi_process_stress`, 40 s each | 3 | 96 | + +Among the saves that took 3 rounds, the inner loop used at most 19 % of its +budget. The fuzz targets were not +run with the counter. + +So no run has come near 64. That is evidence, not a bound: the tests cover +the states someone thought of. + +## Options + +### Option A — Keep 64, record the evidence + +Add the table above to SPEC 05 GC-13 and say the limit is empirical. + +- For: no code change. +- Against: the one release-build panic in the commit path that the proof does + not cover stays as it is. + +### Option B — Derive the outer limit from the proof + +Replace the constant with a budget computed at entry, as the inner loop +does: pending records + ids in the drain pool + ids in the annex pool + +pages of the committed free tree (`branch_pages + leaf_pages` of its record) ++ loose pages + a constant. + +- For: a few lines. The panic then fires only if GC-13 is wrong, which is + what a guard is for. A valid state can no longer trip it. +- Against: if the loop has a defect that makes it slow to converge rather + than endless, the commit spins for thousands of rounds before the panic. + The retry problem above is unchanged when the guard does fire. +- Crash safety: unchanged; nothing is written before C2. + +### Option C — After a few rounds, stop drawing from the pools (recommended) + +Keep a small threshold (8 is proposed; the largest value seen is 3). When the +save passes it, allocations inside the save stop drawing from the drain pool +and the annex pool, and take loose pages or extend the file. Pool draws are +the only way a record written in one round is made stale by the next, so +without them the feedback stops: what remains is the copying of committed +free-tree pages, which is finite and feeds nothing back. The save then ends +a few rounds later, and the commit succeeds with a file a few pages larger +than it could have been. Option B's derived budget stays as the last guard. + +- For: a state that converges slowly costs a few pages of file growth, not a + failed commit. The pages left in the pools are not lost: they stay listed + and are reusable by later transactions. +- Against: a second allocation mode inside the save, reached almost never, + so it needs its own tests (a test-only threshold of 1 makes every save + take that path) and its own termination argument in SPEC 05. About 30 + lines of engine code. +- Crash safety: unchanged. Extending the file inside the save is already an + allowed outcome (GC-12). +- The argument that the save ends once draws stop is sketched here, not + proven. It must be written into GC-13 and reviewed before the code. + +### Option D — Return an error where the code panics + +`freelist_save` returns an error, the commit fails, and the transaction is +aborted. + +- For: the application sees a normal error. No unwinding through the commit. +- Against: it does not fix the retry problem; the batch fails every time, with + an error in place of a panic. The engine's LMDB-shaped error set has no + variant that fits (LMDB's `MDB_PROBLEM` is not among them), so a new one + is needed. +- It can be combined with B or C for the last guard. + +### Option E — Remove the limit, as LMDB + +- Against: a defect becomes a commit that never returns while holding the + writer lock. Rejected. + +## Decision + +**Option C, with Option B's derived budget as the last guard.** + +The argument that the save ends once draws stop is SPEC 05 GC-13a; the +budget is derived in GC-13. + +## Consequences + +- `freelist.rs`: `FREELIST_SAVE_DRAW_ROUNDS = 8` replaces the constant 64; + the `assert!` uses the budget `3·(P + D + A) + G + 64`, never + below the old limit. +- `alloc.rs`: a third allocation mode, `GcSaveExtend`, in which the save + takes loose pages or extends. +- Tests: `just save-no-draw`, also a CI job, builds with the threshold at 0 + and at 1 (`--cfg zerodb_save_draw_rounds`) and runs the tests of + `zerodb-core` and `zerodb` and 200 crash cycles. The tests check contents + and page accounting (every page reachable or listed free, never both) + through the existing `check`. Two tests are skipped in that mode because + they assert a file size and page numbers that hold only when the save + reuses pool pages: `over_cap_freed_set_spills_whole_to_gc_tree` and + `delete_rebalance_logical_images_match_golden`. At the default threshold + no test reaches the degraded mode. +- SPEC 05: GC-12, GC-13 (the budget) and the new GC-13a. +- No format change. Files written by either version are read by the other. +- The differential oracle is unaffected: page reclamation is not observable + through the heed API. + +## Open questions for human review + +1. **Threshold.** 8 leaves a margin over the 3 observed. The nightly fuzz + and the full crash run were not measured with the counter. +2. **Last guard: panic or error?** It is still a panic. If an error, which + variant (Option D). +3. **Keep a counter?** A permanent maximum-rounds statistic, readable from + `stat`, would show in production whether the degraded mode is ever + reached. diff --git a/justfile b/justfile index 7a27394f..37ea210a 100644 --- a/justfile +++ b/justfile @@ -19,6 +19,22 @@ miri: loom: RUSTFLAGS="--cfg loom" CARGO_PROFILE_RELEASE_DEBUG_ASSERTIONS=true cargo test -p zerodb-core --lib --release loom_ +# The free-list save with its pool draws cut off from the first round, then +# after one round (ADR-0044, SPEC 05 GC-13a), over the engine's tests and the +# crash smoke. The two skipped tests assert a file size and page numbers that +# hold only when the save reuses pool pages. +save-no-draw: + #!/usr/bin/env bash + set -euo pipefail + export CARGO_TARGET_DIR=target-save-no-draw + for rounds in 0 1; do + export RUSTFLAGS="--cfg zerodb_save_draw_rounds=\"$rounds\"" + cargo test -p zerodb-core -p zerodb -- \ + --skip over_cap_freed_set_spills_whole_to_gc_tree \ + --skip delete_rebalance_logical_images_match_golden + cargo run -p zerodb-oracle --bin crash-harness -- --cycles 200 + done + # Unoptimized on purpose: debug assertions keep the writer-side GC shadow gate # armed. A ~5 s variant runs in the normal `cargo test` suite; this one runs # nightly in CI and before a consumer integration gate (ADR-0006). From b066e15c042558f7f5ae04f6f414991d51341bf1 Mon Sep 17 00:00:00 2001 From: Quentin de Quelen Date: Thu, 12 Nov 2026 20:01:27 +0100 Subject: [PATCH 2/3] tools: drop a redundant #[must_use] that clippy 1.100 rejects `run` returns `ExitCode`, which is itself `#[must_use]`. Clippy in Rust 1.100 reports the attribute as `double_must_use`, and CI runs clippy with `-D warnings` on the current stable toolchain. Co-Authored-By: Claude Fable 5.2 --- crates/zerodb-tools/src/lib.rs | 1 - 1 file changed, 1 deletion(-) diff --git a/crates/zerodb-tools/src/lib.rs b/crates/zerodb-tools/src/lib.rs index 3bfb388b..e4800268 100644 --- a/crates/zerodb-tools/src/lib.rs +++ b/crates/zerodb-tools/src/lib.rs @@ -62,7 +62,6 @@ const DEFAULT_MAP_SIZE: u64 = 256 << 20; /// Parse `argv` (excluding the program name) and run the selected subcommand. /// Returns the process exit code: `0` success, `1` = `check` found violations, /// `2` = usage/other error. -#[must_use] pub fn run(args: &[String]) -> ExitCode { match dispatch(args) { Ok(code) => code, From b1f76f61d5b4c60facc788b9ac4f04708b79d44b Mon Sep 17 00:00:00 2001 From: Quentin de Quelen Date: Thu, 12 Nov 2026 20:45:51 +0100 Subject: [PATCH 3/3] Free-list save: draw from the pools when the map is full Once the save has stopped drawing (GC-13a), an allocation the pools could serve extended the file instead. With the file at the map's end that returned `MapFull` and failed a commit that succeeded before. In that case the save now draws from the pools as it does in its first rounds; it then ends by GC-13 and the round budget. `save_at_map_limit.rs` covers it: under `just save-no-draw` it fails without the fallback. Also corrects comments that still described the save's allocation mode as fixed for the whole procedure. Co-Authored-By: Claude Fable 5.2 --- crates/zerodb-core/src/rwtxn/alloc.rs | 45 ++++++----- crates/zerodb-core/src/rwtxn/freelist.rs | 8 +- crates/zerodb-core/src/rwtxn/mod.rs | 11 +-- crates/zerodb/tests/save_at_map_limit.rs | 87 ++++++++++++++++++++++ docs/SPEC/05-gc.md | 11 ++- docs/adr/0044-freelist-save-round-bound.md | 7 +- 6 files changed, 139 insertions(+), 30 deletions(-) create mode 100644 crates/zerodb/tests/save_at_map_limit.rs diff --git a/crates/zerodb-core/src/rwtxn/alloc.rs b/crates/zerodb-core/src/rwtxn/alloc.rs index 62dc197e..422bdf90 100644 --- a/crates/zerodb-core/src/rwtxn/alloc.rs +++ b/crates/zerodb-core/src/rwtxn/alloc.rs @@ -226,8 +226,11 @@ pub(super) enum AllocMode { /// Set at `freelist_save` entry; the commit consumes the txn. GcSave, /// Inside `freelist_save` once its drawing rounds are spent (GC-13a): - /// loose pages or extend only. Neither pool is drawn from, so no record - /// already written can come to list a handed-out page. + /// loose pages or extend. Neither pool is drawn from, so no record + /// already written can come to list a handed-out page. The one + /// exception is a full map: a save the pools can serve must not fail + /// with `MapFull`, so the pools are drawn from when the file cannot be + /// extended. GcSaveExtend, } @@ -315,25 +318,18 @@ impl<'env> RwTxn<'env> { } } AllocMode::GcSave => { - // GC-12 (ADR-0005): never *read* the GC tree in-save, but - // draw from the already-loaded drain pool, which keeps file - // growth bounded (`freelist_save` gives the argument). Each - // entry touched here is rewritten again before the fixed - // point (`save_touched`), so no rewritten entry lists a - // handed-out page. - if let Some(start) = self.save_pool_draw(n) { + if let Some(start) = self.save_draw(n) { return Ok(start); } - // The carried annex pool (SPEC 05 §2a GC-30): gate-checked - // like every annex draw; the step-(c) placement re-reads - // `annex.live()` after every put, so a draw here can never - // leave a handed-out id in the persisted set — the GC-12 - // pool argument. - if let Some(start) = self.annex_draw(n) { - return Ok(start); + } + AllocMode::GcSaveExtend => { + let mp = map_pages(self.env.inner().map_size(), self.psize); + if is_map_full(self.next_pgno, n, mp) { + if let Some(start) = self.save_draw(n) { + return Ok(start); + } } } - AllocMode::GcSaveExtend => {} } let mp = map_pages(self.env.inner().map_size(), self.psize); if is_map_full(self.next_pgno, n, mp) { @@ -355,6 +351,19 @@ impl<'env> RwTxn<'env> { } } + /// The in-save draws (GC-12, ADR-0005): the GC tree is never *read* + /// in-save, but the already-loaded drain pool is drawn from, which keeps + /// file growth bounded (`freelist_save` gives the argument). Each entry + /// touched is rewritten again before the fixed point (`save_touched`), + /// so no rewritten entry lists a handed-out page. Then the carried annex + /// pool (SPEC 05 §2a GC-30), gate-checked like every annex draw: the + /// step-(c) placement re-reads `annex.live()` after every put, so a draw + /// can never leave a handed-out id in the persisted set. + #[inline(never)] + fn save_draw(&mut self, n: u64) -> Option { + self.save_pool_draw(n).or_else(|| self.annex_draw(n)) + } + /// In-save draw from the **already-loaded drain pool** (GC-12; /// see [`RwTxn::allocate`]). Deterministic like GC-19: smallest /// reclaimable record first (`drains` is a `BTreeMap` in key order), @@ -365,7 +374,7 @@ impl<'env> RwTxn<'env> { /// pages are absent. #[inline(never)] pub(super) fn save_pool_draw(&mut self, n: u64) -> Option { - debug_assert!(self.alloc_mode == AllocMode::GcSave); + debug_assert!(self.alloc_mode != AllocMode::Normal); let mut hit: Option<(GcKey, u64)> = None; for (key, remaining) in &work!(self).drains { if let Some(start) = find_run(remaining.live(), n) { diff --git a/crates/zerodb-core/src/rwtxn/freelist.rs b/crates/zerodb-core/src/rwtxn/freelist.rs index 6ae9cc77..93ca879a 100644 --- a/crates/zerodb-core/src/rwtxn/freelist.rs +++ b/crates/zerodb-core/src/rwtxn/freelist.rs @@ -135,8 +135,9 @@ impl<'env> RwTxn<'env> { /// the old `free_db` root whose entry `F` lists that page as free and /// whose trees do not reference it (`F ≤ oldest_reader() ≤ N−1`, TXN-62). /// - /// **Allocation restriction (GC-12):** `alloc_mode = GcSave` for the - /// whole procedure — `allocate` never **reads** the GC tree; it draws + /// **Allocation restriction (GC-12):** `alloc_mode` is `GcSave`, then + /// `GcSaveExtend`, never `Normal` — `allocate` never **reads** the GC + /// tree; it draws /// loose pages, pages from the **already-loaded drain pool** /// ([`RwTxn::save_pool_draw`]), the carried annex pool, or extends. Every /// request is for one page: GC records are always inline (GC-5). @@ -147,7 +148,8 @@ impl<'env> RwTxn<'env> { /// rewrite set, and the loop only exits when no rewrite is pending, so /// the final tree state never lists a handed-out page. After /// [`FREELIST_SAVE_DRAW_ROUNDS`] rounds `alloc_mode` becomes - /// `GcSaveExtend` and the pools are no longer drawn from (GC-13a). + /// `GcSaveExtend` and the pools are no longer drawn from, unless the + /// map is full (GC-13a). /// /// **Termination (GC-13):** each iteration that does not exit strictly /// shrinks a finite pool (a drain-pool or annex-pool draw), grows `freed` diff --git a/crates/zerodb-core/src/rwtxn/mod.rs b/crates/zerodb-core/src/rwtxn/mod.rs index 9256cbc9..c404ac6d 100644 --- a/crates/zerodb-core/src/rwtxn/mod.rs +++ b/crates/zerodb-core/src/rwtxn/mod.rs @@ -25,8 +25,9 @@ //! C1, `RwTxn::freelist_save` rewrites drained entries, releases trailing //! loose pages (GC-10), and writes this txn's freed set under //! `BE(writer_txnid)` in a fixed-point loop (GC-11..13) during which -//! `allocate` is restricted to loose pages / extend only (GC-12, -//! `AllocMode::GcSave`). The GC tree is mutated by the **same** split/COW +//! `allocate` never reads the GC tree: it takes loose pages, draws from the +//! pools already loaded, or extends (GC-12, GC-13a; `AllocMode::GcSave`, +//! then `AllocMode::GcSaveExtend`). The GC tree is mutated by the **same** split/COW //! code as the main tree, selected by `TreeId`. //! //! ## Commit (SPEC 04 §9, SPEC 06 §2) @@ -221,9 +222,9 @@ pub(crate) struct WriterCarry { /// The base meta's free-list annex pool (SPEC 05 §2a, ADR-0022): the /// pages freed by txn `base.txnid`, parsed (and `validate_pil_ids`- /// checked, GC-33) at begin when `base.free_annex > 0`. Drawn by - /// [`RwTxn::annex_draw`] (allocate step 2b, gated per GC-31); whatever - /// remains is folded into `freed` at the top of `freelist_save` (the - /// GC-30 carry) — by then the pool is empty, so GcSave never sees it. + /// [`RwTxn::annex_draw`] (allocate step 2b, gated per GC-31), during + /// ops and inside `freelist_save`; whatever remains at the save's fixed + /// point is carried into this txn's own entry (GC-30). annex: Drain, /// Entries of `drains` whose `remaining` shrank via an **in-save pool /// draw** (GC-12 — see `freelist_save`): each must be diff --git a/crates/zerodb/tests/save_at_map_limit.rs b/crates/zerodb/tests/save_at_map_limit.rs new file mode 100644 index 00000000..5dc2a4f6 --- /dev/null +++ b/crates/zerodb/tests/save_at_map_limit.rs @@ -0,0 +1,87 @@ +//! Commits when the file is within a few pages of the map size and holds +//! more free pages than the meta annex lists (SPEC 05 GC-13a, GC-17): once +//! the free-list save can no longer extend the file, every page it needs +//! must come from the free pages it already holds, and the commit must +//! still succeed. The case matters when the save has stopped drawing from +//! its pools, which `just save-no-draw` forces on every commit. + +mod common; +use common::TempDir; + +use zerodb::{check, EnvOpenOptions}; + +const PS: u32 = 4096; +const MAP_PAGES: u64 = 1024; +/// Pages left between the file's end and the map's end. +const HEADROOM: u64 = 6; + +fn key(i: u32) -> Vec { + format!("key{i:05}").into_bytes() +} + +#[test] +fn commits_succeed_when_the_file_cannot_grow() { + let dir = TempDir::new(); + let mut opts = EnvOpenOptions::new(); + opts.map_size((MAP_PAGES * u64::from(PS)) as usize); + opts.page_size(PS); + // SAFETY: only ZeroDB writes this environment's files. + let env = unsafe { opts.open(dir.path()) }.expect("open env"); + let db = { + let mut w = env.write_txn().unwrap(); + let db = env.create_database(&mut w, None).unwrap(); + for i in 0..60 { + db.put(&mut w, &key(i), &[0xA5u8; 500]).unwrap(); + } + w.commit().unwrap(); + db + }; + + // One value on an overflow run takes the file to `HEADROOM` pages from + // the map's end; deleting it leaves the file that long and the run free. + // The run is longer than the meta annex, so it is listed in free-page + // records, which later commits drain and rewrite. + let used = env.info().last_pgno + 1; + let run = MAP_PAGES - used - HEADROOM - 4; + { + let mut w = env.write_txn().unwrap(); + db.put( + &mut w, + b"big", + &vec![0x5Au8; (run * u64::from(PS)) as usize - 64], + ) + .unwrap(); + w.commit().unwrap(); + } + { + let mut w = env.write_txn().unwrap(); + assert!(db.delete(&mut w, b"big").unwrap()); + w.commit().unwrap(); + } + assert!( + env.info().last_pgno + 1 + HEADROOM + 4 >= MAP_PAGES, + "the file is not near the map's end: last page {}", + env.info().last_pgno + ); + + // Each commit copies tree pages and rewrites free-page records. A save + // that extends the file uses up the headroom within a few commits. + for round in 0u8..80 { + let mut w = env.write_txn().unwrap(); + for i in (0..60).filter(|i| i % 4 == u32::from(round) % 4) { + db.put(&mut w, &key(i), &[round; 500]).unwrap(); + } + w.commit().unwrap_or_else(|e| panic!("round {round}: {e}")); + } + + let rtxn = env.read_txn().unwrap(); + assert_eq!(db.len(&rtxn).unwrap(), 60); + drop(rtxn); + drop(env); + let bytes = std::fs::read(dir.path().join(zerodb::DATA_FILE_NAME)).unwrap(); + let violations = check::check_image(&bytes, PS); + assert!( + violations.is_empty(), + "invariant violations: {violations:#?}" + ); +} diff --git a/docs/SPEC/05-gc.md b/docs/SPEC/05-gc.md index 6eeaacad..6252429f 100644 --- a/docs/SPEC/05-gc.md +++ b/docs/SPEC/05-gc.md @@ -258,7 +258,8 @@ in-loop allocator so the loop cannot leak GC entries. ``` freelist_save(txn): - alloc_mode = GC_SAVE # GC-12 scope: whole procedure + alloc_mode = GC_SAVE # GC-12; GC_SAVE_EXTEND after + # DRAW_ROUNDS rounds (GC-13a) # base_annex.remainder stays drawable (GC-30 pool); merged in step (c) pending = keys(txn.drains) # every ops-drained entry needs # its GC-20 rewrite at least once @@ -365,7 +366,8 @@ in-loop allocator so the loop cannot leak GC entries. drained during §6 reclamation — provided every entry so touched is **re-rewritten** (GC-20) before the loop's fixed point is declared, or **(3)** file extend (`next_pgno`, GC-15). After `DRAW_ROUNDS` rounds source - (2) is closed (GC-13a). The anti-leak property is carried + (2) and the annex pool (GC-30) are closed, unless the map is full + (GC-13a). The anti-leak property is carried by the fixed point, not by a draw ban: the loop may not exit while any touched entry awaits its rewrite, so the final tree state never lists a handed-out page (INV-22) and never drops an undrained one. Pool draws @@ -455,6 +457,11 @@ in-loop allocator so the loop cannot leak GC entries. own entry (GC-30). They are reusable by later transactions; the cost is a file a few pages larger than it could have been. + *Full map.* When the file cannot be extended (GC-17), the pools are drawn + from as before round `DRAW_ROUNDS + 1`: a save that the pools can serve + must not fail with `MapFull`. The argument below then does not apply, and + the save ends by GC-13 alone, within the round budget. + *Why the save then ends.* Without draws no record is re-marked pending, and the annex pool no longer shrinks, so quantities (i) and (ii) are out of the loop. Records are deleted only in the first round after the switch: diff --git a/docs/adr/0044-freelist-save-round-bound.md b/docs/adr/0044-freelist-save-round-bound.md index 1b3a33d6..46d9ab09 100644 --- a/docs/adr/0044-freelist-save-round-bound.md +++ b/docs/adr/0044-freelist-save-round-bound.md @@ -133,7 +133,9 @@ budget is derived in GC-13. the `assert!` uses the budget `3·(P + D + A) + G + 64`, never below the old limit. - `alloc.rs`: a third allocation mode, `GcSaveExtend`, in which the save - takes loose pages or extends. + takes loose pages or extends. When the map is full it draws from the + pools as before, so that a save the pools can serve does not fail with + `MapFull`; that case ends by GC-13 and the budget alone. - Tests: `just save-no-draw`, also a CI job, builds with the threshold at 0 and at 1 (`--cfg zerodb_save_draw_rounds`) and runs the tests of `zerodb-core` and `zerodb` and 200 crash cycles. The tests check contents @@ -142,7 +144,8 @@ budget is derived in GC-13. they assert a file size and page numbers that hold only when the save reuses pool pages: `over_cap_freed_set_spills_whole_to_gc_tree` and `delete_rebalance_logical_images_match_golden`. At the default threshold - no test reaches the degraded mode. + no test reaches the degraded mode. `save_at_map_limit.rs` covers the full + map: it fails in that run if the save does not fall back to the pools. - SPEC 05: GC-12, GC-13 (the budget) and the new GC-13a. - No format change. Files written by either version are read by the other. - The differential oracle is unaffected: page reclamation is not observable