Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,9 @@ Ratios are ZeroDB time ÷ LMDB time on Graviton4 unless stated; details in
- Full scans read entries from the leaf in hand, and trusted mode no longer
records leaves: `scan/full/fwd` 2.25× → 1.45×, and about 1.0× in trusted
mode.
- `first` and `last` descend without building a cursor, as point gets do
(#147): 61.9 → 41.7 ns per call against LMDB's 54.5 (1.14× → 0.77×,
laptop).
- `skip` crosses whole leaves without decoding the entries it passes.
- Short read transactions reuse their memory on the same thread (#25).
- The meta checksum uses the ARMv8 `crc32c` instructions on aarch64
Expand Down
1 change: 1 addition & 0 deletions benches/results/perf-ledger.jsonl
Original file line number Diff line number Diff line change
Expand Up @@ -111,3 +111,4 @@
{"date": "2026-11-02T07:33:25+00:00", "rung": "^(put/gc/ovf|del/bulk)", "hypothesis": "Owed re-measure for #141 (write cursor keeps its path across splits): #141 saw put/gc/ovf_across +4.2% and del/bulk/all +3.1% and was never re-run after its last fix. Before = aa6cbe3 (the commit #141 was built on), after = main at 3c4f37f.", "outcome": "kept", "commit": "987ce3e", "verdict": "invalid", "base": "aa6cbe351f86a46befe79b82bbc3a0b70eeeb628", "host": "Darwin arm64", "rounds": 3, "rungs": [{"rung": "del/bulk/all", "state": "unreliable", "ratio_vs_lmdb_before": 0.9936896285905952, "ratio_vs_lmdb_after": 0.9259079292614257, "speedup": 0.931787856712052}, {"rung": "del/bulk/half", "state": "improved", "ratio_vs_lmdb_before": 1.2715363571835225, "ratio_vs_lmdb_after": 0.991449157233737, "speedup": 0.7797253705193424}, {"rung": "put/gc/ovf_across", "state": "unreliable", "ratio_vs_lmdb_before": 0.8452338529426783, "ratio_vs_lmdb_after": 1.1733910898597482, "speedup": 1.3882443134223643}, {"rung": "put/gc/ovf_miss", "state": "unreliable", "ratio_vs_lmdb_before": 3.204429671760977, "ratio_vs_lmdb_after": 1.9395986177080942, "speedup": 0.5698584235538426}], "notes": "macOS laptop (Apple Silicon, 16 KiB pages), 3 interleaved rounds at load average 26 with nine other build sessions running: verdict INVALID, indicative only. LMDB drifted 20-56% between halves on del/bulk/all, put/gc/ovf_across and put/gc/ovf_miss, so those rows say nothing. The one reliable row: del/bulk/half 1.27x -> 0.99x LMDB (0.78 of before, LMDB drift 0.997). No regression is visible, but the +4.2% / +3.1% question is not settled: the referee is still a Graviton run of the same filter on a quiet host. The current harness needs two shims to build against aa6cbe3 (the heed-zerodb package rename and the oracle os_page_size helper), so a re-run must apply them too."}
{"date": "2026-11-02T08:11:39+00:00", "rung": "put/order/append", "hypothesis": "APPEND zero-fill: skip the fill(0) of a reused spare frame (heap) and of the map page (in-place WRITE_MAP) when insert_tree_frame creates a split half, as LMDB leaves a reused me_dpages page and a WRITEMAP page unzeroed; fresh heap allocations stay zeroed. The insert_tree_frame contract (fresh frames all-zero) forbids it, so it needs an ADR.", "outcome": "blocked-adr", "notes": "macOS laptop, 16 KiB pages, load average ~100, indicative. Scratch driver, 50k APPEND puts of 256-byte values in one txn, loop time only, min of 31 reps, 5 interleaved runs: WRITE_MAP 81.6 -> 73.6 ns/entry (0.90, every run agrees), heap 113.1 -> 109.4 (0.97, noise; only spare frames are affected, and the put/order/append rung opens a fresh env per iteration so it has none). Not kept: measured on a throwaway build only. Safety argument and recommendation: header and cells are fully written by init and the inserts, nothing reads the free gap (ADR-0030), the stale bytes are this env's own; a split MDB_RESERVE placeholder must then be zeroed explicitly. Referee: Graviton put/order/append under ZERODB_BENCH_WRITE_MAP=1."}
{"date": "2026-11-02T21:16:32+00:00", "rung": "^(put/cursor|del/cursor|put/over/same_size)", "hypothesis": "Write cursor and keyed writes, per-entry cost: (1) touch_path returns after one dirty-set test when the path ends on a frame of this txn and nothing is spilled (a page is copied only below a touched parent), instead of one probe per level; (2) delete_apply reads the value shape and removes the cell through one frame access; (3) the cursor in-leaf step and current-key reads check the handle and read the leaf directly, without building a Tree.", "outcome": "kept", "verdict": "improved", "base": "779ce3772a543337d2c5ca5cf9ebd71214a9efb8", "host": "Darwin arm64", "rounds": 3, "rungs": [{"rung": "del/cursor/drain", "state": "improved", "ratio_vs_lmdb_before": 1.4473069653649546, "ratio_vs_lmdb_after": 1.3635500887773762, "speedup": 0.942696812319092}, {"rung": "del/cursor/rev", "state": "improved", "ratio_vs_lmdb_before": 1.5557379392236108, "ratio_vs_lmdb_after": 1.4181563952222385, "speedup": 0.9081392248656213}, {"rung": "put/cursor/rewrite", "state": "improved", "ratio_vs_lmdb_before": 1.4903260942357737, "ratio_vs_lmdb_after": 1.3721839822237063, "speedup": 0.9230954658696557}, {"rung": "put/cursor/rewrite_reserved", "state": "improved", "ratio_vs_lmdb_before": 1.519072261691081, "ratio_vs_lmdb_after": 1.4212508766951422, "speedup": 0.9353051010587302}, {"rung": "put/cursor/rewrite_reserved_ovf", "state": "unreliable", "ratio_vs_lmdb_before": 1.141535860898173, "ratio_vs_lmdb_after": 1.2033279149977745, "speedup": 1.0541306289326582}, {"rung": "put/order/append", "state": "flat", "ratio_vs_lmdb_before": 1.3689426360392456, "ratio_vs_lmdb_after": 1.3620864566063446, "speedup": 0.9932515325105543}, {"rung": "put/order/rand", "state": "flat", "ratio_vs_lmdb_before": 0.9062615045875345, "ratio_vs_lmdb_after": 0.8864357119964356, "speedup": 0.9781235410632141}, {"rung": "put/order/seq", "state": "flat", "ratio_vs_lmdb_before": 0.8388673563926385, "ratio_vs_lmdb_after": 0.8205656357264116, "speedup": 0.9772166920185364}, {"rung": "put/over/grow", "state": "flat", "ratio_vs_lmdb_before": 1.0856392607006042, "ratio_vs_lmdb_after": 1.0614109982943436, "speedup": 0.9758267762159284}, {"rung": "put/over/same_size", "state": "improved", "ratio_vs_lmdb_before": 1.1505396293473196, "ratio_vs_lmdb_after": 1.1076922211165607, "speedup": 0.962614914206971}], "notes": "macOS laptop (Apple Silicon, 16 KiB pages), load 4.4, 3 interleaved rounds, before = origin/main 779ce37: improved 5, regressed 0, flat 7, drift 1 (rewrite_reserved_ovf). put/cursor/rewrite 1.49x -> 1.37x LMDB, del/cursor/drain 1.45x -> 1.36x, del/cursor/rev 1.56x -> 1.42x, put/over/same_size 1.15x -> 1.11x. Profile of the rewrite loop before: the second page lookup per entry (step leaf view + put frame) was ~3% of an entry; a slab-handle dirty index to remove it was not tried (DirtyStore restructure, small gain). Also measured and not kept: threading an owned-path flag from the keyed descent into put_apply/delete_apply (no gain over the one-test touch_path, more signature churn). Graviton referee owed."}
{"date": "2026-11-09T22:41:31+00:00", "rung": "scan/edge/first_last", "hypothesis": "Database::first/last descend without a cursor (Tree::first_entry / last_entry): no 32-frame path stack to initialise, no leaf cache, no cursor state, as the cursor-free point get; LMDB pays mdb_cursor_init plus mdb_page_search.", "outcome": "kept", "verdict": "invalid", "base": "f77726057c2ec513b7c13a04e3654dfd7c56c41e", "host": "Darwin arm64", "rounds": 3, "rungs": [{"rung": "scan/edge/first_last", "state": "unreliable", "ratio_vs_lmdb_before": 1.1296049331348756, "ratio_vs_lmdb_after": 0.8051961349910187, "speedup": 0.7113026946209665}], "notes": "macOS laptop (Apple Silicon, 16 KiB pages), load 4-10 from a sibling session: the bench-ab verdict is INVALID (LMDB drift over 3% on 4 of 5 rungs; first_last read 0.71 of before). Load-independent evidence, read_census at codegen-units=1, thread CPU ns/op, p10 over 200 batches, median of 5 interleaved rounds: first_last LMDB 54.5, before 61.9, after 41.7 (1.14x -> 0.77x LMDB, 0.675 of before; every after round below every before round and every LMDB round on min). Unchanged paths, 7 rounds of 300 batches: get/access/hot 34.5 -> 33.9 (flat), scan/full/fwd 3.94 -> 3.81. A Graviton bench-ab gate is owed."}
61 changes: 61 additions & 0 deletions crates/zerodb-core/src/btree/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -617,6 +617,61 @@ impl<'a> Tree<'a> {
Err(depth_exceeded())
}

/// The minimum entry (`MDB_FIRST` with no cursor kept), or `None` for an
/// empty tree. Same result as [`Cursor::first`] on a fresh cursor.
///
/// # Errors
///
/// A [`PageError`] only if the tree is structurally corrupt.
pub fn first_entry(&self) -> PosResult<'a> {
self.edge_entry(false)
}

/// The maximum entry (`MDB_LAST` with no cursor kept), or `None` for an
/// empty tree. Same result as [`Cursor::last`] on a fresh cursor.
///
/// # Errors
///
/// A [`PageError`] only if the tree is structurally corrupt.
pub fn last_entry(&self) -> PosResult<'a> {
self.edge_entry(true)
}

/// Root-to-leaf descent along the leftmost (`max == false`) or rightmost
/// edge, returning that leaf's first or last entry. No cursor: the caller
/// keeps no position, so there is no path to record, as in
/// [`Tree::find_exact`], whose level bound and error this shares.
#[inline(always)]
fn edge_entry(&self, max: bool) -> PosResult<'a> {
if self.root == PGNO_INVALID {
return Ok(None);
}
let mut pgno = self.root;
for _ in 0..(self.depth as usize + 2).min(CURSOR_STACK) {
match node_view(self.src, self.psize, pgno, self.valid)? {
NodeView::Leaf(leaf) => {
let n = leaf.num_keys();
if n == 0 {
return Ok(None);
}
let ki = if max { n - 1 } else { 0 };
let k = leaf.key(ki);
let v = resolve_value(self.src, self.psize, &leaf, ki, self.valid)?;
return Ok(Some((k, v)));
}
NodeView::Branch(br) => {
let ci = if max {
br.num_keys().saturating_sub(1)
} else {
0
};
pgno = br.child_pgno(ci);
}
}
}
Err(depth_exceeded())
}

/// Catalog lookup for the named-DB resolver (SPEC 02 §6): like [`Tree::get`]
/// but also returns the leaf **node flags** of the matched entry, so the
/// caller can tell an `F_SUBDATA` sub-DB record from a plain user key
Expand Down Expand Up @@ -1644,6 +1699,9 @@ mod tests {
let mut c = tree.cursor();
let mut out = Vec::new();
let mut e = c.first().unwrap();
// The cursor-free edge read agrees with the cursor on every tree
// these tests walk.
assert_eq!(tree.first_entry().unwrap(), e);
while let Some((k, v)) = e {
out.push((k.to_vec(), v.to_vec()));
e = c.next().unwrap();
Expand All @@ -1655,6 +1713,7 @@ mod tests {
let mut c = tree.cursor();
let mut out = Vec::new();
let mut e = c.last().unwrap();
assert_eq!(tree.last_entry().unwrap(), e);
while let Some((k, v)) = e {
out.push((k.to_vec(), v.to_vec()));
e = c.prev().unwrap();
Expand All @@ -1669,6 +1728,8 @@ mod tests {
assert!(t.is_empty());
assert_eq!(t.get(b"x").unwrap(), None);
let mut c = t.cursor();
assert_eq!(t.first_entry().unwrap(), None);
assert_eq!(t.last_entry().unwrap(), None);
assert_eq!(c.first().unwrap(), None);
assert_eq!(c.last().unwrap(), None);
assert_eq!(c.next().unwrap(), None);
Expand Down
10 changes: 2 additions & 8 deletions crates/zerodb-core/src/rotxn.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1009,10 +1009,7 @@ impl Database {
&self,
txn: &'txn T,
) -> Result<Option<(&'txn [u8], &'txn [u8])>> {
self.checked_tree(txn)?
.cursor()
.first()
.map_err(map_page_err)
self.checked_tree(txn)?.first_entry().map_err(map_page_err)
}

/// `last(txn)` — the maximum entry (SPEC 00 row 42).
Expand All @@ -1021,10 +1018,7 @@ impl Database {
///
/// [`MdbError::Invalid`] on a corrupt tree.
pub fn last<'txn, T: TxnRead>(&self, txn: &'txn T) -> Result<Option<(&'txn [u8], &'txn [u8])>> {
self.checked_tree(txn)?
.cursor()
.last()
.map_err(map_page_err)
self.checked_tree(txn)?.last_entry().map_err(map_page_err)
}

/// `get_greater_than_or_equal_to(txn, key)` — first entry `≥ key`
Expand Down