Skip to content

Review fixes: page-view bounds, leaner core, honest docs, Graviton4 + gp3 benchmark - #171

Merged
qdequele merged 2 commits into
mainfrom
review-fixes
Nov 12, 2026
Merged

qdequele merged 2 commits into
mainfrom
review-fixes

Conversation

@qdequele

Copy link
Copy Markdown
Owner

Summary

Outcome of a full review of the repository (code, tests, docs, benchmarks), plus a new benchmark run on the target hardware. Two commits: docs and benchmark report, then code.

Fixes

  • LeafRef::new and BranchRef::new accepted a buffer shorter than the page size and then read with get_unchecked: an out-of-bounds read from safe code when called directly (the engine always passes whole pages). They now return PageError::BufferTooSmall. LeafMut::lookup had a similar hole and now bounds-checks its reads.
  • zerodb-tools check read the whole data file into memory in its raw fallback. It now walks a read-only map. One behavior change: a meta naming a map size above the supported maximum is reported as an INV-1 violation and the trees are not walked.
  • pwritev is retried on EINTR.
  • remove_span leaves the page unmodified when it returns an error.
  • zerodb-tools rejects unexpected extra arguments.
  • check gains check_image_with_key_order / check_snapshot_with_key_order, so a caller holding a custom comparator can have key order checked under it. The default entry points and the CLI are unchanged.

Leaner code

  • The batch tree builder is compiled for tests only (it stays as the independent reference for the streaming builder).
  • dirty and EnvInner are crate-private; StampKind, a forwarding cursor method and one force_sync layer are removed.
  • ComparatorError, LockOpenError and DumpError derive thiserror::Error, with tests pinning every message.
  • Comments in the four engine crates no longer carry history, measurements, issue numbers or internal codes. Each cleanup was checked to be comment-only by comparing the files with comments stripped. Several false doc statements were corrected.

Docs

  • README and USING state that the docs describe main, which is ahead of 0.2.0, and list the differences.
  • docs/ARCHITECTURE.md: a ZeroDB and LMDB comparison table.
  • docs/PROVENANCE.md (new): how the project was built, what is verified and what is not.
  • docs/DECISIONS.md grouped by status; reserved numbers 0036 and 0040 explained.
  • ADR-0043 (Proposed): bitflags and tempfile.
  • "Lock-free readers" reworded: readers never wait on a write transaction; beginning one takes a short mutex.

Benchmark

Current main on AWS c8g.xlarge (Graviton4) with an EBS gp3 volume, default and validating: benches/results/2026-11-12-ladder-meilisearch-graviton-gp3.md. ZeroDB time ÷ LMDB time, default options:

Area Ratio
Meilisearch v1.55.0, movies: indexing, search 0.98×, 1.00×
Durable commits 0.98–1.00×
Point reads 0.62–0.95×
Full scans 1.01–1.03×
Per-key deletes 1.27–1.28×
Empty write transaction 3.49×
Create an environment 27.8×

The bench harness gains ZERODB_BENCH_VALIDATE=1.

Checks

  • macOS aarch64: cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings, cargo test --workspace, cargo doc --workspace --no-deps (no warnings).
  • Linux aarch64 (Graviton4): fmt, clippy, cargo test --workspace, just crash-test-quick equivalents (200 cycles, 50 multi-process cycles).
  • Not run locally on the final branch: miri, the fuzz gate, loom, stress, the consumer suites. CI covers miri, fuzz, loom and the crash smoke.

For the maintainer to decide

  • Read the provenance wording in README.md and docs/PROVENANCE.md before publishing.
  • The comparator-aware check functions are new public API; CONTRIBUTING rule 5 asks for an ADR.
  • Not changed, because they need an ADR: the release-mode assert! on the free-list save round count in commit (the 64-round bound is not proven), the validation caches, WRITE_MAP, and replacing the /proc/self/mountinfo parsing with fstatfs.

🤖 Generated with Claude Code

Quentin de Quelen and others added 2 commits November 12, 2026 10:59
…raviton4 + gp3 benchmark run

- README and USING say the docs describe main, ahead of 0.2.0, and list the
  differences.
- Performance figures come from a new run on Graviton4 with EBS gp3 (current
  main, default and validating); the report is in benches/results.
- ARCHITECTURE gains a ZeroDB and LMDB comparison table.
- PROVENANCE.md states how the project was built and what is and is not
  verified.
- DECISIONS.md is grouped by status and explains the reserved numbers.
- ADR-0043 (proposed): bitflags and tempfile.
- Reader wording: readers never wait on a write transaction; beginning one
  takes a short mutex.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Fixes:
- LeafRef::new and BranchRef::new reject a buffer shorter than the page
  size; LeafMut::lookup bounds-checks its reads.
- zerodb-tools check walks a read-only map in its raw fallback and no longer
  reads the whole file.
- pwritev is retried on EINTR.
- remove_span leaves the page unmodified on error.
- zerodb-tools rejects extra positional arguments.
- check gains entry points that take each database's key order.

Leaner:
- The batch tree builder is compiled for tests only.
- dirty and EnvInner are crate-private; StampKind and a few forwards are gone.
- ComparatorError, LockOpenError and DumpError derive thiserror::Error.

Comments in the four engine crates no longer carry history, measurements,
issue numbers or internal codes. The bench harness gains
ZERODB_BENCH_VALIDATE.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 108 files, which is 8 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7df70865-1887-4bd5-8a7c-51115e9a39d1
Commits

Reviewing files that changed from the base of the PR and between d70dbe9 and 8932752.

Files ignored due to path filters: 1

  • Cargo.lock is excluded by !**/*.lock
Files selected for processing: 108

  • CHANGELOG.md
  • CONTRIBUTING.md
  • README.md
  • benches/results/2026-11-12-ladder-meilisearch-graviton-gp3.md
  • benches/results/README.md
  • crates/heed-zerodb/Cargo.toml
  • crates/heed-zerodb/src/database.rs
  • crates/heed-zerodb/src/env.rs
  • crates/heed-zerodb/src/error.rs
  • crates/heed-zerodb/src/flags.rs
  • crates/heed-zerodb/src/iterator.rs
  • crates/heed-zerodb/src/txn.rs
  • crates/heed-zerodb/src/unique.rs
  • crates/zerodb-core/Cargo.toml
  • crates/zerodb-core/src/btree/memo.rs
  • crates/zerodb-core/src/btree/mod.rs
  • crates/zerodb-core/src/builder.rs
  • crates/zerodb-core/src/builder/batch.rs
  • crates/zerodb-core/src/check.rs
  • crates/zerodb-core/src/cmp.rs
  • crates/zerodb-core/src/dirty.rs
  • crates/zerodb-core/src/env/backing.rs
  • crates/zerodb-core/src/env/info.rs
  • crates/zerodb-core/src/env/mod.rs
  • crates/zerodb-core/src/env/open.rs
  • crates/zerodb-core/src/env/shared_env_tests.rs
  • crates/zerodb-core/src/env/writer_lock.rs
  • crates/zerodb-core/src/error.rs
  • crates/zerodb-core/src/handles.rs
  • crates/zerodb-core/src/lib.rs
  • crates/zerodb-core/src/nested.rs
  • crates/zerodb-core/src/page/crc32c.rs
  • crates/zerodb-core/src/page/header.rs
  • crates/zerodb-core/src/page/meta.rs
  • crates/zerodb-core/src/page/mod.rs
  • crates/zerodb-core/src/page/raw.rs
  • crates/zerodb-core/src/page/tree.rs
  • crates/zerodb-core/src/readers.rs
  • crates/zerodb-core/src/rotxn.rs
  • crates/zerodb-core/src/rwtxn/alloc.rs
  • crates/zerodb-core/src/rwtxn/api.rs
  • crates/zerodb-core/src/rwtxn/catalog.rs
  • crates/zerodb-core/src/rwtxn/commit.rs
  • crates/zerodb-core/src/rwtxn/cursor.rs
  • crates/zerodb-core/src/rwtxn/delete.rs
  • crates/zerodb-core/src/rwtxn/finger.rs
  • crates/zerodb-core/src/rwtxn/freelist.rs
  • crates/zerodb-core/src/rwtxn/insert.rs
  • crates/zerodb-core/src/rwtxn/mod.rs
  • crates/zerodb-core/src/rwtxn/spill.rs
  • crates/zerodb-core/src/rwtxn/tests.rs
  • crates/zerodb-core/src/stamps.rs
  • crates/zerodb-core/src/sync.rs
  • crates/zerodb-core/tests/view_bounds.rs
  • crates/zerodb-io/Cargo.toml
  • crates/zerodb-io/src/fault.rs
  • crates/zerodb-io/src/file.rs
  • crates/zerodb-io/src/lib.rs
  • crates/zerodb-io/src/lockfile.rs
  • crates/zerodb-io/src/mmap.rs
  • crates/zerodb-io/src/testmap.rs
  • crates/zerodb-oracle/Cargo.toml
  • crates/zerodb-oracle/benches/engine_comparison/backend.rs
  • crates/zerodb-oracle/benches/engine_comparison/suites/del.rs
  • crates/zerodb-oracle/benches/engine_comparison/suites/put.rs
  • crates/zerodb-oracle/examples/free_run_churn.rs
  • crates/zerodb-oracle/examples/freelist_big_record.rs
  • crates/zerodb-oracle/examples/writemap_bulk_census.rs
  • crates/zerodb-oracle/tests/differential/gc_churn_parity.rs
  • crates/zerodb-oracle/tests/lmdb_observations/lmdb_dbi_handle_observation.rs
  • crates/zerodb-oracle/tests/lmdb_observations/lmdb_free_run_observation.rs
  • crates/zerodb-tools/Cargo.toml
  • crates/zerodb-tools/src/commands.rs
  • crates/zerodb-tools/src/common.rs
  • crates/zerodb-tools/src/dump_format.rs
  • crates/zerodb-tools/src/lib.rs
  • crates/zerodb-tools/src/naming.rs
  • crates/zerodb-tools/tests/check_raw_file.rs
  • crates/zerodb/Cargo.toml
  • crates/zerodb/src/copy.rs
  • crates/zerodb/src/lib.rs
  • crates/zerodb/tests/check_key_order.rs
  • crates/zerodb/tests/read_api.rs
  • docs/ARCHITECTURE.md
  • docs/DECISIONS.md
  • docs/LMDB-COMPATIBILITY.md
  • docs/PERFORMANCE.md
  • docs/PROVENANCE.md
  • docs/README.md
  • docs/SPEC/03-btree.md
  • docs/USING.md
  • docs/adr/0002-on-disk-format.md
  • docs/adr/0006-reader-table.md
  • docs/adr/0007-nested-read-txns.md
  • docs/adr/0008-crash-harness.md
  • docs/adr/0009-copy-and-tools.md
  • docs/adr/0011-dupsort.md
  • docs/adr/0012-prefetch-access-hints.md
  • docs/adr/0014-trusted-file-mode.md
  • docs/adr/0021-writemap-in-place.md
  • docs/adr/0023-multi-process.md
  • docs/adr/0025-nested-write-txns.md
  • docs/adr/0028-delete-rebalance-node-move.md
  • docs/adr/0030-cow-copies-used-bytes.md
  • docs/adr/0031-hardware-crc32c.md
  • docs/adr/0032-raw-copy-block-clone.md
  • docs/adr/0033-kv-compare-bench.md
  • docs/adr/0043-bitflags-and-tempfile.md

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@qdequele
qdequele merged commit 9115969 into main Nov 12, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant