Release BIMRI v5.1.1: audit checkpoint, exact-current fast path, drift receipts - #6
Merged
Merged
Conversation
The checkpoint is a cache, never an authority record. Divergence from it is a cache miss that forces the full semantic audit: a pass rebaselines and preserves an append-only drift receipt under .bimri/audit-drift/ (bounded, deduplicated, surfaced by doctor); a failure refuses into the existing damaged-authority recovery lane. No automatic path records a block state. audit-blocked.json survives only as the owner-approved quarantine repair baseline, with working exits through restore and doctor. Retained from the performance work: the warm-read gate (head hash, grammar, pointers, residency, checkpoint seal), atomic checkpoint transitions, the fail-closed resolution retry lane, and every measured number. Unknown files in witnessed roots are never deleted or blocked on. Also fixes defects the block latch had masked: the warm fast path skipped the legacy-marker guards and the migration receipt; interrupted close and touch replays could never finalize (skip_validation_id now honored); restore's strict comparison now scopes to the whole owner-repair session; a corrupt transition marker no longer wedges doctor; read-only doctor is byte-exact and treats a loadable in-flight marker as a warning. Suite: 236 tests, 0 failures, 11 skipped. Warm recall 250-290 ms CLI end-to-end and start 360 ms on a copy of the live store; cold first audit 34.5 s once. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…racts Verified and fixed from the independent red team of 993eb07: - An obstruction on the audit-transition marker path that cannot be preserved and retired is a hard error on every surface; doctor never reports health past it and the checkpoint stays untouched. - The archive crash-window exemption is a proof, not a month allowlist: a changed month is transition-explained only when its bytes reduce to the prior witnessed content plus appended rows stamped by the operation's own scope. Foreign rows are preserved and receipted. - Drift receipts are sealed and self-contained: a complete per-path delta with prior and current hashes (capped with explicit truncation counts), monotonic sequence numbers, delta-based deduplication, bounded rolling retention, and bounded corrupt-marker blobs. - Read-only doctor, slow reads and warm exact reads enforce the same legacy-lineage refusals as writable load. - MIGRATION states the real one-way contract (decided proposals keep their v5.1.1 receipts as immutable authority; rollback only from the pre-update backup), contract-tested against the real v5.1.0 engine. - The protocol normative drift clause now states the owner-ruled receipts contract (ruled 2026-08-27), and the write-path numbers are published as an owner-accepted deviation: propose 1.31 s, sync 1.26 s, authority close 1.24 s on the live-size store. - Reproducible benchmark with an engine-driven growth generator checked in at tests/benchmark_v511.py. Suite: 241 tests, 0 failures, 11 skipped on this exact tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ondition From the second independent red team of bd94f5b, all six findings fixed: - The sealed drift receipt is durably recorded before the replacement checkpoint may publish, in every lane: audit publication, authority-write fallback, retry window, and interrupted-transition recovery. A crash between receipt and publication leaves the prior baseline plus the receipt; the retry deduplicates and completes. - A receipt that cannot be recorded refuses instead of passing: doctor fails with the reason, writes raise into recovery, and the prior checkpoint remains the baseline. - Every transition-recovery receipt carries its per-path delta computed against the marker's prior manifest; the lifecycle state/log receipt states explicitly that no inventory delta applies. - Receipts are validated before being trusted (schema, filename/sequence binding, content digest, recomputed seal, delta shape); doctor reports damaged evidence and never prints a forged reason. - The completeness contract is honest and kept: 2,000 entries per section with the remainder counted, and a truncated receipt retains its referenced complete prior manifest generation while the receipt lives. - Corrupt-marker blobs cited by retained receipts (structured preserved_evidence field) outlive the unreferenced-blob bound, so no retained receipt cites pruned bytes. Also: MIGRATION requires the complete pre-update folder snapshot and declares the first v5.1.1 proposal an intentional one-way boundary; the benchmark takes --gate metric=max_ms thresholds; protocol, README and CHANGELOG state the bounded receipt contract in the words the code enforces. Suite green on this exact tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Closes the five reproduced receipt-contract defects with one principle applied at every boundary rather than five patches: - The writer re-reads and fully validates the receipt it wrote (seal, filename binding, attachments, post-prune existence) before reporting success, and deduplication only ever reuses a receipt that validates. An invalid pre-crash receipt is never reused for publication. - Oversized deltas and preserved corrupt-marker bytes are hash-and-size-pinned attachments, validated with their receipt and retained while it is retained: truncation no longer loses the exact boundary, and no retained receipt can cite pruned or missing bytes without being reported as damaged evidence. - A checkpoint whose referenced manifest evidence is unavailable is damaged evidence: it refuses rebaselining with the repair path named instead of adopting new bytes with a null delta. - The receipt sequence grammar is unbounded and sorted numerically; the millionth receipt neither collides nor prunes as oldest, and the post-write validation would refuse rather than lose it. - The benchmark carries committed canonical ceilings (--gate default; non-finite values breach, never bypass), CI runs it, and the workflow triggers on agent/** pushes so the four-leg matrix runs before a PR. The README gains an explicit support-envelope section releasing v5.1.1 as a bounded, single-store exact-current performance patch. Five regression tests cover exactly the five reproductions; nothing speculative added. Suite green on this exact tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Interrupted-transition recovery refuses when the marker's prior manifest evidence cannot be loaded, keeping the marker and prior checkpoint in place, instead of accepting a reasons-only receipt and rebaselining blind. - Receipt deduplication is bound to the event: prior witness and manifest hashes, head, epoch and transition marker must all match before a receipt is reused. Equivalent drift from a different baseline writes its own receipt. - The corrupt transition marker is retired only after its preserved bytes verify and the receipt citing them is durable; a death or obstruction in between leaves the marker for the next attempt. - The benchmark cold read fails on command failure instead of timing a refusal, and a unit test drives the receipt store across the 200-receipt pruning window as the sequence crosses one million. Suite green on this exact tree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pre-merge red team on disposable copies of a live-shaped store (2026-09-02). Two reproduced implementation defects are fixed with regression tests, and three behaviours the adversarial lane observed are now stated in the docs. - Console output is UTF-8 on every host. A piped or redirected stdout on Windows defaulted to the ANSI code page, so get, recall and search died with a codec error on any entry carrying a character outside it (present since v5.1.0, latent on the development store). stdout and stderr are reconfigured to UTF-8 at startup; the test forces cp1252 through the environment so it runs on every CI leg. - The missing-lock refusal on read paths names the lifecycle commands that recreate the lock file instead of blaming the updater. - Stated: closed run logs sit outside the witnessed inventory; an interrupted authority write whose records the recovery pass cannot settle withholds the checkpoint until that run's own sync, with doctor listing the unfinished decisions; a killed propose may need a second command, or doctor, to finish healing. - MIGRATION: once a v5.1.1 proposal exists, v5.1.0 refuses status, doctor, recall and sync while its start opens only a degraded run; a start-and-close-only store stays readable by v5.1.0. Suite green on this exact tree: 254 tests, 0 failures, 11 skips. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Owner-ruled 2026-09-02. A full audit that condemned authority left the earlier checkpoint readable, so the next warm start printed a clean brief and exact reads kept serving a store whose writes were paused, against BIMRI-PROTOCOL 9.5. The round-one decision to keep a stale witness after a failed write path had not noticed the collision. - condemn_audit_checkpoint: when the full semantic audit condemns authority, advance the state's audit epoch past the checkpoint and save state. The witness bytes stay on disk as the prior baseline for drift receipts, quarantine and restore, the same retained-but-invalid shape a failed resolution leaves. Quarantine baselines, damaged prior evidence and strict restore comparisons keep their own contracts; a blocked receipt sink still keeps the prior checkpoint readable. - The next warm start re-proves the store and prints AUTHORITY RECOVERY NEEDED; get and recall --key refuse; exact byte restoration lets the next audit pass, receipt the epoch advance and republish. - The one test that asserted the old clean brief after a failed status now asserts the banner. New regression covers condemn, degrade, retained baseline, repair and republish. - CHANGELOG, README and PROTOCOL state the invalidation and the retained baseline. Suite green on this exact tree: 255 tests, 0 failures, 11 skips. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The ubuntu-latest / Python 3.8 leg of the d0b1929 run failed on test_pending_proposal_does_not_reserve_a_soft_tier_target: it compared Path.glob output against a two-element list in a fixed order, and directory order is filesystem-dependent (unsorted on ext4, sorted on NTFS). Both sides are now sorted. Engine unchanged; the other 254 tests passed on that leg and every leg passed on 562a497. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The top of the README now answers what BIMRI is and why it matters in the words people actually search for and use: persistent memory for AI agents that lasts across sessions, in plain files you own, working with Claude Code, OpenAI Codex and any local agent. The reader-facing half is built from the live agentguru.ai/bimri copy (the landlord frame, the one memory example, what you get, several agents one memory), states speed and test coverage at measured strength, places BIMRI as the memory layer of the compounding intelligence engine, and answers the questions people ask, including whether Claude Code remembers between sessions. The full technical documentation is unchanged in substance below a Reference divider; em dashes were removed throughout and every prior anchor still resolves. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The executable change is an engine-only performance patch. The authority format stays v5.1.0 and the readable hot-memory grammar stays v5.0.2, so existing stores update in place with
bimri.mdand every immutable record byte-identical.What ships:
.bimri/audit-witness.json) with the path-and-hash inventory kept as separate evidence. Warm exact-current reads,start,journaland no-proposalclosestop enumerating the historical authority tree.get --keyandrecall --keythat returns only the accepted current generation. Held candidates and history stay reachable through--history,--queryandreview..bimri/audit-drift/(bounded to the newest 200, complete per-path prior and current hashes, oversized deltas pinned in validated attachments) and only then rebaselines. A failing audit refuses into the existing damaged-authority recovery lane. No crash or drift state ever requires hand deletion of derived files.tests/benchmark_v511.py) with canonical gates, run by CI on the four-leg matrix, which now also triggers onagent/**pushes.Owner-ruled behavior changes (2026-08-27): warm reads defer whole-tree verification to authority-changing writes and explicit audit, review, search and historical-recall boundaries; plain exact recall is current-only. Authority writes measure about 1.3 s on the live-size store, recorded as an accepted deviation from the one-second brief target; incremental manifests are v5.2 work.
Measured on the development store (~500 runs, ~216 revisions, Windows 11):
recall --key, warm, CLI end to endstartjournalpropose/syncOne-way boundary: the first v5.1.1 proposal stamps a receipt that a v5.1.0 engine rejects permanently. Rollback exists only from the complete pre-update folder snapshot that MIGRATION.md now requires. A store the new engine has only started and closed is still readable by v5.1.0.
Support envelope: a bounded, single-store exact-current performance patch, validated on a store shaped like this project. Ranked or non-ASCII retrieval, large current-state sets, high reader concurrency, fleets and indefinite-lifetime storage are explicitly not claims of this release (README, "Support envelope").
Verification:
c467901, on562a497(round six: UTF-8 console output, clearer missing-lock message, three stated boundaries), on4baf61d(round seven: a condemning audit invalidates the checkpoint sostartprints the recovery banner, owner-ruled 2026-09-02; round eight: one test's directory-order comparison made deterministic), and exact-head run33656216072on6d24e06.