Skip to content

fix(gdpr): Strict mode no longer claims deleted documents are not zeroed (#299) - #318

Open
CaffeinatedCoder wants to merge 2 commits into
EntglDb:mainfrom
CaffeinatedCoder:fix/299-gdpr-erase-warning
Open

CaffeinatedCoder wants to merge 2 commits into
EntglDb:mainfrom
CaffeinatedCoder:fix/299-gdpr-erase-warning

Conversation

@CaffeinatedCoder

Copy link
Copy Markdown
Collaborator

What was wrong

Opening a database with GdprMode.Strict traced event 9006: "SecureEraseOnDelete is not yet available as an engine-level setting. Deleted document slots are not currently zeroed at the page layer. Track MISSING_FEATURES.md §3 for the planned implementation." (#299). The second sentence has been untrue since #96: DocumentCollection.DeleteCore and SlottedPageUtils.CompactAndErase zero-fill the bytes a deleted document occupied, and PageFile.FreePage rewrites a freed overflow page zeroed. docs/DPIA_CHECKLIST.md gave a third answer ("call VacuumAsync after deletion"). A GDPR review reading them concludes that erasure needs extra work.

The one true caveat is timing: a delete writes the zeroed page through the WAL, and the database file receives it at the next checkpoint. Until then the .db file can still hold the deleted bytes (a DocumentDbContext checkpoints on dispose; a BLiteEngine does not).

Fix

  • Event 9006 now says what holds: deleted documents are zero-filled on their page, the database file receives the zeroed page at the next checkpoint, call CheckpointAsync() after deleting personal data, and VacuumAsync(SecureErase = true) additionally zero-fills the free space of every page. The constant is renamed GdprStrictSecureEraseCheckpointHint (internal; the id 9006 and the TraceWarning level are unchanged, so existing listeners see the same event). The pointer to MISSING_FEATURES.md is gone.
  • GdprMode.Strict XML doc: same correction.
  • docs/DPIA_CHECKLIST.md: the residual-data row, the two checklist items and the Art. 17 row say delete + checkpoint, with vacuum as a maintenance step, and that zero-filling covers the database file only (not the OS page cache, SSD wear-levelling or swap, which the encryption Strict mode requires covers).

The README section is corrected in #298 (README.md:338-351 there); this PR does not touch the README to avoid a conflict.

Tests

GdprStrictTests:

  • GdprStrict_SecureErase_EmitsCheckpointHint_NotAnUnavailableWarning replaces the old test: exactly one 9006 event, it names CheckpointAsync and "zero-filled", and contains none of "not currently zeroed", "not yet available", "MISSING_FEATURES". Fails on upstream/main on the wording.
  • Delete_ThenCheckpoint_LeavesNoBytesOfTheDocumentInTheFiles (new): inserts a document carrying a marker string into a plaintext BLiteEngine, checkpoints and confirms the marker is in the .db file; reopens, deletes, checkpoints; the marker occurs zero times in the .db and the .wal file. This is the behaviour the event describes, and the GDPR row of the oracle plan in Epic: reliability before features — durability, AOT, query correctness and oracle coverage #288.

Full suite dotnet test tests/BLite.Tests (macOS, net10.0): 3090 passed, 4 skipped, 3 failed, all TimeoutException on a lock (BTreeSplitTests ×2, IndexPageReleaseTests ×1) while a second suite ran on the same machine; the #303 class of failure, see #302. All 35 tests of those two classes pass when rerun on their own.

Fixes #299

🤖 Generated with Claude Code

…oed (EntglDb#299)

Event 9006 said "Deleted document slots are not currently zeroed at the page
layer" and pointed at MISSING_FEATURES.md §3. Both delete paths have
zero-filled the deleted bytes since EntglDb#96, and freed overflow pages are
rewritten zeroed. The one true caveat is timing: the zeroed page reaches the
database file at the next checkpoint.

The event now says that, and asks for CheckpointAsync() after deleting
personal data. docs/DPIA_CHECKLIST.md says the same instead of "call
VacuumAsync after deletion" (vacuum remains a maintenance step).
GdprStrictSecureEraseUnavailable is renamed GdprStrictSecureEraseCheckpointHint;
the id 9006 is unchanged.

Tests: the Strict-mode test asserts the new wording and the absence of the
old claims; a new test deletes a document, checkpoints, and finds no byte of
it in the .db or .wal file.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@CaffeinatedCoder CaffeinatedCoder added bug Something isn't working package/core Affects BLite.Core gdpr GDPR compliance primitives labels Oct 18, 2026

@mrdevrobot mrdevrobot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested. Verified that deletes zero-fill and that the new 9006 text is accurate. docs/DPIA_CHECKLIST.md says a DocumentDbContext checkpoints on dispose; only DisposeAsync() does, Dispose() does not (DocumentDbContext.cs ~590–627). Please say DisposeAsync there.

)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CaffeinatedCoder

Copy link
Copy Markdown
Collaborator Author

Done in 0e875ff: docs/DPIA_CHECKLIST.md now says a DocumentDbContext checkpoints in DisposeAsync(), not in Dispose().

This branch was successfully deployed

1 active deployment
Production — 0e875ffc Deployed Oct 19, 2026 by CaffeinatedCoder via build_and_publish #796
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working gdpr GDPR compliance primitives package/core Affects BLite.Core

Projects

None yet

Development

Successfully merging this pull request may close these issues.

GdprMode.Strict warns that deleted documents are not zeroed, but every delete zero-fills them

2 participants