Repository navigation
fix(storage): make the collection slot map safe for lock-free readers (#312) - #314
Merged
mrdevrobot merged 1 commit intoOct 18, 2026
Merged
Conversation
…EntglDb#312) In collection-per-file mode every read, write and free of a collection page resolves the page's slot to the collection name (GetPageFile -> GetCollectionNameBySlot) without _collectionSlotLock. The slot -> name map was a plain Dictionary<int, string>, written under the lock when a collection got its slot. A lookup that ran while the dictionary resized (at the 4th, 8th, 18th and 38th collection) could miss, and GetPageFile then sent the page to the main file: "Page 3254779953 is outside the page file (file holds 65 pages): the file is corrupt". The map is now a ConcurrentDictionary<int, string>, and a new slot is published slot -> name before name -> slot, so a thread that finds the slot by name (GetCollectionPageIds) also finds the file by the slot. The new test races slot lookups and page reads against all 63 slot assignments of an engine, 100 times; it fails in 10 of 10 runs on the old code (also with DOTNET_PROCESSOR_COUNT=2), within about a second. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CaffeinatedCoder
had a problem deploying
to
Production
October 17, 2026 14:10 — with
GitHub Actions
Failure
Contributor
|
Verified: the new test fails 3 of 3 on 0450b97 and passes here; full suite passes. Minimal change (ConcurrentDictionary plus reversed publication order). |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #312.
Problem
In collection-per-file mode (
PageFileConfig.Server), every read, write and free of a collection page resolves the page's slot to a collection name without taking_collectionSlotLock(GetPageFile→GetCollectionNameBySlot). The slot → name map was a plainDictionary<int, string>. It is written under that lock whenever a collection gets its slot.A lookup that runs while the dictionary resizes can miss. The resizes happen at the 4th, 8th, 18th and 38th collection. On a miss,
GetPageFilefalls through to the main file, and the operation fails withInvalidDataException: Page … is outside the page file.Change
StorageEngine.cs:_collectionSlotToNameis now aConcurrentDictionary<int, string>. The comment on the registry says why both maps are read without the lock.StorageEngine.Memory.cs: a new slot is published slot → name before name → slot, inGetOrAssignCollectionSlotand inLoadCollectionSlots. A thread that finds the slot by name, such asGetCollectionPageIdsfrom another session's collection instance, then also finds the file by the slot.MultiFileStorageTests.CollectionPageRouting_WhileCollectionsAreCreated_FindsTheCollectionFile. Reader threads call the private slot lookup through a delegate, and one of them reads a page of collectionc0throughReadPage. Meanwhile the test assigns the other 63 slots of the engine through the privateGetOrAssignCollectionSlot, which skips creating the files so that a round stays cheap. It runs 100 rounds.DOTNET_PROCESSOR_COUNT=2), first miss between rounds 14 and 63, about 1 sAn earlier version of the test, which created the 63 collection files and used only
ReadPagereaders, also failed on the old code with the issue's exact error (Page 3221225536 is outside the page file (file holds 65 pages)). It did so in only 2 of 6 runs, which is why the test calls the slot lookup directly.Tests
dotnet test tests/BLite.Tests -c Release: 3093 passed, 4 skipped, 0 failed.BLite.Corebuilds for net10.0 and netstandard2.1.BLite.NetStandard21.Testsbuilds, but I could not run it here because it needs the .NET 6 runtime.Independent of #313 (the checkpoint drain fix for #311): different files, no conflicts.
🤖 Generated with Claude Code