[CaffeinatedCoder]

What was wrong

A typed DocumentCollection stores a document larger than one page (about 16 KB with the default page size) across overflow pages. The schema-less API could not read such a collection: DynamicCollection.FindAllAsync threw NotSupportedException: Skipping type 69 not supported at position 7 at the first large document, and so did everything built on it: FindByIdAsync, ScanAsync, BLQL, BLite Studio's collection view, and BLiteMigration.ToMultiFileAsync / ToSingleFileAsync, which copy every collection through the schema-less API (#301). The migration also left the empty multi-file layout (.idx, wal/, collections/) next to the source.

Mechanism

A slot flagged SlotFlags.HasOverflow holds [TotalLength (4)] [FirstOverflowPage (4)] [first chunk], and each overflow page holds PageSize - SlottedPageHeader.Size bytes after its header and names the next page in NextOverflowPage (DocumentCollection.InsertWithOverflow). DynamicCollection never checked the flag and handed the slot bytes to BsonSpanReader as a document; byte 4 of the payload is a page id, not a BSON type.

Fix

• Storage/OverflowChainReader.cs (new, internal): reassembles the document from that layout, sync (Read) and async (ReadAsync, copying what it needs from the page buffer before the first await), returning null rather than a partial document when a chain is inconsistent; ChainPages lists the chain's page ids and stops at a page that is not an overflow page. It is the same loop DocumentCollection has in three places (ReadRawBytesAt, ReadDocumentAt, its async twin); those are left as they are in this PR, so the typed hot path does not change.
• DynamicCollection: ReadDocumentAt, ReadDocumentAtAsync and ScanAsync (whose predicate now sees the whole document) use the reader. Every public read path goes through these three (FindAllAsync, FindByIdAsync, CountAsync, QueryIndexAsync, BLQL, the LINQ provider). DeleteSlot frees the chain with FreePageOnCommit, as DocumentCollection.FreeOverflowChain does, so a schema-less delete, or an update that relocates the document (TryUpdateInPlace already refuses a flagged slot), leaks no pages.
• Writes: DynamicCollection still does not write overflow pages. The limit is now public as DynamicCollection.MaxDocumentSize (page size minus header, slot and reserve), the rejection message names it, and the README's schema-less section says so. The write side is DynamicCollection cannot write a document larger than one page: no overflow on the schema-less write path #320; Moving from LiteDB: a guide, an API mapping and an import tool #297 (LiteDB import) will need it.
• BLiteMigration: before inserting a document into the target DynamicCollection, a document larger than MaxDocumentSize is refused with a NotSupportedException that names the collection, _id and size and says the source is unchanged, instead of a BLiteValidationException from inside InsertBulkAsync. On failure ToMultiFileAsync now also removes the companion files it created (.idx, the WAL directory, the collection directory; ones that existed before the call are kept), and ToSingleFileAsync removes a target file it created at another path.

No format change. Overflow pages are read exactly as the typed API reads them; nothing is written differently.

Tests

DynamicOverflowReadTests, 9 tests. Each writes a small user and one with a 40,000-character name through TestDbContext, then uses BLiteEngine:

Test	upstream/main	this PR
FindAllAsync returns both, the large one whole; CountAsync is 2	NotSupportedException: Skipping type 171	passes
FindByIdAsync of the large one	same exception	passes
BLQL Eq, OrderByDescending, and Gte through a new index on age	same exception	passes
ScanAsync predicate sees all 40 KB	empty result	passes
Typed delete of the large document is seen by the schema-less API	passes	passes
Schema-less delete is seen by the typed API; a new 40 KB document then fits in the freed pages (file grows by less than 40 KB)	fails: cannot delete	passes
Schema-less update replaces the large document for both APIs	passes (the chain leaked)	passes
Schema-less insert over MaxDocumentSize is rejected naming the limit	does not compile (new API)	passes
ToMultiFileAsync refuses by collection and _id; source intact for both APIs; no .migrating, .idx, WAL or collection directory left	fails: the message is the BSON reader's, and .idx / wal/ / collections/ remain	passes

Full suite dotnet test tests/BLite.Tests (macOS, net10.0): 3101 passed, 0 failed, 4 skipped. BLite.Core also builds for netstandard2.1.

Compatibility

Reads that threw now return the document. DynamicCollection.MaxDocumentSize is a new public property. The two rejection messages for an oversized write changed wording ("exceeds the N bytes a DynamicCollection document may occupy (one page)"); the exception type is unchanged. BLiteMigration throws NotSupportedException for an oversized document where it threw NotSupportedException from the BSON reader before. netstandard2.1 and AOT are unaffected.

Fixes #301

🤖 Generated with Claude Code

[mrdevrobot]

Changes requested. Verified: the schema-less reads of overflow documents fail on 0450b97 and pass here. The PR adds an unrequested public DynamicCollection.MaxDocumentSize; make it internal (the tests see internals through InternalsVisibleTo) and drop the property name from the README note.

[CaffeinatedCoder]

Done in df2e66c: DynamicCollection.MaxDocumentSize is now internal (the tests read it through InternalsVisibleTo), and the README note says "about 16 KB of BSON with the default page size" without the property name.