[Blackclaws]

Every checksummed structure - object headers, the superblock, the fixed array index header and data block - was written to the stream, read back, checksummed from what was read, and the checksum written after it. That costs an async read per structure, and on a buffered FileStream each read-back flushes the write buffer first.

Each structure is now encoded into a buffer (H5SpanWriteDriver, pooled for object headers), checksummed there, and written in one piece. A chunked dataset's object header, which is patched once its index is known, is kept as an encoded copy and patched in memory instead of being read back from the file. Only chunked datasets keep that copy, since they are the only ones whose header changes after it is written.

The writer no longer reads from its destination at all. This PR does not lift the CanRead requirement on the stream - that would be a behaviour change of its own - but it is what makes lifting it possible; the TODO next to the check now says so.

Numbers

300,000 datasets of 6 doubles each, declared through BeginWrite and then written one by one; median of 3 runs, dev against this branch:

                      BeginWrite      Dispose       allocated         peak RSS
contiguous            3.27 -> 1.96 s  0.01 -> 0.01  1531 -> 1295 MB   316 -> 302 MB
chunked, deflate      4.02 -> 2.90 s  2.28 -> 1.67  2274 -> 2036 MB   584 -> 625 MB

The chunked case keeps about 40 MB more resident: the encoded header copies of 300,000 chunked datasets, which replace reading each header back at Dispose.

Tests

WriteWithoutReadingTests writes through a stream that throws on any read - deferred contiguous data, a single filtered chunk, several filtered chunks (fixed array index), variable-length strings and attributes - and checks the result with h5dump. To make that check able to fail, TestUtils gains RunH5Dump, which reports the exit code and both streams unmodified: DumpH5File strips h5dump's error stack so that the fixture comparisons do not depend on the installed version, which also strips every trace of a rejected file. A deliberately corrupted checksum fails all four tests.

🤖 Generated with Claude Code