[Blackclaws]

Two independent changes to the element-wise decode path.

Fixed-length strings are trimmed as bytes

The decoder turned the whole field into a string, padding included, and then split it at the terminator: two strings the width of the field for every element. A wide declared width is cheap on disk once compressed - padding deflates to almost nothing - and the reader undid that. For 100,000 elements 8,192 bytes wide that was 3.1 GB of garbage to return 5 MB of text.

The field is now trimmed as bytes by the same rules - at the first NUL for null-terminated strings, of trailing NULs or spaces for null- and space-padded ones - and only the value is decoded. In UTF-8 a 0x00 or 0x20 byte is only ever that character, never part of a longer sequence, so cutting there never splits a character. The read is about six times faster on that case. FixedLengthStringReadCostTests pins the allocation.

The bulk read before element-wise decoding is bounded

Reference, compound and variable-length decoding bulk-read the selection into a pooled buffer and decode from memory (#163, and the contiguous compound path). The bulk read took the whole selection at once, so the pool rented a buffer as large as the selection rounded up to a power of two, and kept it: 1 GB for a 781 MB column. DecodeInBatches now reads 1 MB at a time and decodes each batch before reading the next. BatchedDecodeTests covers batch boundaries for fixed-size and variable-length elements.

Related to #148. #149 targets variable-length strings in the global heap path; this touches the fixed-length path and the bulk-read buffer, so the two do not overlap in behaviour, though both edit DatatypeMessage.Reading.cs.

🤖 Generated with Claude Code