Repository navigation
fix(writing): correct how an attribute handles null - #186
Merged
Apollo3zehn merged 4 commits intoOct 21, 2026
Merged
Conversation
A null dataspace has no data elements, so the attribute message carries none. The size was derived from the dataspace dimensions, of which a null dataspace has none, and the product of no dimensions is one - so one element's worth of space was reserved, written to the file, and, for a scalar datatype, asked for. Asking for it threw IndexOutOfRangeException, because a scalar datatype is encoded by indexing element zero. That made `new H5Attribute<int>()` and `new H5Attribute<string>()` unwritable; `new H5Attribute<int[]>()` survived only because the array encoder walks a span, and still padded the message with a datatype's worth of zeroes. The HDF5 C library writes no data for a null dataspace whatever the datatype. Its attribute messages measure 34 bytes for a null-dataspace int and 44 for a null-dataspace variable-length string, both of which are exactly name, datatype and dataspace with nothing after them; PureHDF now matches byte for byte. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A scalar attribute whose value is null produced a null dataspace, which holds no elements at all. WriteUtils.ToMemory is handed an object?, so a null reference is indistinguishable there from an attribute carrying no data, and it answers with no dimensions - which is how DataspaceMessage.Create is asked for a null dataspace. The result could not be read back: an attribute with a null dataspace is refused by the reader, so a caller who wrote null had no way to recover it. A nullable value type was already correct, because ToMemory answers it as one element, so int? and string? disagreed about what a null meant. The HDF5 C library accepts a null pointer as a variable-length string value and encodes it as a zero length and a zero global heap ID. PureHDF already wrote that form for a null element inside an array, and now reaches it for a scalar too: the 16 data bytes are byte-identical to the C library's, h5dump prints NULL rather than "", and PureHDF reads the C library's own file back as null. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Apollo3zehn
force-pushed
the
fix/attribute-null-handling
branch
from
October 21, 2026 11:26
fdf4231 to
9c820db
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two defects in how an attribute handles null, both reachable on
devtoday and neither caused by #179 — which is where the first of them was reported.A null dataspace reserves data it has none of
A null dataspace has no data elements, but the data size was derived from the dataspace dimensions, of which a null dataspace has none — and the product of no dimensions is one. So one element's worth of space was reserved, written to the file, and, for a scalar datatype, asked for.
Asking for it threw
IndexOutOfRangeException, because a scalar datatype is encoded by indexing element zero:No strings involved.
new H5Attribute<string>()throws the same way, andnew H5Attribute<int[]>()survives only because the array encoder walks a span instead of indexing — while still padding its message with a datatype's worth of zeroes.The C library writes no data for a null dataspace whatever the datatype. Its attribute messages measure 34 bytes for a null-dataspace
intand 44 for a null-dataspace variable-length string, both exactly name plus datatype plus dataspace with nothing after them. PureHDF now matches.A scalar null becomes an absence of elements
WriteUtils.ToMemoryis handed anobject?, so a null reference is indistinguishable there from an attribute carrying no data at all. It answers with no dimensions, and no dimensions is howDataspaceMessage.Createis asked for a null dataspace.The result cannot be read back — the reader refuses an attribute with a null dataspace — so a caller who wrote null had no way to recover it. A nullable value type was already correct, because
ToMemoryanswers it as one element, soint?andstring?disagreed about what a null meant.The C library settles what the right representation is:
H5Awriteaccepts a nullchar*for a variable-length string and encodes it as a zero length and a zero global heap ID. PureHDF already wrote exactly those bytes for a null element inside an array, and now reaches them for a scalar too. The 16 data bytes are byte-identical to the C library's,h5dumpprintsNULLrather than"", and PureHDF reads the C library's own file back asnull.Worth knowing for anyone writing a repro:
new H5Attribute<string?>(default)binds to the null-dataspace constructor, not the value one. The value case needsnew H5Attribute<string?>((string?)null).One thing this does not close
An attribute with a null dataspace can now be written correctly, but PureHDF still cannot read one —
NativeAttributerefuses it with "Attributes with null dataspace cannot be read." Sonew H5Attribute<int>()goes from throwing on write to producing a valid file that PureHDF itself will not read back, and h5py'sEmptyattributes have the same problem in the other direction.That is a separate gap and not something this PR tries to settle, but it seemed better to say so than to let it be discovered. Removing the guard looks close to free — the element count is already zero for a null dataspace, so an array-typed read falls out as an empty array; only the scalar case needs a clearer message than the element-count mismatch it currently produces. Happy to follow up if you want it.
Tests
CanWrite_Space_Null_ScalarDatatypepins the first against anh5dumpfixture;CanWrite_Space_Scalar_NullValuepins the second on a round trip.🤖 Generated with Claude Code