Skip to content

fix(writing): correct how an attribute handles null - #186

Merged
Apollo3zehn merged 4 commits into
Apollo3zehn:devfrom
Blackclaws:fix/attribute-null-handling
Oct 21, 2026
Merged

Apollo3zehn merged 4 commits into
Apollo3zehn:devfrom
Blackclaws:fix/attribute-null-handling

Conversation

@Blackclaws

Copy link
Copy Markdown
Contributor

Two defects in how an attribute handles null, both reachable on dev today and neither caused by #179 — which is where the first of them was reported.

A null dataspace reserves data it has none of

A null dataspace has no data elements, but the data size was derived from the dataspace dimensions, of which a null dataspace has none — and the product of no dimensions is one. So one element's worth of space was reserved, written to the file, and, for a scalar datatype, asked for.

Asking for it threw IndexOutOfRangeException, because a scalar datatype is encoded by indexing element zero:

new H5File { Attributes = { ["a"] = new H5Attribute<int>() } }.Write(stream);
// System.IndexOutOfRangeException

No strings involved. new H5Attribute<string>() throws the same way, and new H5Attribute<int[]>() survives only because the array encoder walks a span instead of indexing — while still padding its message with a datatype's worth of zeroes.

The C library writes no data for a null dataspace whatever the datatype. Its attribute messages measure 34 bytes for a null-dataspace int and 44 for a null-dataspace variable-length string, both exactly name plus datatype plus dataspace with nothing after them. PureHDF now matches.

A scalar null becomes an absence of elements

WriteUtils.ToMemory is handed an object?, so a null reference is indistinguishable there from an attribute carrying no data at all. It answers with no dimensions, and no dimensions is how DataspaceMessage.Create is asked for a null dataspace.

The result cannot be read back — the reader refuses an attribute with a null dataspace — so a caller who wrote null had no way to recover it. A nullable value type was already correct, because ToMemory answers it as one element, so int? and string? disagreed about what a null meant.

The C library settles what the right representation is: H5Awrite accepts a null char* for a variable-length string and encodes it as a zero length and a zero global heap ID. PureHDF already wrote exactly those bytes for a null element inside an array, and now reaches them for a scalar too. The 16 data bytes are byte-identical to the C library's, h5dump prints NULL rather than "", and PureHDF reads the C library's own file back as null.

Worth knowing for anyone writing a repro: new H5Attribute<string?>(default) binds to the null-dataspace constructor, not the value one. The value case needs new H5Attribute<string?>((string?)null).

One thing this does not close

An attribute with a null dataspace can now be written correctly, but PureHDF still cannot read one — NativeAttribute refuses it with "Attributes with null dataspace cannot be read." So new H5Attribute<int>() goes from throwing on write to producing a valid file that PureHDF itself will not read back, and h5py's Empty attributes have the same problem in the other direction.

That is a separate gap and not something this PR tries to settle, but it seemed better to say so than to let it be discovered. Removing the guard looks close to free — the element count is already zero for a null dataspace, so an array-typed read falls out as an empty array; only the scalar case needs a clearer message than the element-count mismatch it currently produces. Happy to follow up if you want it.

Tests

CanWrite_Space_Null_ScalarDatatype pins the first against an h5dump fixture; CanWrite_Space_Scalar_NullValue pins the second on a round trip.

🤖 Generated with Claude Code

Blackclaws and others added 4 commits October 21, 2026 13:09
A null dataspace has no data elements, so the attribute message carries none. The
size was derived from the dataspace dimensions, of which a null dataspace has none,
and the product of no dimensions is one - so one element's worth of space was
reserved, written to the file, and, for a scalar datatype, asked for.

Asking for it threw IndexOutOfRangeException, because a scalar datatype is encoded
by indexing element zero. That made `new H5Attribute<int>()` and
`new H5Attribute<string>()` unwritable; `new H5Attribute<int[]>()` survived only
because the array encoder walks a span, and still padded the message with a
datatype's worth of zeroes.

The HDF5 C library writes no data for a null dataspace whatever the datatype. Its
attribute messages measure 34 bytes for a null-dataspace int and 44 for a
null-dataspace variable-length string, both of which are exactly name, datatype and
dataspace with nothing after them; PureHDF now matches byte for byte.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A scalar attribute whose value is null produced a null dataspace, which holds no
elements at all. WriteUtils.ToMemory is handed an object?, so a null reference is
indistinguishable there from an attribute carrying no data, and it answers with no
dimensions - which is how DataspaceMessage.Create is asked for a null dataspace.

The result could not be read back: an attribute with a null dataspace is refused by
the reader, so a caller who wrote null had no way to recover it. A nullable value
type was already correct, because ToMemory answers it as one element, so int? and
string? disagreed about what a null meant.

The HDF5 C library accepts a null pointer as a variable-length string value and
encodes it as a zero length and a zero global heap ID. PureHDF already wrote that
form for a null element inside an array, and now reaches it for a scalar too: the
16 data bytes are byte-identical to the C library's, h5dump prints NULL rather than
"", and PureHDF reads the C library's own file back as null.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Apollo3zehn
Apollo3zehn force-pushed the fix/attribute-null-handling branch from fdf4231 to 9c820db Compare October 21, 2026 11:26
@Apollo3zehn
Apollo3zehn merged commit 772d78b into Apollo3zehn:dev Oct 21, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants