Skip to content

feat(writing): size strings per dataset and attribute - #203

Open
Blackclaws wants to merge 1 commit into
Apollo3zehn:devfrom
Blackclaws:feat/string-info
Open

Blackclaws wants to merge 1 commit into
Apollo3zehn:devfrom
Blackclaws:feat/string-info

Conversation

@Blackclaws

Copy link
Copy Markdown
Contributor

Closes #138. Independent of the writing stack in #190; based on dev.

H5WriteOptions sizes strings for the whole file. A plain string dataset only has DefaultStringLength, and a compound member is sized by the field and property mappers, which see the same FieldInfo or PropertyInfo for every dataset of a row type. So many tables of one row type cannot have different column widths.

This adds H5StringInfo, passed to the H5Dataset and H5Attribute constructors next to opaqueInfo, as the issue suggests:

new H5Dataset(names, stringInfo: new H5StringInfo(Length: 12));

new H5Dataset(rows, stringInfo: new H5StringInfo(
    MemberLength: member => member.Name switch
    {
        nameof(Row.Name) => 32,
        nameof(Row.Unit) => 8,
        _ => null
    }));
Property Effect
Length Takes the place of DefaultStringLength for the object, and of AttributeStringLength for an attribute: 0 for variable-length, a positive value for fixed-length, null to keep the file-wide setting. It is also the width of a compound string member that nothing more specific sizes.
MemberLength Called for each string field or property of the element type, before FieldStringLengthMapper and PropertyStringLengthMapper. Returning null leaves the member to them.
Padding H5StringPadding.NullTerminate, NullPad or SpacePad for the object's fixed-length strings, compound members included. null keeps what the writer declares today.

Worth flagging

  • No character set. The writer encodes every string as UTF-8 and the reader decodes ASCII as UTF-8, so declaring ASCII would change the label but not the bytes. I left it out until a reader needs it.
  • The datatype cache keys on the widths MemberLength returns, not on the delegate. A caller that builds an H5StringInfo per dataset passes a new delegate each time, and keying on it would give every dataset a compound message of its own. Datasets asking for the same widths share one.
  • MemberLength covers the element type's own members. A compound nested inside it is sized by the mappers and Length. Covering nested members would mean walking the type a second time to build the cache key.
  • SpacePad fills the field with spaces. The reader already strips trailing spaces for it, so a value's own trailing spaces do not survive.
  • An attribute given an explicit Length keeps it. The fallback to variable-length when a measured width makes the attribute message too large applies only to measured widths.
  • Binary compatibility. stringInfo is a new optional constructor parameter. Source compiles unchanged; assemblies built against an earlier version need recompiling.
  • Overlap with perf(writing): hold less per object while writing #195, which moves an H5Dataset's rarely set options into a record of their own. Whichever is merged second moves StringInfo there; I'll rebase.
  • doc/writing/data-types.md has a new section. No CHANGELOG.md entry, assuming you would rather write it yourself.

StringInfoTests covers:

  • plain string datasets of different widths
  • Length: 0 overriding a DefaultStringLength
  • two datasets of one row type with different member widths
  • the precedence of MemberLength, the mappers and Length
  • one shared datatype per distinct set of widths across 50 datasets
  • each padding, including the bytes a SpacePad field holds and h5dump reporting H5T_STR_SPACEPAD
  • a dataset declared through BeginWrite and written later
  • StringOverflow under a per-object width
  • negative widths
  • attribute precedence over AttributeStringLength
  • member widths on a compound attribute

🤖 Generated with Claude Code

H5StringInfo, passed to an H5Dataset or H5Attribute constructor, sizes the
strings of that one object in place of the file-wide options: Length for
its strings, MemberLength for the string members of its compound type
(consulted before the field and property mappers), and Padding for its
fixed-length strings, including SpacePad.

Datasets of one element type can now have different member widths. The
datatype cache keys on the widths a member length resolves to rather
than on the delegate, so objects asking for the same widths share one
message.

Closes Apollo3zehn#138
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

String size cannot be set for individual datasets

1 participant