feat: unnamed union

docs/sphinx/source/library/fields/common.rst

@@ -294,6 +294,20 @@ Special Structs
 
     .. versionadded:: 2.9.0
 
+.. autoclass:: caterpillar.fields.Unnamed
+    :members:
+
+    .. versionadded:: 2.10.0
+
+.. autoclass:: caterpillar.fields.sized
+    :members:
+
+    Inline alternative to ``.sized(length)``, given as the last argument of
+    ``Unnamed``'s subscript instead: ``Unnamed[TypeA, TypeB, sized[length]]``.
+
+    At most one may be given, and it must be the last argument.
+
+    .. versionadded:: 2.10.0
 
 .. autodata:: caterpillar.fields.Pass
 

docs/sphinx/source/reference/datamodel/standard.rst

@@ -12,6 +12,7 @@ compatibility with older versions of the library, making them particularly impor
     sequence.rst
     struct
     union
+    unnamed
     bitfield
     templates
 

docs/sphinx/source/reference/datamodel/unnamed.rst

@@ -0,0 +1,71 @@
+.. _unnamed-reference:
+
+Unnamed
+=======
+
+**Unnamed** provides a different take on the same C-union idea as :ref:`union-reference`:
+instead of a whole struct where every field shares memory, :class:`~caterpillar.fields.Unnamed`
+is a single *field* whose value may be one of several possible types, all overlaid on the
+same fixed-size window of bytes. It is declared with a subscription syntax rather than a
+decorator:
+
+.. code-block:: python
+
+    >>> @struct
+    ... class Format:
+    ...     value: Unnamed[uint32, CString(4)]
+    ...
+
+Unlike :func:`~caterpillar.model.union`, an :class:`~caterpillar.fields.Unnamed` value does
+not eagerly decode every alternative. The fixed-size byte window is read once, immediately,
+like any other field - but each alternative is only decoded (and then cached) the first time
+it is actually requested, via :code:`value[i]` or the shorthand attribute form
+:code:`value._i`:
+
+.. code-block:: python
+
+    >>> obj = unpack(Format, b"\x01\x00\x00\x00ABC\x00")
+    >>> obj.value[0]
+    1
+    >>> obj.value._1
+    'ABC'
+
+Sizing
+------
+
+By default, the field's size is the *largest* of its member types' sizes - every member
+must therefore have a statically-known, fixed :code:`__size__`. An explicit size (an
+:code:`int` or a context lambda) can be supplied via :code:`.sized(...)` to override this,
+which is required if any member has a greedy or otherwise unsized :code:`__size__`:
+
+.. code-block:: python
+
+    >>> @struct
+    ... class Format:
+    ...     n: uint16
+    ...     value: Unnamed[uint32, uint16].sized(this.n)
+    ...
+
+Access
+------
+
+* :code:`value[i]` / :code:`value._i` - decode (or return the cached) member ``i``.
+  Negative indices are supported.
+* :code:`value[i:j]` - returns a plain :code:`tuple` of the decoded members in that range.
+* :code:`len(value)` - the number of members.
+* Iterating a value decodes and yields every member in order.
+
+An :class:`~caterpillar.fields.Unnamed` value can also be constructed directly, without a
+prior unpack - this is how you produce a value for packing from scratch:
+
+>>> Unnamed()                  # nothing assigned yet
+Unnamed()
+>>> Unnamed(1, 2, 3)           # positional: fills index 0, 1, 2
+Unnamed(1, 2, 3)
+>>> Unnamed(_3=0xBEEF)         # keyword: only index 3 is set
+Unnamed(?, ?, ?, 48879)
+
+Positional arguments fill indices left to right starting at ``0``; keyword arguments named
+``_<index>`` (e.g. ``_0``, ``_3``) set an arbitrary index directly.
+
+For more information and examples, see :ref:`tutorial-unnamed`.

docs/sphinx/source/tutorial/advanced/index.rst

@@ -22,6 +22,7 @@ create and manipulate your own custom struct classes in Python.
     actions
     bitfield
     union
+    unnamed
     templates
 
 

docs/sphinx/source/tutorial/advanced/unnamed.rst

@@ -0,0 +1,86 @@
+.. _tutorial-unnamed:
+
+Unnamed
+=======
+
+The previous section introduced :ref:`tutorial-union`, which mimics a C union at the level
+of a whole struct: every field of the struct shares the same memory, and assigning one field
+updates all the others. :class:`~caterpillar.fields.Unnamed` brings the same idea down to a
+single *field*: one position in a struct that may hold any one of several possible types, all
+read from - and written to - the very same window of bytes.
+
+Where a *union* struct eagerly keeps every field in sync, an :class:`~caterpillar.fields.Unnamed`
+field is lazy: the fixed-size bytes are read once, immediately, like any other field, but no
+particular interpretation of those bytes is decoded until you actually ask for it.
+
+A basic example
+----------------
+
+Suppose a record stores a 4-byte value that should sometimes be read as a 32-bit integer, and
+sometimes as a short ASCII tag. Instead of declaring two separate fields and manually keeping
+track of which one is meaningful, declare a single :code:`Unnamed[...]` field listing both
+possible types:
+
+.. code-block:: python
+    :caption: A field that can be read as either an integer or a tag
+
+    @struct
+    class Record:
+        kind: uint16
+        value: Unnamed[uint32, CString(4)]
+
+Unpacking reads the 2-byte :code:`kind` field plus exactly 4 bytes for :code:`value` - the
+size of the largest member - up front. Neither :code:`uint32` nor :code:`CString` is decoded
+yet at this point:
+
+>>> obj = unpack(Record, b"\x01\x00ABCD")
+>>> obj.value[0]          # decode as uint32, then cache it
+1145258561
+>>> obj.value._1          # decode as CString, then cache it
+'ABCD'
+>>> obj.value
+Unnamed(1145258561, 'ABCD')
+
+Both :code:`value[i]` (works with any integer, including negative indices) and the shorthand
+attribute form :code:`value._i` are equivalent.
+
+Sizing
+------
+
+By default, the field is sized as the *largest* member (so every member must have a fixed,
+statically-known size). Use :code:`.sized(...)` to declare an explicit size instead - a plain
+:code:`int`, or a context lambda evaluated against the enclosing struct, exactly like sizing a
+sequence:
+
+.. code-block:: python
+    :caption: Explicit, context-dependent sizing
+
+    @struct
+    class Record:
+        length: uint16
+        value: Unnamed[uint32, uint16].sized(this.length)
+        # using sized[...] expression is also possible (must be last item)
+        .. value: Unnamed[uint32, uint16, sized[this.length]]
+
+Building a value for packing
+------------------------------
+
+An :class:`~caterpillar.fields.Unnamed` value does not have to come from :code:`unpack(...)` -
+you can construct one directly to pack it. All of the following are valid:
+
+>>> Unnamed()                  # nothing assigned yet - cannot be packed like this
+Unnamed()
+>>> Unnamed(1, 2, 3)           # positional arguments fill index 0, 1, 2, ...
+Unnamed(1, 2, 3)
+>>> Unnamed(_1="XY")           # keyword arguments (``_<index>``) set one index directly
+Unnamed(?, 'XY')
+
+Since a positional-then-keyword call is processed left to right, whichever index was assigned
+*last* is the one that ends up encoded when the value is packed.
+
+>>> pack(Record(kind=1, value=Unnamed(_1="XY")))
+b'\x01\x00XY\x00\x00'
+>>> pack(Record(kind=2, value=Unnamed(0xCAFEBABE)))
+b'\x02\x00\xbe\xba\xfe\xca'
+
+

src/caterpillar/_common.py

@@ -61,7 +61,10 @@ def __enter__(self) -> None:
         self.context[self.name] = self.value
 
     def __exit__(
-        self, exc_type: type, exc_value: Exception, traceback: TracebackType
+        self,
+        exc_type: type | None,
+        exc_value: Exception | None,
+        traceback: TracebackType | None,
     ) -> None:
         self.context[self.name] = self.old_value
         # We have to apply the right field as instance of the Field class

src/caterpillar/fields/_base.py

@@ -65,6 +65,21 @@ def singleton(cls: type[_T]) -> _T:
 INVALID_DEFAULT: object = object()
 DEFAULT_OPTION: object = object()
 
+# Constant representing a default value that caterpillar must not use as a
+# concrete value.
+IGNORED_DEFAULT: object = object()
+
+
+def has_default(value: object) -> bool:
+    """
+    Check whether *value* is a concrete, usable default value.
+
+    :param value: The value to check, typically a :attr:`Field.default`.
+    :return: True if *value* can be safely used as a fallback/packing value.
+    """
+    return value is not INVALID_DEFAULT and value is not IGNORED_DEFAULT
+
+
 
 class Field(Generic[_IT, _OT], PackMixin[_IT], UnpackMixin[_OT]):
     """Represents a field in a data structure.
@@ -526,7 +541,7 @@ def __unpack__(self, context: _ContextLike) -> _OT:
                 if not isinstance(exc, StructException):
                     exc = StructException(str(exc), context)
                 value = self.default
-                if value is INVALID_DEFAULT or isinstance(exc, ValidationError):
+                if not has_default(value) or isinstance(exc, ValidationError):
                     raise exc
                 return value
 
@@ -561,7 +576,7 @@ def __unpack__(self, context: _ContextLike) -> _OT:
                     exc = StructException(str(exc), context)
                 # Any exception leads to a default value if configured
                 value = self.default
-                if value is INVALID_DEFAULT or isinstance(exc, ValidationError):
+                if not has_default(value) or isinstance(exc, ValidationError):
                     raise exc
         else:
             # Context functions should be executed with top priority

src/caterpillar/fields/common.py

@@ -53,7 +53,7 @@
 from caterpillar.options import GLOBAL_FIELD_FLAGS, Flag
 from caterpillar.shared import ATTR_BYTEORDER, getstruct, typeof
 
-from ._base import INVALID_DEFAULT, Field, singleton
+from ._base import Field, INVALID_DEFAULT, singleton, has_default
 from ._mixin import ByteOrderMixin, FieldStruct
 
 # Explicitly report deprecation warnings
@@ -924,7 +924,7 @@ def decode(self, parsed: _IT, context: ContextLike) -> _EnumT:
         default = self.default
         field = context.get(CTX_FIELD)
         if default is INVALID_DEFAULT and field:
-            default = field.default
+            default = field.default if has_default(default) else INVALID_DEFAULT
 
         if default is INVALID_DEFAULT:
             if (field and field.has_flag(ENUM_STRICT)) or self.strict: