[m96-chan]

Summary

Second stage of #72: struct-typed properties were skipped outright, so their contents were unreachable from Python. On this machine 1512 properties across 904 providers are declared as structs, and a single Kernel-Processor-Power core-parking event carries ten of them — all previously missing from the property dict.

Changes

• Nested structures decode into EventValue::Struct. TDH addresses a struct member with a chain of PROPERTY_DATA_DESCRIPTORs — the struct with the index of the instance, then the member — so property_bytes grows into property_bytes_at, taking the whole path. A struct that is itself an array becomes an Array of Structs.
• Recursion is bounded by MAX_STRUCT_DEPTH. A schema whose members pointed back at an enclosing struct would otherwise not terminate.
• read_layout walks all PropertyCount properties, not just the top-level ones: struct members live past TopLevelPropertyCount, and structType addresses them by index into that same array. Only top-level entries become keys of the returned map.
• Fixes Property index alignment breaks if a property has no name, mis-resolving PropertyParamCount #81 as a consequence. Keeping that vector positionally aligned with EventPropertyInfoArray is what the struct work needs to be correct at all, and it is exactly what Property index alignment breaks if a property has no name, mis-resolving PropertyParamCount #81 asks for. Unnamed properties now hold their slot as a placeholder rather than being dropped and shifting every later index.
• New fixture nested_struct.etl + generator script, and four tests.

Related Issue

Closes #81

Advances #72. Deliberately not closing it — see below.

Test Plan

Added/Modified Tests

• tdh.rs: five new unit tests — depth guard, leaf_index nesting, placeholder handling, and both PropertyParamCount fallbacks (unnamed counter, out-of-range index). The last three pass a null record on purpose: reaching TDH would dereference it, so passing proves they short-circuit.
• test_etl_reader.py: four tests over the new fixture — members reachable, members not leaking to the top level, structs surviving to_dict(), nested values Python-native recursively.

Test Results

Verified against tracerpt on live events rather than only by unit test. For one core-parking event tracerpt reports:

<ComplexData Name="Group">       <Data Name="Number">0</Data> <Data Name="Affinity">0xFFFF</Data>
<ComplexData Name="OldPark">     <Data Name="Number">0</Data> <Data Name="Affinity">0x50C0</Data>
<ComplexData Name="NewPark">     <Data Name="Number">0</Data> <Data Name="Affinity">0x50C0</Data>
<ComplexData Name="NewSoftPark"> <Data Name="Number">0</Data> <Data Name="Affinity">0x50C0</Data>
<Data Name="UnparkCap">16</Data>
<Data Name="UnparkFloor">12</Data>

and this branch now produces, for the same event:

event_id=137 provider=Microsoft-Windows-Kernel-Processor-Power
  Group         dict  {'Number': 0, 'Affinity': 0xffff}
  OldPark       dict  {'Number': 0, 'Affinity': 0x50c0}
  NewPark       dict  {'Number': 0, 'Affinity': 0x50c0}
  NewSoftPark   dict  {'Number': 0, 'Affinity': 0x50c0}
  UnparkCap     int   16
  UnparkFloor   int   12
  ... 17 struct-valued properties in total

Arrays of structs, HardwareTable with 64 elements:

example array of structs: event 181 'HardwareTable' len=64
    {'LogicalProcessorIndex': 0, 'WorkLoadClassIndex': 0, 'PerformanceClass': 51, 'EfficiencyClass': 56}
    {'LogicalProcessorIndex': 0, 'WorkLoadClassIndex': 1, 'PerformanceClass': 62, 'EfficiencyClass': 60}
    {'LogicalProcessorIndex': 0, 'WorkLoadClassIndex': 2, 'PerformanceClass': 92, 'EfficiencyClass': 97}
    {'LogicalProcessorIndex': 0, 'WorkLoadClassIndex': 3, 'PerformanceClass': 41, 'EfficiencyClass': 50}

against tracerpt's 0x33 0x38, 0x3E 0x3C, 0x5C 0x61, 0x29 0x32 — every element matches. Note tracerpt flattens all 64 into one repeated list because XML cannot express the grouping; these stay grouped, which is the more useful shape.

Suite:

pytest        208 passed, 102 skipped     (the 102 are #77, fixed in #82)
cargo test     60 passed,   0 failed      (26 in tdh::)
cargo clippy --all-targets -- -D warnings  clean
cargo fmt --check / ruff / black           clean

Checklist

• Tests have been added/updated
• All tests pass
• Ready for code review
• Documentation updated (if applicable)
• CHANGELOG.md updated (if applicable)

Additional Notes

Why #72 stays open. Its proposed approach lists three more things this does not do, and I would rather they stay visible than be closed silently:

• WPP and providers with no installed manifest still yield an empty dict. This is not a gap in the code — TdhGetEventInformation has no schema to give — so it needs a different mechanism (TMF files for WPP), not more of this one. It is not rare: 73 of the 149 events in the old sample.etl had no resolvable schema.
• TdhFormatProperty string output is not offered as an alternative to typed values. Still an open design question from the issue.
• Arrays of structs have no fixture. The events carrying them fire rarely enough that capturing one reliably meant a multi-megabyte file. Covered by unit tests and the manual tracerpt comparison above, but not in CI.

The fixture is 40 KB, the same order as sample.etl (32 KB). Keeping it there took picking the Algorithm keyword (0x20) specifically; wider keyword masks add hundreds of KB of unrelated telemetry without adding a single struct.

One overlap to note: git status on this branch shows an untracked src/pyetwkit/pyetwkit_core.pdb, left by maturin develop. That is ignored by #83, not here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01HhPcm483PrDmuQEUSALBFn

[m96-chan]

Not merging yet — CI is red on all five Test Python jobs:

2 failed, 206 passed, 102 skipped
test_struct_properties_are_dicts_of_their_members FAILED
  AssertionError: no struct-valued property was decoded
  assert {}
test_struct_members_do_not_leak_into_the_top_level FAILED
  pytest.fail("no event carried a NewPark struct")

The implementation is not what I think is wrong. In the same run, on the same code path, the sample.etl tests pass:

TestGenericPropertyParsing::test_properties_are_not_limited_to_the_legacy_guess_list PASSED
TestGenericPropertyParsing::test_properties_round_trip_into_to_dict PASSED
TestGenericPropertyParsing::test_property_values_are_python_native_types PASSED

So TDH resolves a schema for Kernel-Process on the runner and decodes its properties, while Kernel-Processor-Power yields nothing at all. The ETL content is committed and identical everywhere, so the only variable is the machine: windows-latest is Windows Server, and it looks like it has no registered manifest for Microsoft-Windows-Kernel-Processor-Power — core parking being a client power feature. Worth confirming rather than taking from me, since I cannot run Windows: on a runner, does TdhGetEventInformation return a schema for those events, and are the scalar properties empty too, or only the structs?

Note the other two tests in the class passed, but both pass vacuously against an empty property dict, so they are not evidence either way.

What I would not do: turn these into skips when the schema is unavailable. That is how the ETL tests spent their whole existence green and inert, and #77 has only just finished undoing that. A test that quietly stops testing on the machine that matters is worse than no test.

Better options, roughly in order of preference:

1. Find a struct-carrying provider whose manifest ships on Server as well as client, and recapture. That keeps the assertion strong everywhere.
2. Synthesise the event rather than capturing it, if a hand-built EVENT_RECORD plus schema can drive the struct path in a Rust unit test. It removes the machine dependency entirely, and the struct-walking logic is exactly the sort of thing that deserves a deterministic test.
3. Keep the capture but assert on a struct-typed property from a provider proven present, so failure means the code is wrong rather than the host is different.

The code itself reads well, for whenever the test situation is sorted:

• Iterating 0..PropertyCount and pushing PropertyDesc::placeholder() for unnamed entries keeps the vector positionally aligned with EventPropertyInfoArray. That also fixes Property index alignment breaks if a property has no name, mis-resolving PropertyParamCount #81, which I filed against main yesterday — worth saying so in the description so the issue gets closed with this.
• top_level_count correctly separates "properties that become keys" from "properties the schema's indices can reach", which is the distinction the struct path needs.
• MAX_STRUCT_DEPTH bounds the recursion, and building a PROPERTY_DATA_DESCRIPTOR path per level is the right way to ask TDH for a nested member rather than computing offsets.

One question on element_count: the FromProperty branch tries the parent-relative path and then falls back to property_bytes(record, name, WHOLE_PROPERTY) at top level. Is that fallback reachable for a counter that genuinely lives inside a struct, and if so could it read an unrelated top-level property of the same name? A comment on why the fallback is safe would help the next reader.

Happy to merge once CI is green.

[m96-chan]

CI caught something the local run could not: the fixture decoded on my machine and produced zero struct properties on every runner.

The cause was the fixture, not the parser. It was a capture of Kernel-Processor-Power, a manifest provider, and decoding a manifest event needs that manifest installed and of a matching version. The runner is a different Windows build, so it had nothing to decode with. A fixture that only decodes where it was recorded is worse than no fixture — it looks like coverage.

Re-recorded from a TraceLogging provider defined inside the generator script. Those events embed their own schema, so the ETL decodes identically anywhere; that is a property of the format rather than something to hope for. Windows ships the C# compiler the script uses, so it still needs nothing beyond a stock machine, and the fixture drops 40 KB → 16 KB. Each event now carries two { Number, Affinity } structs with known values, so the tests assert contents rather than just shape.

Writing that fixture surfaced a second, unrelated bug, fixed in the following commit: TDH returns a counted string's two-byte length as part of the value, so it was decoded as content. The new fixture's Label holds "core-state" and came back as '\x14core-state'. The same thing was visible in an earlier TraceLogging capture where "hello-from-32bit" read as " hello-from-32bit" — that leading space was the 32-byte count, quiet enough that I missed it at the time. Affects win:CountedString, win:CountedAnsiString, win:CountedBinary and the MOF equivalents. The prefix is only stripped when it actually describes the rest of the buffer, so a value TDH already stripped keeps its first character.

Local: cargo test 66 passed, pytest 209 passed, clippy/fmt/ruff/black clean. #85 has been rebased on this.