Skip to content

fix(agent): refuse remaining invisible metadata code points - #2170

Merged
Benoît Cortier (CBenoit) merged 26 commits into
masterfrom
claude/metadata-gaps-6b80dd
Oct 28, 2026
Merged

Benoît Cortier (CBenoit) merged 26 commits into
masterfrom
claude/metadata-gaps-6b80dd

Conversation

@CBenoit

@CBenoit Benoît Cortier (CBenoit) commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Metadata values exclude invisible format characters so that no value can reorder or hide text where an administrator reads it. The list still let through code points that render invisibly without being format controls, such as the Hangul fillers, the combining grapheme joiner and the variation selectors, so a host name could still hide text.

The Agent Identity contract now lists every Unicode default-ignorable code point, reserved ones included so that a character assigned there later is covered, plus the interlinear annotation characters U+FFF9–U+FFFB: U+00AD, U+034F, U+061C, U+115F–U+1160, U+17B4–U+17B5, U+180B–U+180F, U+200B–U+200F, U+202A–U+202E, U+2060–U+206F, U+3164, U+FE00–U+FE0F, U+FEFF, U+FFA0, U+FFF0–U+FFFB, U+1BCA0–U+1BCA3, U+1D173–U+1D17A, U+E0000–U+E0FFF. It stays a closed list of code points, so that the agent and every authority agree whatever Unicode version their runtimes ship.

  • The agent strips the listed code points from every metadata value before truncating it, and sends the Linux /etc/machine-id as-is instead of trimming it.
  • The authority library (version 2026.10.28.1) and the mock authority refuse a value holding one with invalid_request.
  • The conformance suite checks every range at both ends and inside, and accepts the code points next to each range.
  • The conformance agent check compares machine_id only when the host provides a value, so an agent on a Linux host whose /etc/machine-id is empty or missing, which omits the key, now passes.

DVLS gets the extended list with the next authority library package; until it takes that package, its conformance run fails the new refusal checks.

Issue: DGW-478
Issue: DGW-479
Security: yes

🤖 Generated with Claude Code

A metadata value can hold bidirectional overrides and zero-width
characters that reorder or hide text where an administrator reads it,
such as the friendly name an authority evaluates from the host name.
Metadata values now exclude a closed list of invisible format
characters, and the agent removes them before sending.
The list is fixed as code points rather than a Unicode general
category, so that the agent and every authority agree on each code
point whatever Unicode version their runtimes ship.

Issue: DGW-478
Issue: DGW-479
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An authority refuses each end of every listed range at enrollment without consuming a token use, and accepts and stores the code points just outside the ranges and unlisted format characters.
The agent removes listed code points from the metadata it sends and sends `unknown` for a required key left empty.
The friendly-name trim case moves to a format character that the list leaves out, since U+200B is now refused.

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each refused enrollment now differs from the accepted control only by the listed code point, and an unknown key holding one is refused too.
The friendly-name trim also covers a format character in the Basic Multilingual Plane, the agent check covers an optional key left empty, and harness tests pin the code point lists and the agent metadata check.

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mock authority now answers `invalid_request` to a metadata value holding
any invisible format character that the contract lists, so that no value can
reorder or hide text where an administrator reads it, such as a friendly name.
The list is matched as fixed code points rather than as Unicode category `Cf`,
so the mock agrees with every party whatever Unicode version it ships.

Issue: DGW-478
Security: yes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent now removes from each metadata value the invisible format
characters that the Agent Identity contract forbids, such as
bidirectional overrides, zero-width characters and tag characters,
along with the control characters it already removed. It therefore
never sends a value an authority refuses, and no reported value can
reorder or hide text where an administrator reads it, such as in a
friendly name evaluated from metadata.

The characters are matched as the contract's fixed list of code points
rather than a Unicode category, so the agent and every authority agree
whatever Unicode version their runtimes ship.

Issue: DGW-478
Security: yes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…adata

`AgentMetadata.Validate` now refuses a value that holds one of the
invisible format characters the contract lists: U+00AD, U+061C, U+180E,
U+200B to U+200F, U+202A to U+202E, U+2060 to U+2064, U+2066 to U+206F,
U+FEFF, U+FFF9 to U+FFFB, U+E0001 and U+E0020 to U+E007F. Such a
character can reorder or hide text where an administrator reads it, as
in a friendly name evaluated from metadata. Enroll, renew and check-in
bodies and the `Hello` of a channel stream therefore answer such a value
with `invalid_request`.

The list is a fixed set of code points rather than the runtime's Format
category, so the agent and every authority agree whatever Unicode
version their runtimes ship, and the visible format marks such as
U+0600 stay accepted. The failure detail names neither the character
nor its code point, since the value comes from the request.

Issue: DGW-479
Security: yes
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478
Issue: DGW-479
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The library now refuses metadata it accepted before, so authorities
need a new package to pick up the rule.

Issue: DGW-479

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ependent

The check compares machine_id with /etc/machine-id on Linux, so the fixture now carries the host's value there, and each refused case asserts the format-character error so another check cannot make it pass.

Issue: DGW-478

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The agent metadata check required `machine_id` to equal `/etc/machine-id` whenever the file existed.
CONTRACT §3.1 makes `machine_id` optional and omits an optional key that is empty after sanitising, so on a host whose file is empty a conforming agent omits it and the check failed.
The check now derives the expected value from the file after the sanitising of §3.1, compares it when there is one, and requires the key absent when the file is missing or leaves nothing.

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…etadata

The invisible format character list in identity metadata missed code
points that render invisibly without being format controls, such as
the Hangul fillers, the combining grapheme joiner and the variation
selectors, so a metadata value could still hide text where an
administrator reads it.
The list now holds every Unicode default-ignorable code point,
reserved ones included so that a character later assigned there is
covered, plus the interlinear annotation characters.
It stays a closed list of code points, so that the agent and every
authority agree on it whatever Unicode version their runtimes ship.

Issue: DGW-478
Issue: DGW-479
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tadata

The suite now checks the extended list of invisible format characters
that no metadata value holds, on both the authority and the agent side.
Each end of every listed range is refused at enrollment without
consuming a token use and in `Hello`, and each code point just outside
a range, plus unlisted format characters, is accepted and stored as
sent.
The accepted control groups those code points two per unknown key, so
that it stays within the limit of 32 keys, and U+13430 replaces
U+1D173, which the list now holds, as an unlisted format character.

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mock refused only part of the invisible format characters, so a
metadata value could still hide or disguise text where an administrator
reads it, for instance with a variation selector, a Hangul filler, the
combining grapheme joiner, or a tag outside U+E0020-U+E007F.

The mock now refuses the full closed list of code points: Unicode
Default_Ignorable_Code_Point plus U+FFF9-U+FFFB, reserved code points
included. The list is fixed code point ranges, so every authority
agrees on it whatever Unicode version its runtime ships.

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…etadata

The agent removed only part of the invisible format characters before
sending metadata, so values could still carry code points that render
as nothing, such as variation selectors, Hangul fillers or the rest of
the tag block, and hide or disguise text where an administrator reads
it, such as a friendly name evaluated from metadata.

The removed set is now Unicode's Default_Ignorable_Code_Point plus the
interlinear annotation controls, reserved code points included, written
as fixed code point ranges so the agent and every authority agree
whatever Unicode version they ship.

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s in metadata

The protocol's list of invisible format characters now covers every
default-ignorable code point, plus U+FFF9 to U+FFFB, so that fillers,
variation selectors and the other characters that render as nothing
cannot hide or disguise text where an administrator reads metadata,
such as a friendly name. The list stays closed and written as code
points, so the agent and every authority agree whatever Unicode version
their runtimes ship. Each range is tested at both ends and just
outside them, including the supplementary-plane ones.

Issue: DGW-479
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The library now refuses metadata it accepted before, so authorities
need a distinct package to pick up the extended list.

Issue: DGW-479
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e range

Issue: DGW-479
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… checks

The metadata check takes the expected machine ID as a parameter, so harness
tests feed fixed values and can fail when the call is removed, and the
fixtures no longer read the build host's file.
The control-character predicate is shared by the check and the expectation,
and the mismatch message names the sent and expected values.

The invisible format character tests send a code point inside each range
that a list kept in pieces would miss, in the authority and agent checks.

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The contract sends `/etc/machine-id` as-is and has the agent remove only control characters and invisible format characters. Trimming the file contents also removed leading and trailing spaces and tabs, which the contract does not allow removing. `sanitize` already drops the trailing newline as a C0 control character and omits the value when nothing is left, so the extra trim is dropped.

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le is unreadable

Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478
Security: yes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CBenoit

Copy link
Copy Markdown
Member Author

Evidence

Range on top of #2166's head 3e51fb6a5; this PR's own diff is 3e51fb6a5..HEAD until #2166 merges and the branch is rebased.

Lines changed (3e51fb6a5..HEAD)

Kind Files + / −
Production crates/agent-identity/src/metadata.rs (list, machine ID; file also holds its unit tests), dotnet/.../Bodies/AgentMetadata.cs, .csproj version 60 / 36 (incl. those inline tests)
Mock authority (test infrastructure, list and unit tests) crates/agent-identity-mock/src/name_eval.rs 25 / 15
Tests conformance agent.rs, client.rs, protocol.rs; AgentMetadataTests.cs 460 / 122
Docs CONTRACT.agent-identity.md (one line, approved wording), COVERAGE.md 6 / 4

Checks (Windows, this worktree's own target directory)

  • The §3 list was checked against Unicode 18.0 DerivedCoreProperties.txt Default_Ignorable_Code_Point; the contract, mock, agent, .NET library and conformance suite hold the same 17 ranges (compared range by range).
  • cargo fmt --check; cargo clippy -p agent-identity -p agent-identity-mock -p agent-identity-conformance --all-targets -- -D warnings: clean.
  • cargo test -p agent-identity -p agent-identity-mock -p agent-identity-conformance: 195, 513 + 10, 59 passed (1 ignored).
  • Conformance through cargo test -p testsuite --test integration_tests agent_identity::conformance: 136 PASS, 0 FAIL, 55 SKIP (Windows software provider keys need an elevated run), 4 N/A, including p_metadata_invisible_format_characters, p_channel_hello_invalid_metadata_rejected and p_friendly_name_white_space_and_truncation. Linux CI covers the file key backend and the /etc/machine-id path.
  • Red runs: against the mock and agent of fix(agent): refuse invisible format characters in metadata #2166, p_metadata_invisible_format_characters, p_channel_hello_invalid_metadata_rejected and (file key backend) a_metadata_invisible_format_characters_removed failed on newly listed code points; the moved white-space case passed.
  • .NET: build.ps1 0 warnings, 0 errors; dotnet test 1882 passed.
  • ci/check-oracle-markers.ps1 -Base origin/master: exit 0 (a later local rerun hit the unauthenticated GitHub API rate limit).

Review

Routine rounds on the implementations and on the suite changes (the suite reviewed blind to the implementations), then a final whole-delta round with an intent-drift audit (all touched statements conform), then a fresh round on the post-final fixes. Nothing blocking remains.

Not taken:

  • No property test over the long allowed runs between ranges: a wrongly added range inside one is unlikely, and the suite's neighbour checks catch the common off-by-one.
  • The accepted neighbour U+DFFFF is a noncharacter; §3's list is closed and noncharacters are valid scalar values, so an authority refusing it deviates from §3. Watch for it in the DVLS run.
  • When the suite cannot read /etc/machine-id for a reason other than its absence, it skips the machine_id comparison without marking the run incomplete: no test is skipped, and a world-readable file that cannot be read is a host anomaly.

Deviations

  • The Linux machine ID read now sends the file as-is (no trim), which the conformance check pins; this goes beyond the two gaps but keeps the agent and suite agreeing with §3.1.

Note

LLM-assisted content (no human feedback).

@CBenoit

Copy link
Copy Markdown
Member Author

UNTESTED: properties

  • crates/agent-identity/src/metadata.rs, Linux machine_id(): the agent sends /etc/machine-id as-is and must not trim it. Trimming and the control-character removal agree on every normal host, so no test sees a reintroduced trim; the conformance suite notices only on a host whose file has surrounding spaces.

Note

LLM-assisted content (no human feedback).

…s-6b80dd

# Conflicts:
#	crates/agent-identity-conformance/COVERAGE.md
#	crates/agent-identity-conformance/src/agent.rs
#	crates/agent-identity-conformance/src/client.rs
#	crates/agent-identity-conformance/src/protocol.rs
#	crates/agent-identity-mock/src/name_eval.rs
#	crates/agent-identity/src/metadata.rs
#	docs/agent-identity/CONTRACT.agent-identity.md
#	dotnet/Devolutions.Agent.Identity.Authority/Devolutions.Agent.Identity.Authority.Tests/Bodies/AgentMetadataTests.cs
#	dotnet/Devolutions.Agent.Identity.Authority/Devolutions.Agent.Identity.Authority/Bodies/AgentMetadata.cs
#	dotnet/Devolutions.Agent.Identity.Authority/Devolutions.Agent.Identity.Authority/Devolutions.Agent.Identity.Authority.csproj
@CBenoit
Benoît Cortier (CBenoit) merged commit c718be6 into master Oct 28, 2026
53 checks passed
@CBenoit
Benoît Cortier (CBenoit) deleted the claude/metadata-gaps-6b80dd branch October 28, 2026 10:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant