Repository navigation
fix(agent): refuse remaining invisible metadata code points - #2170
Merged
Merged
Conversation
A metadata value can hold bidirectional overrides and zero-width characters that reorder or hide text where an administrator reads it, such as the friendly name an authority evaluates from the host name. Metadata values now exclude a closed list of invisible format characters, and the agent removes them before sending. The list is fixed as code points rather than a Unicode general category, so that the agent and every authority agree on each code point whatever Unicode version their runtimes ship. Issue: DGW-478 Issue: DGW-479 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An authority refuses each end of every listed range at enrollment without consuming a token use, and accepts and stores the code points just outside the ranges and unlisted format characters. The agent removes listed code points from the metadata it sends and sends `unknown` for a required key left empty. The friendly-name trim case moves to a format character that the list leaves out, since U+200B is now refused. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each refused enrollment now differs from the accepted control only by the listed code point, and an unknown key holding one is refused too. The friendly-name trim also covers a format character in the Basic Multilingual Plane, the agent check covers an optional key left empty, and harness tests pin the code point lists and the agent metadata check. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mock authority now answers `invalid_request` to a metadata value holding any invisible format character that the contract lists, so that no value can reorder or hide text where an administrator reads it, such as a friendly name. The list is matched as fixed code points rather than as Unicode category `Cf`, so the mock agrees with every party whatever Unicode version it ships. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent now removes from each metadata value the invisible format characters that the Agent Identity contract forbids, such as bidirectional overrides, zero-width characters and tag characters, along with the control characters it already removed. It therefore never sends a value an authority refuses, and no reported value can reorder or hide text where an administrator reads it, such as in a friendly name evaluated from metadata. The characters are matched as the contract's fixed list of code points rather than a Unicode category, so the agent and every authority agree whatever Unicode version their runtimes ship. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…adata `AgentMetadata.Validate` now refuses a value that holds one of the invisible format characters the contract lists: U+00AD, U+061C, U+180E, U+200B to U+200F, U+202A to U+202E, U+2060 to U+2064, U+2066 to U+206F, U+FEFF, U+FFF9 to U+FFFB, U+E0001 and U+E0020 to U+E007F. Such a character can reorder or hide text where an administrator reads it, as in a friendly name evaluated from metadata. Enroll, renew and check-in bodies and the `Hello` of a channel stream therefore answer such a value with `invalid_request`. The list is a fixed set of code points rather than the runtime's Format category, so the agent and every authority agree whatever Unicode version their runtimes ship, and the visible format marks such as U+0600 stay accepted. The failure detail names neither the character nor its code point, since the value comes from the request. Issue: DGW-479 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478 Issue: DGW-479 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The library now refuses metadata it accepted before, so authorities need a new package to pick up the rule. Issue: DGW-479 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ependent The check compares machine_id with /etc/machine-id on Linux, so the fixture now carries the host's value there, and each refused case asserts the format-character error so another check cannot make it pass. Issue: DGW-478 Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The agent metadata check required `machine_id` to equal `/etc/machine-id` whenever the file existed. CONTRACT §3.1 makes `machine_id` optional and omits an optional key that is empty after sanitising, so on a host whose file is empty a conforming agent omits it and the check failed. The check now derives the expected value from the file after the sanitising of §3.1, compares it when there is one, and requires the key absent when the file is missing or leaves nothing. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…etadata The invisible format character list in identity metadata missed code points that render invisibly without being format controls, such as the Hangul fillers, the combining grapheme joiner and the variation selectors, so a metadata value could still hide text where an administrator reads it. The list now holds every Unicode default-ignorable code point, reserved ones included so that a character later assigned there is covered, plus the interlinear annotation characters. It stays a closed list of code points, so that the agent and every authority agree on it whatever Unicode version their runtimes ship. Issue: DGW-478 Issue: DGW-479 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tadata The suite now checks the extended list of invisible format characters that no metadata value holds, on both the authority and the agent side. Each end of every listed range is refused at enrollment without consuming a token use and in `Hello`, and each code point just outside a range, plus unlisted format characters, is accepted and stored as sent. The accepted control groups those code points two per unknown key, so that it stays within the limit of 32 keys, and U+13430 replaces U+1D173, which the list now holds, as an unlisted format character. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mock refused only part of the invisible format characters, so a metadata value could still hide or disguise text where an administrator reads it, for instance with a variation selector, a Hangul filler, the combining grapheme joiner, or a tag outside U+E0020-U+E007F. The mock now refuses the full closed list of code points: Unicode Default_Ignorable_Code_Point plus U+FFF9-U+FFFB, reserved code points included. The list is fixed code point ranges, so every authority agrees on it whatever Unicode version its runtime ships. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…etadata The agent removed only part of the invisible format characters before sending metadata, so values could still carry code points that render as nothing, such as variation selectors, Hangul fillers or the rest of the tag block, and hide or disguise text where an administrator reads it, such as a friendly name evaluated from metadata. The removed set is now Unicode's Default_Ignorable_Code_Point plus the interlinear annotation controls, reserved code points included, written as fixed code point ranges so the agent and every authority agree whatever Unicode version they ship. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s in metadata The protocol's list of invisible format characters now covers every default-ignorable code point, plus U+FFF9 to U+FFFB, so that fillers, variation selectors and the other characters that render as nothing cannot hide or disguise text where an administrator reads metadata, such as a friendly name. The list stays closed and written as code points, so the agent and every authority agree whatever Unicode version their runtimes ship. Each range is tested at both ends and just outside them, including the supplementary-plane ones. Issue: DGW-479 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The library now refuses metadata it accepted before, so authorities need a distinct package to pick up the extended list. Issue: DGW-479 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e range Issue: DGW-479 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… checks The metadata check takes the expected machine ID as a parameter, so harness tests feed fixed values and can fail when the call is removed, and the fixtures no longer read the build host's file. The control-character predicate is shared by the check and the expectation, and the mismatch message names the sent and expected values. The invisible format character tests send a code point inside each range that a list kept in pieces would miss, in the authority and agent checks. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The contract sends `/etc/machine-id` as-is and has the agent remove only control characters and invisible format characters. Trimming the file contents also removed leading and trailing spaces and tabs, which the contract does not allow removing. `sanitize` already drops the trailing newline as a C0 control character and omits the value when nothing is left, so the extra trim is dropped. Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…le is unreadable Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Issue: DGW-478 Security: yes Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
Author
EvidenceRange on top of #2166's head Lines changed (
|
| Kind | Files | + / − |
|---|---|---|
| Production | crates/agent-identity/src/metadata.rs (list, machine ID; file also holds its unit tests), dotnet/.../Bodies/AgentMetadata.cs, .csproj version |
60 / 36 (incl. those inline tests) |
| Mock authority (test infrastructure, list and unit tests) | crates/agent-identity-mock/src/name_eval.rs |
25 / 15 |
| Tests | conformance agent.rs, client.rs, protocol.rs; AgentMetadataTests.cs |
460 / 122 |
| Docs | CONTRACT.agent-identity.md (one line, approved wording), COVERAGE.md |
6 / 4 |
Checks (Windows, this worktree's own target directory)
- The §3 list was checked against Unicode 18.0
DerivedCoreProperties.txtDefault_Ignorable_Code_Point; the contract, mock, agent, .NET library and conformance suite hold the same 17 ranges (compared range by range). cargo fmt --check;cargo clippy -p agent-identity -p agent-identity-mock -p agent-identity-conformance --all-targets -- -D warnings: clean.cargo test -p agent-identity -p agent-identity-mock -p agent-identity-conformance: 195, 513 + 10, 59 passed (1 ignored).- Conformance through
cargo test -p testsuite --test integration_tests agent_identity::conformance: 136 PASS, 0 FAIL, 55 SKIP (Windows software provider keys need an elevated run), 4 N/A, includingp_metadata_invisible_format_characters,p_channel_hello_invalid_metadata_rejectedandp_friendly_name_white_space_and_truncation. Linux CI covers the file key backend and the/etc/machine-idpath. - Red runs: against the mock and agent of fix(agent): refuse invisible format characters in metadata #2166,
p_metadata_invisible_format_characters,p_channel_hello_invalid_metadata_rejectedand (file key backend)a_metadata_invisible_format_characters_removedfailed on newly listed code points; the moved white-space case passed. - .NET:
build.ps10 warnings, 0 errors;dotnet test1882 passed. ci/check-oracle-markers.ps1 -Base origin/master: exit 0 (a later local rerun hit the unauthenticated GitHub API rate limit).
Review
Routine rounds on the implementations and on the suite changes (the suite reviewed blind to the implementations), then a final whole-delta round with an intent-drift audit (all touched statements conform), then a fresh round on the post-final fixes. Nothing blocking remains.
Not taken:
- No property test over the long allowed runs between ranges: a wrongly added range inside one is unlikely, and the suite's neighbour checks catch the common off-by-one.
- The accepted neighbour U+DFFFF is a noncharacter; §3's list is closed and noncharacters are valid scalar values, so an authority refusing it deviates from §3. Watch for it in the DVLS run.
- When the suite cannot read
/etc/machine-idfor a reason other than its absence, it skips themachine_idcomparison without marking the run incomplete: no test is skipped, and a world-readable file that cannot be read is a host anomaly.
Deviations
- The Linux machine ID read now sends the file as-is (no trim), which the conformance check pins; this goes beyond the two gaps but keeps the agent and suite agreeing with §3.1.
Note
LLM-assisted content (no human feedback).
Member
Author
|
…s-6b80dd # Conflicts: # crates/agent-identity-conformance/COVERAGE.md # crates/agent-identity-conformance/src/agent.rs # crates/agent-identity-conformance/src/client.rs # crates/agent-identity-conformance/src/protocol.rs # crates/agent-identity-mock/src/name_eval.rs # crates/agent-identity/src/metadata.rs # docs/agent-identity/CONTRACT.agent-identity.md # dotnet/Devolutions.Agent.Identity.Authority/Devolutions.Agent.Identity.Authority.Tests/Bodies/AgentMetadataTests.cs # dotnet/Devolutions.Agent.Identity.Authority/Devolutions.Agent.Identity.Authority/Bodies/AgentMetadata.cs # dotnet/Devolutions.Agent.Identity.Authority/Devolutions.Agent.Identity.Authority/Devolutions.Agent.Identity.Authority.csproj
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Metadata values exclude invisible format characters so that no value can reorder or hide text where an administrator reads it. The list still let through code points that render invisibly without being format controls, such as the Hangul fillers, the combining grapheme joiner and the variation selectors, so a host name could still hide text.
The Agent Identity contract now lists every Unicode default-ignorable code point, reserved ones included so that a character assigned there later is covered, plus the interlinear annotation characters U+FFF9–U+FFFB: U+00AD, U+034F, U+061C, U+115F–U+1160, U+17B4–U+17B5, U+180B–U+180F, U+200B–U+200F, U+202A–U+202E, U+2060–U+206F, U+3164, U+FE00–U+FE0F, U+FEFF, U+FFA0, U+FFF0–U+FFFB, U+1BCA0–U+1BCA3, U+1D173–U+1D17A, U+E0000–U+E0FFF. It stays a closed list of code points, so that the agent and every authority agree whatever Unicode version their runtimes ship.
/etc/machine-idas-is instead of trimming it.invalid_request.machine_idonly when the host provides a value, so an agent on a Linux host whose/etc/machine-idis empty or missing, which omits the key, now passes.DVLS gets the extended list with the next authority library package; until it takes that package, its conformance run fails the new refusal checks.
Issue: DGW-478
Issue: DGW-479
Security: yes
🤖 Generated with Claude Code