Skip to content

feat(policy): add the 2.x policy format and its evaluator - #122

Draft
Benoît Cortier (CBenoit) wants to merge 127 commits into
masterfrom
claude/policy-eval-rust
Draft

Benoît Cortier (CBenoit) wants to merge 127 commits into
masterfrom
claude/policy-eval-rust

Conversation

@CBenoit

@CBenoit Benoît Cortier (CBenoit) commented Oct 28, 2026 •

Copy link
Copy Markdown
Member

Policies move to a 2.x format whose evaluation is specified exactly enough that independent implementations decide every request alike, and whose requests are the 2.0 package requests the broker receives.

  • policies/INTENT.md, INTENT.legacy.md and INTENT.wire.md state the design: fail-closed checks, Deny always winning, grants that add up, how 1.x policies are converted, and the 2.0 request encoding.
  • policies/CONTRACT.policy.md specifies document reading, error reporting, request reading per package manager (each request is read the way the broker builds its command, and a request the policy cannot read safely is denied as unreadable), the vocabulary of decisions, unreadable requests and errors, and evaluation with three-valued matching: an unknown property lets a Deny rule apply and keeps an Allow rule, a trusted source or an Except entry from matching.
  • policies/CONTRACT.wire.md specifies the 2.0 package broker wire contract; the evaluator reads its package request without RequestId.
  • policies/test-data/policy-conformance is a language-neutral conformance suite of 587 document cases and 1642 evaluation cases, written from the contracts alone.
  • policies/rust/now-policy-eval is a new crate, with no runtime dependency and no dependency on the 1.x crates, that reads documents, reads requests and evaluates decisions, and passes the whole suite. evaluate returns a decision for every well-formed request, an unreadable one included, and the error BadRequest, with no decision, for a request that is not well-formed. Property tests check that adding an Allow or Deny rule only widens or narrows what is allowed, and that rule order never matters.
  • xtask/unicode-tables is a standalone generator for the Unicode 16.0.0 tables the contract pins; cargo xtask unicode tables runs it on demand. It has its own workspace and lockfile so that xtask gains no dependency; CI does not build it or check its lockfile.

🤖 Generated with Claude Code

State the design intent of the 2.x policy format, the best-effort
handling of 1.x policies, and the normative contract that the Rust and
C# implementations both follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pin the facts that both implementations need to agree on: the npm
registry-name grammar behind unknown package identity, the Bun source
name and registry URL, the WinGet option grammar, how install locations
are counted, the Unicode data for PowerShell source names, the length
unit of source names, a non-empty package identifier, and where a
reversed version range is reported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reject source names with surrounding whitespace or default-ignorable
code points for every manager, derive elevation from the requested
scope field, state the SemVer pre-release identifier rules, report a
key inside an empty grant as an unknown key, and compare fixed source
names and Vcpkg triplets ignoring ASCII case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Read npm identifiers as the broker builds them (a registry name or an
alias:target pair), read name@version as naming the package and
selecting its version for the other managers, follow WinGet's own
argument parser for short-option clusters, attached values and the --
terminator, restrict PowerShell source names to the characters the
broker passes, and bound every request part.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
State the WinGet value-before-terminator order, Scoop option name
matching, Vcpkg port names, version selection by npm and Bun
specifiers, the install-location segment grammar, and that an unknown
exception never excepts a request from a Deny rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…files

`cargo xtask unicode tables` downloads UnicodeData.txt, PropList.txt and
DerivedCoreProperties.txt of Unicode 16.0.0 and writes the simple
uppercase mapping, White_Space and Default_Ignorable_Code_Point as sorted
pairs and ranges, recording each input URL and SHA-256 in the header.
The policy evaluators need the pinned version rather than the platform's
casing so that the Rust and C# implementations agree.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add the now-policy-eval crate, which the broker uses to decide package
requests as policies/CONTRACT.policy.md specifies.

A hand-written strict JSON reader reports what general parsers hide:
duplicate keys after unescaping, nesting deeper than 64 levels, unpaired
surrogates, the raw spelling of numbers and exact JSON Pointers. The
validator turns a document into a policy or into its validation errors.

Requests keep enumerated parts as strings, so a value outside the
contract's set makes a request unreadable rather than unrepresentable.
Reading a request per manager lives apart from the evaluator, because a
change to how the broker builds commands changes only the reading.

The crate does not depend on the format 1.x crates, so removing format
1.x support leaves it untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ection

Each test or case cites the contract section that decides its expected
value, covering document validation and error precedence, timestamps,
request reading per manager, install locations, missing versions,
applicability with unknown properties, grants and forbids, precedence,
audit mode and the rules and opt-ins a decision names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner loads every case file under
policies/test-data/policy-conformance/{documents,evaluation} as that
directory's README defines, reads document inputs as raw text so their
spelling reaches the reader unchanged, and reports every failing case by
name. It passes while the suite has no cases.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A decision depends only on the policy and the request, so an option the
client sends carries its opt-in even when the broker's command drops it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An option the client sends counts as asked for under every manager, so
a decision does not change when a broker version starts or stops passing
that option to the manager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A defect in one list entry is reported at that entry, and a package name
the patterns match only when case is ignored keeps `Packages` unknown,
as the three-valued result for each name already implies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generated valid policies and requests over a small vocabulary check that
adding an Allow rule never denies an allowed request, that adding a Deny
rule never allows a denied one, and that reordering every list in a
policy changes no decision except the order of the rules it names.

Arbitrary bytes and truncated or mutated documents give either a policy
or a non-empty list of errors. This property stands in for a fuzz
target, which the parser does not have yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each option a request sends carries its opt-in under every manager the
broker builds a command for, whether or not the command passes it on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PSResourceGet's `-Version` accepts NuGet version ranges, so a
PowerShell7 version is concrete only when it is a version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An `Options` field counts as its opt-in under every manager, so a
decision depends only on the policy and the request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The broker refuses a source URL for most managers, but WinGet and Scoop
ignore it, and Cargo and Dotnet accept only their registry's own URL, so
a client that always sends one is not refused by every manager.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds cases for WinGet option values that are `--`, Scoop option names,
Npm and Bun version selection, Bun and Vcpkg names, PowerShell7
versions, location spaces and separators, location patterns without a
drive, any non-empty source name in a selector, and which managers
accept a source URL. Unreadable-request cases now use a trigger that
stays unreadable under every manager reading.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…states

A source URL is unreadable unless the manager accepts it: WinGet and Scoop
ignore it, Cargo accepts the crates.io index, and Dotnet accepts the NuGet
index or a blank URL on installs and updates and ignores it on uninstalls.
Scoop option names end at the first `=` and match ignoring ASCII case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ontract states

An identifier that is not a registry name selects the version when it
contains `#` or a part between `:` characters has an `@` after its first
character. PowerShell7 resolves a version as a range unless it is a
version, so its helper and the open-question comments the contract now
settles are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…on patterns

A `CustomInstallLocation` pattern drops a trailing `\` unless it follows a
`:`, whatever the text before the colon. The drive letter is no longer
uppercased, because patterns match ignoring case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CBenoit

Copy link
Copy Markdown
Member Author

Intent drift audit

Audited against policies/INTENT.md (63 statements), INTENT.legacy.md (11), INTENT.wire.md (16), CONTRACT.policy.md, and the package-request half of CONTRACT.wire.md (§values, request reading in §json, §package-request, §opt-ins). The crate was read file by file, except the generated unicode_tables.rs; the suite was checked through its COVERAGE.*.md files and spot-checked cases.

The crate conforms to every CONTRACT.policy.md section checked, including each per-manager reading bullet, and to the request shape, opt-ins and targeting statements of INTENT.wire.md.

Violation: needs a decision on intent

  • INTENT.md §trusted-sources: "A rule may name a source the list does not trust, to govern uninstalls of packages from that source."
    • The contract reads the source of every Uninstall as unknown, because no uninstall command is limited to packages installed from Source.Name.
    • It also requires a selection naming a source to list Operations without Uninstall, which realises §fail-closed: "Rules that silently never match uninstalls … are rejected".
    • So no rule can select uninstalls by source. A rule whose named sources are all untrusted never affects a decision.
    • Either this intent sentence changes, or the contract gives an Uninstall a known source wherever the uninstall command is limited to one source.

Partials

  1. CONTRACT.wire.md §package-request, INTENT.wire.md §targeting: a malformed request, such as Machine with NonElevated, is a BadRequest.

    • Request::is_well_formed implements and tests every bound, but it is crate-private, and evaluate denies such a request as UnreadableRequest.
    • The broker, which answers BadRequest, can get a public check when it is built.
  2. CONTRACT.policy.md §managers: the source of every Uninstall is unknown. No valid policy can observe this, and the code carries an UNTESTED: comment.

  3. INTENT.md §fail-closed, read broadly: some Allow rules still never match some uninstalls without being rejected:

    • Packages and Architectures under Vcpkg;
    • Scopes on Pip and PowerShell uninstalls.

    The sentence conforms when read, with the sentence before it, as being about version conditions.

  4. CONTRACT.policy.md §install-location: the check reads only the text of the path. This is review-only by nature, and the code carries an UNTESTED: comment.

Not yet realised

  • The C# library and its risk guidance.
  • Editors.
  • The broker: command building, the pre/post refusal, and policy replacement.
  • 1.x conversion and the earlier wire paths.
  • /v2/ routing.
  • Every CONTRACT.wire.md section beyond the package request and its opt-ins.

Agreement between the Rust and C# implementations needs the C# implementation. Whether each manager's reading matches its command needs the broker's command builders.

Not covered by intent

  • A malformed request is denied as UnreadableRequest; neither contract decides how an evaluator treats one.
  • Policy exposes no Metadata.Id, Revision or PublishedAt, which policy replacement will need.
  • A key repeated three times yields two identical DuplicateKey errors; the contract does not say whether that is one error or two.
  • xtask/unicode-tables is a standalone tool depending on sha2, outside the workspace; xtask itself gains no dependency.

Note

LLM-assisted content (no human feedback).

The base64 document bytes and a JSON-escaped non-ASCII letter in a case note read as misspellings.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A decision is Allow or Deny and names what produced it, and an unreadable request gets a Deny decision.
An error has no decision: a policy validation error, or a broker error response such as `BadRequest` for a request that is not well-formed.
Rules govern uninstalls by package rather than by source, because no manager reliably records where an installed package came from.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A percent-encoded character, a fragment, an explicit port or user information lets another text fetch the same manifest, which a source selector naming the URL would not match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eval-rust

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…not well-formed

`evaluate` returns `Result<Decision, BadRequest>`.
A request that the wire contract rejects as `BadRequest` gets no decision, and reading no longer treats it as unreadable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…efine them

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
An install whose manifest URL has a percent-encoded character, a fragment or an explicit port is unreadable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…e-wide when elevated

No package manager reliably records which source an installed package came from, so rules govern uninstalls by package rather than by source, and a selection that tests the source applies to installs and updates only.
On a run as administrator, a source name the user can redefine would point a trusted source at another server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… spellings

An error reports no outcome, and only `NotRunnable` carries a decision; an evaluator's report of a request that is not well-formed is an error of implementation-defined form.
A decision's outcome says whether the policy lets the broker run the request, which still may not run.
The wire contract's frame defects and the policy file's `Unreadable` reason no longer share words with errors and unreadable requests.
A Scoop manifest URL with an empty host, a host ending with `.` or a dot segment also lets another text fetch the same manifest, so it is unreadable too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eval-rust

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…coop manifest URLs

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…sed vocabulary

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Each spelling that lets another text fetch the same manifest is unreadable, beside readable near misses: a `:` in the path, `..` inside the host, and segments such as `.x` and `x..y`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed packages

An update's source is the one the request names, and the broker refuses an update whose recorded source differs or cannot be read, so the trusted-source list governs updates as it governs installs.
A decision covers the packages a request names, and the packages a command touches along the way come with them, because a policy cannot practically name every one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"Unreadable" names only a request that a case of request reading holds for, so the policy file state no longer shares the word.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The source of an Update is its `Source.Name` under every manager; where Scoop or Windows PowerShell updates from the source recorded at install, the broker refuses an Update whose record names another source or cannot be read, and the operation ends `Failed` without running anything.
A decision covers the packages a request names, so the ports vcpkg rebuilds or removes beside the named one no longer make an Update or Uninstall unknown, and its identity and architecture are read as for an Install.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An error is any result that is not a decision, including a policy error inside a successful response, and only `NotRunnable` carries one.
A request is unreadable exactly when a case of request reading holds, and an `AnySource` selector tests only the manager.
A Scoop manifest URL is recognised by its scheme ignoring case, and a numeric host is unreadable too, because another text of the same address can fetch the same manifest.
The uninstall source reason keeps only the command-based cause, which the contract can check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eval-rust

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…g triplets for every operation

An Update the broker must check runs no step and ends `Failed` unless the broker has read every record the command would update from and each names `Source.Name`, so a broker that cannot read records has one conforming result.
A vcpkg architecture comes from `Source.Name` whatever the operation, although an Uninstall's source is unknown.
An error is one of the listed results, the outcome of a wire request is the response its rules give, and the PowerShell 7 exclusion names the argument it rests on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The source of an Update is its `Source.Name` under every manager, and only an Uninstall's source is unknown.
A Vcpkg Update or Uninstall names its port and takes its architecture from the triplet, as an Install does, so `Packages` and `Architectures` are known for them.
The decision documentation follows the contract's vocabulary.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… is not well-formed

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
A host whose last label starts with an ASCII digit, such as an IPv4 address or a decimal or hexadecimal number, has other spellings that fetch the same manifest, so an Install through such a URL is unreadable.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ic Scoop hosts

Updates through Scoop, PowerShell, PowerShell 7 and vcpkg are trusted and selected by the source they name, vcpkg Updates and Uninstalls match their port and triplet architecture, and a Deny rule protects only the port a request names.
List cases that relied on an unknown update source now reach an unknown entry through a Scoop manifest URL that differs only in case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…as well

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eval-rust

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@CBenoit

Copy link
Copy Markdown
Member Author

Intent drift audit: update for ff735ef

  • INTENT.md §trusted-sources now conforms. The intent now says the source of an uninstall is unknown and that rules govern uninstalls by package. The contract's rule that a selection naming a source lists Operations without Uninstall realises §fail-closed, which now covers source as well as version.
  • The source of an update is the source the request names under every manager. Where Scoop or Windows PowerShell updates from the source recorded at install, the broker must check every such record before running anything. If it has not read records that all name the request's source, the operation ends Failed without running a step. That check is broker behaviour: no conformance case can observe it, so it is verified by review only.
  • A decision covers the packages a request names. Vcpkg Updates and Uninstalls now read their port and triplet architecture like an Install's.
  • Partial 1 is resolved. A request that is not well-formed now gets the error BadRequest from evaluate and no decision. A well-formed unreadable request still gets Deny with UnreadableRequest. The conformance runner reports the error as a failed case; it never maps it to a decision.
  • Partials 2 to 4 and "Not yet realised" are unchanged. Still not realised: §broker-limits on resolving source names only through machine-wide configuration on elevated runs, and the recorded-source check. Both belong to the broker.

Note

LLM-assisted content (no human feedback).

…sing versions

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request with a bad install location or without a concrete version gets a Deny decision from a check that runs before any rule, not an error.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eval-rust

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant