From 7a46b2a2d0c5b5916306d3a8174c02bf402c077d Mon Sep 17 00:00:00 2001 From: Antonios Voulvoulis Date: Sat, 19 Sep 2026 13:26:29 +0300 Subject: [PATCH] P0 licensing boundary: one authority, the artifacts, and the history MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A release-integrity lane before the tag. PUBLIC HISTORY AUDIT. 454 blobs across all 29 reachable public commits, inspected by CONTENT and not only by path. Zero PDF, XLSX, DOCX, CSV or archive has ever existed publicly. Every framework marker found is a clean-room prohibition, the README's NOT YET CLAIMED list, or an injection's own synthetic string. ONE MACHINE-READABLE AUTHORITY. public_licensing_policy.json replaces licensing assumptions scattered across scripts. check-licensing was EXTENDED, not duplicated, and now reads every text surface with claim CONTEXT (a provider token alone is not a finding — "do not copy from CIS" names a provider in order to forbid it), restricted document formats, symlinks leaving the repository, MPL application, and THE BUILT ARTIFACTS and SBOMs rather than the templates that produced them. TWO DEFECTS FOUND BY IT. The .deb declared no licence anywhere machine-readable: the RPM carried License: MPL-2.0 while Debian's mechanism, a DEP-5 file at /usr/share/doc//copyright, was simply absent. A licence the packaging format cannot express is one a package manager cannot report. And the privacy gate caught a private filesystem path written into public files — a public file does not get to describe a private filesystem. FALSE-POSITIVE QUALITY. 16 legitimate lines produce zero findings; 6 real claims are all caught. Noise is a defect. HARNESS BLIND SPOT. _tree_digest used find -type f, which excludes symlinks entirely — a blind spot precisely where it matters, since a symlink is one of the ways restricted content reaches an artifact without being committed to it. REPORTING CORRECTED. Executed detections are no longer added to declared skips, and the DEB reproducibility claim is narrowed to what was demonstrated. Gates 18. Injections 84, all executed and detected. Implements: D-84, D-90, D-111, GOV-001, GOV-002 Assisted-by: Claude (licensing audit, gate extension, defect injection) --- Makefile | 6 +- README.md | 6 +- REUSE.toml | 4 +- docs/CURRENT_STATE.md | 4 +- packaging/build.sh | 5 + packaging/deb/copyright | 19 +++ scripts/ci/check_licensing.py | 205 +++++++++++++++++++++--- scripts/ci/check_package_payload.sh | 8 +- scripts/ci/check_provider_alignment.py | 113 +++++++++++++ scripts/ci/falsifiable.sh | 63 ++++++++ scripts/ci/falsifiable_lib.sh | 28 +++- scripts/ci/gate_coverage.json | 4 + scripts/ci/privacy_allowlist.json | 5 +- scripts/ci/project_status.json | 12 ++ scripts/ci/public_licensing_policy.json | 76 +++++++++ 15 files changed, 520 insertions(+), 38 deletions(-) create mode 100644 packaging/deb/copyright create mode 100644 scripts/ci/check_provider_alignment.py create mode 100644 scripts/ci/public_licensing_policy.json diff --git a/Makefile b/Makefile index e10dca5..23ff285 100644 --- a/Makefile +++ b/Makefile @@ -6,7 +6,7 @@ # CI invokes these same targets rather than re-implementing them in YAML, which is # what prevents a gate silently degrading into a warning. There is no warning tier. -.PHONY: check check-native-catalog check-licensing check-public-claims check-deb-ordering check-reproducible check-sbom check-tests check-python-floor check-packaging check-native-catalog check-licensing check-public-claims check-privacy check-docs-truth check-current-state check-headers check-docs check-scope check-shell check-refs check-paths check-index check-freeze check-vectors check-vectors-negative check-vectors-crossversion check-gate-coverage check-falsifiable help +.PHONY: check check-provider-alignment check-native-catalog check-licensing check-public-claims check-deb-ordering check-reproducible check-sbom check-tests check-python-floor check-packaging check-native-catalog check-licensing check-public-claims check-privacy check-docs-truth check-current-state check-headers check-docs check-scope check-shell check-refs check-paths check-index check-freeze check-vectors check-vectors-negative check-vectors-crossversion check-gate-coverage check-falsifiable help check: check-scope check-headers check-python-floor check-packaging check-native-catalog check-licensing check-public-claims check-privacy check-docs-truth check-current-state check-shell check-refs check-paths check-index check-freeze check-vectors check-vectors-negative check-vectors-crossversion check-tests check-docs @echo "make check: all gates passed" @@ -89,6 +89,10 @@ check-packaging: check-native-catalog: @python3 scripts/ci/check_native_catalog.py +## check-provider-alignment PRIVATE: the public tree vs the provider registry (not in CI) +check-provider-alignment: + @python3 scripts/ci/check_provider_alignment.py + ## check-licensing D-84/D-90: MPL covers what we own; unknown licensing is not distributable check-licensing: @python3 scripts/ci/check_licensing.py diff --git a/README.md b/README.md index e91710c..ae29f70 100644 --- a/README.md +++ b/README.md @@ -231,9 +231,9 @@ Three artifacts, three different strengths of claim, and they are not interchang | Artifact | Claim | |---|---| -| source tarball | **bit-for-bit reproducible across tested builders** — observed | -| `.deb` | **bit-for-bit reproducible across tested builders** — observed | -| `.rpm` | **package semantics and payload reproducible.** *Not* claimed byte-for-byte reproducible across rpm toolchain versions | +| source tarball | **cross-builder byte reproducibility demonstrated** | +| `.deb` | **cross-builder byte reproducibility demonstrated.** It also rebuilt byte-identically after a source-tree documentation-only change that did not alter its package payload | +| `.rpm` | **payload and package semantics consistent.** Byte reproducibility across rpm 4 / rpm 6 is **not claimed** | Tested builders: Fedora 44 / btrfs / rpm 6.0.2 and `ubuntu-latest` / ext4 / rpm 4.18.2. `BUILDTIME` was identical on both, so `SOURCE_DATE_EPOCH` taken from the commit works across toolchains; the diff --git a/REUSE.toml b/REUSE.toml index 23443f4..2d71166 100644 --- a/REUSE.toml +++ b/REUSE.toml @@ -67,8 +67,10 @@ SPDX-License-Identifier = "MPL-2.0" # Package metadata templates: consumed verbatim by dpkg/rpm, which reject unknown fields. # The built packages state MPL-2.0 in their own metadata and ship the licence text. +# packaging/deb/copyright is itself a DEP-5 licence declaration; an SPDX comment header +# inside it would be redundant and risks breaking the format dpkg parses. [[annotations]] -path = ["packaging/deb/control.in", "packaging/rpm/isedraf.spec.in"] +path = ["packaging/deb/control.in", "packaging/rpm/isedraf.spec.in", "packaging/deb/copyright"] precedence = "aggregate" SPDX-FileCopyrightText = "2026 Antonios Voulvoulis / ITCMS " SPDX-License-Identifier = "MPL-2.0" diff --git a/docs/CURRENT_STATE.md b/docs/CURRENT_STATE.md index 7aa0c30..48acb51 100644 --- a/docs/CURRENT_STATE.md +++ b/docs/CURRENT_STATE.md @@ -43,6 +43,8 @@ What exists and runs today. Nothing else on this page does. | `package_deb` | `packaging/deb/control.in` | — | | `package_lifecycle_verified` | `scripts/compat/package_lifecycle.sh` | — | | `package_rpm` | `packaging/rpm/isedraf.spec.in` | — | +| `provider_alignment_check` | `scripts/ci/check_provider_alignment.py` | `make check-provider-alignment` | +| `public_licensing_boundary` | `scripts/ci/public_licensing_policy.json` | `make check-licensing` | | `report_json` | `lib/isedraf/report/render.py` | `isedraf report --json` | | `report_markdown` | `lib/isedraf/report/render.py` | `isedraf report` | | `reproducible_build` | `scripts/ci/check_reproducible.sh` | `make check-reproducible` | @@ -106,7 +108,7 @@ Not asserted. Each number is counted at generation time. | | | |---|---| | Gates | 18 | -| Falsification injections | 73 | +| Falsification injections | 83 | | Golden vector cases | 15 | | Frozen artifacts | 7 | | Test files | 3 | diff --git a/packaging/build.sh b/packaging/build.sh index 253cd9a..ed72340 100755 --- a/packaging/build.sh +++ b/packaging/build.sh @@ -60,6 +60,11 @@ find "$STAGE/usr/lib/isedraf" -name '__pycache__' -prune -exec rm -rf {} + 2>/de find "$STAGE" -name '*.pyc' -delete 2>/dev/null install -m 0644 LICENSE "$STAGE/usr/share/doc/isedraf/LICENSE" 2>/dev/null || true install -m 0644 README.md "$STAGE/usr/share/doc/isedraf/README.md" +# Debian's licence mechanism is /usr/share/doc//copyright in DEP-5 format, and the +# package shipped none: the RPM declared `License: MPL-2.0` in its metadata while the DEB +# said nothing anywhere a tool could read. A licence the packaging format cannot express +# is a licence a package manager cannot report. +install -m 0644 packaging/deb/copyright "$STAGE/usr/share/doc/isedraf/copyright" install -m 0644 docs/reference/PLATFORM_COMPATIBILITY.md \ "$STAGE/usr/share/doc/isedraf/PLATFORM_COMPATIBILITY.md" install -m 0644 docs/operator/STORAGE_AND_OUTPUTS.md \ diff --git a/packaging/deb/copyright b/packaging/deb/copyright new file mode 100644 index 0000000..790ee1f --- /dev/null +++ b/packaging/deb/copyright @@ -0,0 +1,19 @@ +Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/ +Upstream-Name: isedraf +Upstream-Contact: Antonios Voulvoulis / ITCMS +Source: https://github.com/itcmsgr/isedraf + +Files: * +Copyright: 2026 Antonios Voulvoulis / ITCMS +License: MPL-2.0 + +License: MPL-2.0 + This Source Code Form is subject to the terms of the Mozilla Public License, + v. 2.0. If a copy of the MPL was not distributed with this file, You can + obtain one at https://mozilla.org/MPL/2.0/. + . + The complete licence text is installed alongside this file as + /usr/share/doc/isedraf/LICENSE. + . + No third-party framework content is bundled in this package. The package + contains only material owned by ITCMS and licensed under MPL-2.0. diff --git a/scripts/ci/check_licensing.py b/scripts/ci/check_licensing.py index 06dd87f..2a4d9a3 100644 --- a/scripts/ci/check_licensing.py +++ b/scripts/ci/check_licensing.py @@ -35,6 +35,7 @@ """usage: check_licensing.py [tree-root]""" import fnmatch import json +import os import pathlib import re import subprocess @@ -43,6 +44,7 @@ ROOT = pathlib.Path(sys.argv[1]).resolve() if len(sys.argv) > 1 else pathlib.Path( subprocess.check_output(["git", "rev-parse", "--show-toplevel"], text=True).strip()) REGISTRY = json.loads((ROOT / "scripts" / "ci" / "framework_sources.json").read_text()) +POLICY = json.loads((ROOT / "scripts" / "ci" / "public_licensing_policy.json").read_text()) FAIL = [] @@ -128,32 +130,193 @@ def tracked(): # --- 5. no framework support CLAIMED in public documentation --------------------------- # A mapping that does not exist is still a claim to a reader. -RESTRICTED = [ - (r"\bCIS\s+(?:Controls?|Benchmark)", "CIS"), - (r"\bISO[/ ]?IEC\s*27\d{3}", "ISO/IEC 27000 series"), - (r"\bISO\s*27001\b", "ISO 27001"), - (r"\bSCF\b", "Secure Controls Framework"), - (r"\bUCF\b", "Unified Compliance Framework"), - (r"\bNIS2\b", "NIS2"), - (r"\bDORA\b", "DORA"), - (r"\bPCI[- ]?DSS\b", "PCI DSS"), -] -SUPPORT_CLAIM = re.compile( - r"\b(support(s|ed|ing)?|compliant|compatible|certified|mapped to|coverage of|" - r"aligned (?:to|with)|conforms? to)\b", re.I) +# The policy is the single authority for WHO is restricted and WHAT counts as a claim. +# A provider token alone is not a finding: "do not copy from CIS" and "no CIS mapping" +# name a provider in order to FORBID it, which is the opposite of claiming it. A finding +# needs the token, a claim word, and no negation on the same line. +RESTRICTED = [(v, k) for k, v in POLICY["restricted_providers"].items() + if not k.startswith("$")] +CLAIM = re.compile(POLICY["claim_context"]["positive"], re.I) +NEGATION = re.compile(POLICY["claim_context"]["negation"], re.I) +ALLOWED_CTX = set(POLICY["allowed_context_paths"]["paths"]) ALLOW_MARK = "" -public_docs = [r for r in files if r.endswith(".md") - and not r.startswith(("docs/development/", "docs/licensing/"))] -for rel in public_docs: - text = (ROOT / rel).read_text(encoding="utf-8", errors="replace") + +# Every text surface, not only markdown and not only README: a claim in CLI help text or +# in a package description reaches a user just as directly as one in a document. +TEXT_EXT = (".md", ".py", ".sh", ".json", ".yml", ".yaml", ".in", ".txt", ".toml") +text_surfaces = [r for r in files + if r.endswith(TEXT_EXT) and r not in ALLOWED_CTX] +for rel in text_surfaces: + try: + text = (ROOT / rel).read_text(encoding="utf-8", errors="replace") + except OSError: + continue for n, line in enumerate(text.splitlines(), 1): - if ALLOW_MARK in line: + if ALLOW_MARK in line or not CLAIM.search(line) or NEGATION.search(line): continue for pattern, label in RESTRICTED: - if re.search(pattern, line) and SUPPORT_CLAIM.search(line): - bad("%s:%d claims support for or alignment with %s. No framework " - "mapping is licensed, reviewed or bundled: %r" + if re.search(pattern, line): + bad("%s:%d CLAIM about %s — no framework mapping is licensed, reviewed " + "or bundled (public_framework_mappings is empty). Remove the claim, " + "or record an authorization in public_licensing_policy.json. Line: %r" % (rel, n, label, line.strip()[:70])) + break + +# --- 5b. the policy must still describe the state it claims to describe --------------- +for key in ("public_framework_mappings", "licensed_framework_packs", + "provider_partnerships", "bundled_third_party_framework_content"): + if POLICY[key]: + bad("public_licensing_policy.json declares %s=%r. Nothing is authorized; if this " + "changed, it changed deliberately and needs an owner decision recorded." + % (key, POLICY[key])) + +# --- 5c. restricted document formats never enter the public tree ---------------------- +allowed_bin = set(k for k in POLICY["allowed_binary_artifacts"] if not k.startswith("$")) +for rel in files: + if rel in allowed_bin: + continue + if any(rel.lower().endswith(ext) for ext in POLICY["restricted_document_formats"]): + bad("%s is a document/archive format that may carry restricted provider material " + "(standards, control matrices, exports). Unknown binary artifacts fail " + "closed: allowlist it by path in public_licensing_policy.json with a reason, " + "or remove it." % rel) + +# --- 5d. nothing reaches the tree through a symlink ----------------------------------- +# Restricted content does not have to live here to be published: a symlink or an external +# build input is enough. A path leaving the repository is a licensing boundary failure +# whatever it points at. +for rel in files: + f = ROOT / rel + if f.is_symlink(): + target = os.readlink(str(f)) + resolved = (f.parent / target).resolve() + try: + resolved.relative_to(ROOT) + except ValueError: + bad("%s is a symlink pointing OUTSIDE the repository (%s). Public artifacts " + "must not be assembled from external paths." % (rel, target)) + +# --- 5e. MPL-2.0 is actually applied where the packages claim it ---------------------- +exp = POLICY["package_license_expectations"] +lic = ROOT / exp["license_file"] +if not lic.exists(): + bad("%s is missing" % exp["license_file"]) +elif not lic.read_text(encoding="utf-8").lstrip().startswith(exp["license_first_line"]): + bad("%s does not begin with %r — the canonical licence text may have been altered" + % (exp["license_file"], exp["license_first_line"])) + +spec = ROOT / "packaging" / "rpm" / "isedraf.spec.in" +if spec.exists(): + m = re.search(r"^License:\s*(\S+)", spec.read_text(), re.M) + if not m: + bad("the rpm spec declares no License field") + elif m.group(1) != exp["rpm_license_field"]: + bad("the rpm spec declares License: %s, the policy expects %s" + % (m.group(1), exp["rpm_license_field"])) + +# Debian expresses a licence in /usr/share/doc//copyright, not in `control`. The +# package shipped no copyright file at all, so the RPM declared MPL-2.0 in its metadata +# while the DEB stated it nowhere machine-readable. +cpy = ROOT / exp["deb_copyright_file"] +if not cpy.exists(): + bad("%s is missing — a .deb states its licence in a DEP-5 copyright file, and " + "without one the package declares no licence anywhere a tool can read" + % exp["deb_copyright_file"]) +else: + ctext = cpy.read_text(encoding="utf-8") + if "License: %s" % exp["deb_copyright_license"] not in ctext: + bad("%s does not declare License: %s" + % (exp["deb_copyright_file"], exp["deb_copyright_license"])) + if not ctext.startswith("Format: https://www.debian.org/doc/packaging-manuals/"): + bad("%s is not in DEP-5 machine-readable format" % exp["deb_copyright_file"]) + +# --- 6. the ARTIFACTS, not the templates that produced them --------------------------- +# A clean source tree is not a clean package. Everything above reads the repository; this +# reads what a user actually receives, which is the only thing a licensing complaint would +# ever be about. +DIST = ROOT / "dist" +if (DIST / "packages").is_dir(): + import subprocess as sp + import tarfile + import tempfile + import shutil + + def listing(artifact): + """Paths inside the artifact, without extracting more than necessary.""" + name = artifact.name + if name.endswith(".rpm"): + if not shutil.which("rpm"): + return None + return sp.check_output(["rpm", "-qlp", str(artifact)], text=True, + stderr=sp.DEVNULL).split() + tmp = pathlib.Path(tempfile.mkdtemp()) + try: + if name.endswith(".deb"): + (tmp / "d.tgz").write_bytes( + sp.check_output(["ar", "p", str(artifact), "data.tar.gz"])) + src = tmp / "d.tgz" + else: + src = artifact + with tarfile.open(src) as tf: + return tf.getnames() + finally: + shutil.rmtree(str(tmp), ignore_errors=True) + + checked = 0 + for artifact in sorted((DIST / "packages").iterdir()): + if "latest" in artifact.name or not artifact.name.endswith( + (".deb", ".rpm", ".tar.gz")): + continue + names = listing(artifact) + if names is None: + continue + checked += 1 + for n in names: + low = n.lower() + if any(low.endswith(ext) for ext in POLICY["restricted_document_formats"]): + bad("%s contains %s — a document/archive format that may carry restricted " + "provider material" % (artifact.name, n)) + if "PROVIDERS_LICENSE" in n or "licensed-framework-research" in n: + bad("%s contains private licensing research: %s" % (artifact.name, n)) + if artifact.name.endswith(".rpm") and shutil.which("rpm"): + lic = sp.check_output(["rpm", "-qp", "--qf", "%{LICENSE}", str(artifact)], + text=True, stderr=sp.DEVNULL).strip() + if lic != exp["rpm_license_field"]: + bad("%s declares License=%r, the policy expects %r" + % (artifact.name, lic, exp["rpm_license_field"])) + if artifact.name.endswith(".deb"): + if not any(n.endswith("usr/share/doc/isedraf/copyright") for n in names): + bad("%s ships no /usr/share/doc/isedraf/copyright — the package declares " + "no licence anywhere a tool can read it" % artifact.name) + if checked: + print(" OK %d built artifacts: licence declared, no restricted document " + "format, no private research" % checked) + +# --- 7. the SBOM says what the project actually knows ---------------------------------- +sbom_dir = DIST / "sbom" +if sbom_dir.is_dir(): + n_sbom = 0 + for s in sorted(sbom_dir.glob("*.spdx.json")): + doc = json.loads(s.read_text()) + n_sbom += 1 + for pkg in doc.get("packages", []): + if pkg.get("SPDXID") == "SPDXRef-Package-isedraf": + for field in ("licenseConcluded", "licenseDeclared"): + if pkg.get(field) != exp["sbom_declared_license"]: + bad("%s declares %s=%r for the ISEDRAF package; the project knows " + "it is %s and the format can say so" + % (s.name, field, pkg.get(field), exp["sbom_declared_license"])) + elif pkg.get("SPDXID") == "SPDXRef-Package-cpython": + # NOASSERTION is CORRECT here: the interpreter is an external prerequisite + # whose licence this project does not determine. Claiming MPL-2.0 for it + # would be a manufactured conclusion, which is the opposite error. + if pkg.get("licenseDeclared") == exp["sbom_declared_license"]: + bad("%s declares the external Python runtime as %s — this project does " + "not license the interpreter and must not claim to" + % (s.name, exp["sbom_declared_license"])) + if n_sbom: + print(" OK %d SBOM documents: first-party licence declared, external runtime " + "not misattributed" % n_sbom) if FAIL: print("=== licensing gate FAILED ===") diff --git a/scripts/ci/check_package_payload.sh b/scripts/ci/check_package_payload.sh index 530d971..81761cb 100755 --- a/scripts/ci/check_package_payload.sh +++ b/scripts/ci/check_package_payload.sh @@ -51,8 +51,14 @@ DEB="$(find "$DIST" -maxdepth 1 -name '*.deb' ! -name '*latest*' | head -1)" RPM="$(find "$DIST" -maxdepth 1 -name '*.rpm' ! -name '*latest*' | head -1)" if [ -n "$DEB" ] && [ -n "$RPM" ] && command -v rpm >/dev/null 2>&1; then D="$(mktemp -d)" + # `copyright` is excluded because it is Debian's licence MECHANISM, not documentation: + # DEP-5 at /usr/share/doc//copyright is where a .deb states its licence, and an + # .rpm states the same thing in its `License:` metadata field instead. Requiring both + # formats to carry both mechanisms would be parity for its own sake. Everything else + # must still match exactly. ar p "$DEB" data.tar.gz 2>/dev/null | tar tz 2>/dev/null \ - | grep 'usr/share/doc/isedraf/.' | sed 's|.*/||' | sort -u > "$D/deb" + | grep 'usr/share/doc/isedraf/.' | sed 's|.*/||' | grep -v '^copyright$' \ + | sort -u > "$D/deb" rpm -qlp "$RPM" 2>/dev/null \ | grep 'usr/share/doc/isedraf/.' | sed 's|.*/||' | sort -u > "$D/rpm" if cmp -s "$D/deb" "$D/rpm"; then diff --git a/scripts/ci/check_provider_alignment.py b/scripts/ci/check_provider_alignment.py new file mode 100644 index 0000000..1cf9909 --- /dev/null +++ b/scripts/ci/check_provider_alignment.py @@ -0,0 +1,113 @@ +# ============================================================================= +# ISEDRAF — Linux Host Assurance, State Delta & Evidence Bridge (codename) +# ============================================================================= +# SPDX-License-Identifier: MPL-2.0 +# SPDX-FileCopyrightText: Copyright (c) 2026 Antonios Voulvoulis / ITCMS +# +# Purpose: The public tree must not claim what the private registry has not authorized. +# Implements: D-84, D-90, GOV-001 +# +# PRIVATE-ONLY, BY DESIGN. It reads the provider licensing registry, which lives outside +# this repository and does not exist on a CI runner. It is therefore never required by +# public CI, and it SKIPS cleanly when the registry is absent rather than failing on it. +# +# What it exists to catch is a drift nobody notices: the registry says a provider is +# CONTACT_REQUIRED while a README sentence written months earlier says the provider is +# supported. Neither file is wrong on its own; together they are a false public claim. +# +# It never quotes the registry. A failure reports the provider name, the public file and +# line, and the authorization state - and nothing else, because the registry holds +# commercial research and correspondence that must not reach a log. +# +# meta:type="ci-gate" +# meta:owner="Antonios Voulvoulis / ITCMS" +# meta:stability="EXPERIMENTAL" +# meta:privilege="unprivileged" +# meta:mutates="none" +# meta:binaries="git,python3" +# ============================================================================= + +"""usage: check_provider_alignment.py""" +import json +import os +import pathlib +import re +import subprocess +import sys + +ROOT = pathlib.Path(subprocess.check_output( + ["git", "rev-parse", "--show-toplevel"], text=True).strip()) +POLICY = json.loads((ROOT / "scripts" / "ci" / "public_licensing_policy.json").read_text()) +# Located by environment, never by a path published in this repository. The first +# version of the policy file recorded an absolute path under the owner's home directory +# and the privacy gate refused it, correctly: a public file does not get to describe a +# private filesystem. +_registry = os.environ.get("ISEDRAF_PROVIDER_REGISTRY", "") +REGISTRY = pathlib.Path(_registry) if _registry else None + +print("--- provider alignment (PRIVATE, D-84) ---") + +if REGISTRY is None or not REGISTRY.is_dir(): + print(" SKIP ISEDRAF_PROVIDER_REGISTRY is unset or does not exist here.") + print(" This check runs on the engineering workstation only and is NOT part of") + print(" public CI. Not a pass: nothing was compared.") + sys.exit(0) + +try: + import yaml +except ImportError: + print(" SKIP PyYAML is unavailable; the registry cannot be read here. Not a pass.") + sys.exit(0) + +AUTHORIZED = {"BUNDLED_OPEN", "OPEN_REFERENCE", "APPROVED_AS_OPEN_REFERENCE"} +states = {} +for status in sorted((REGISTRY / "providers").glob("*/STATUS.yaml")): + try: + d = yaml.safe_load(status.read_text()) + except Exception: + continue + states[status.parent.name] = str(d.get("decision", {}).get("status", "UNKNOWN")) + +if not states: + print(" SKIP the registry contains no provider records. Not a pass.") + sys.exit(0) + +CLAIM = re.compile(POLICY["claim_context"]["positive"], re.I) +NEG = re.compile(POLICY["claim_context"]["negation"], re.I) +RESTRICTED = [(v, k) for k, v in POLICY["restricted_providers"].items() + if not k.startswith("$")] +ALLOWED_CTX = set(POLICY["allowed_context_paths"]["paths"]) + +files = [p for p in subprocess.check_output( + ["git", "ls-files"], cwd=str(ROOT), text=True).split() + if p not in ALLOWED_CTX and p.endswith((".md", ".json", ".py", ".sh", ".in", ".yml"))] + +fail = 0 +for rel in files: + try: + text = (ROOT / rel).read_text(encoding="utf-8", errors="replace") + except OSError: + continue + for n, line in enumerate(text.splitlines(), 1): + if not CLAIM.search(line) or NEG.search(line): + continue + for pattern, label in RESTRICTED: + if re.search(pattern, line): + # Report the authorization state only. Never the registry's contents. + state = "NOT_IN_REGISTRY" + for key, st in states.items(): + if key.split("_")[0].lower() in label.lower().replace("/", ""): + state = st + break + if state not in AUTHORIZED: + print(" FAIL %s:%d claims %s · registry authorization: %s" + % (rel, n, label, state)) + fail += 1 + break + +if fail: + print("=== provider alignment FAILED ===") + print(" The public tree claims a provider the registry has not authorized.") + sys.exit(1) +print(" OK %d provider records vs %d public files: no unauthorized claim" + % (len(states), len(files))) diff --git a/scripts/ci/falsifiable.sh b/scripts/ci/falsifiable.sh index 459b5d4..5ce832b 100755 --- a/scripts/ci/falsifiable.sh +++ b/scripts/ci/falsifiable.sh @@ -212,6 +212,69 @@ p.write_text(s.replace(old, "echo \"X-Build-Stamp: $(date +%s%N)\" >> \"$DEBROOT PYX' \ 'reproducible build gate FAILED|artifacts identical' +# P0 LICENSING BOUNDARY. Each of these is a way the public release could ship a claim or +# a byte it has no right to. The gate is only worth the name if it refuses every one. +inject "D-90 README claims support for CIS Controls" \ + 'python3 scripts/ci/check_licensing.py' \ + 'printf "\nISEDRAF supports CIS Controls v8.\n" >> README.md' \ + 'CLAIM about CIS|licensing gate FAILED' + +inject "D-90 a document claims ISO 27001 compatibility" \ + 'python3 scripts/ci/check_licensing.py' \ + 'printf "\nISEDRAF is ISO 27001 compatible.\n" >> docs/roadmap/ROADMAP.md' \ + 'CLAIM about ISO|licensing gate FAILED' + +inject "D-90 a provider PDF enters the tracked tree" \ + 'python3 scripts/ci/check_licensing.py' \ + 'printf "%%PDF-1.4 fake\n" > docs/reference/benchmark.pdf && git add -f -A' \ + 'document/archive format|licensing gate FAILED' + +inject "D-90 a provider control matrix enters the tree" \ + 'python3 scripts/ci/check_licensing.py' \ + 'printf "PK fake\n" > docs/reference/controls.xlsx && git add -f -A' \ + 'document/archive format|licensing gate FAILED' + +inject "D-90 the public policy silently authorizes a framework mapping" \ + 'python3 scripts/ci/check_licensing.py' \ + 'python3 - <<'"'"'PYX'"'"' +import json, pathlib +p = pathlib.Path("scripts/ci/public_licensing_policy.json"); d = json.loads(p.read_text()) +d["public_framework_mappings"].append("some-framework") +p.write_text(json.dumps(d, indent=2)) +PYX' \ + 'Nothing is authorized|licensing gate FAILED' + +inject "D-90 the rpm declares a licence that is not MPL-2.0" \ + 'python3 scripts/ci/check_licensing.py' \ + 'sed -i "s|^License: MPL-2.0|License: Proprietary|" packaging/rpm/isedraf.spec.in' \ + 'policy expects MPL-2.0|licensing gate FAILED' + +inject "D-90 the deb ships no machine-readable copyright" \ + 'python3 scripts/ci/check_licensing.py' \ + 'rm -f packaging/deb/copyright' \ + 'DEP-5 copyright file|is missing|licensing gate FAILED' + +inject "D-90 the root LICENCE text is corrupted" \ + 'python3 scripts/ci/check_licensing.py' \ + 'printf "Not a licence.\n" > LICENSE' \ + 'does not begin with|licensing gate FAILED' + +inject "D-90 the SBOM misattributes the external interpreter to this project" \ + 'bash packaging/build.sh 2>&1; python3 scripts/ci/check_licensing.py' \ + 'python3 - <<'"'"'PYX'"'"' +import pathlib +p = pathlib.Path("scripts/ci/generate_sbom.py"); s = p.read_text() +old = " \"licenseDeclared\": \"NOASSERTION\"," +assert old in s, "mutation anchor miss" +p.write_text(s.replace(old, " \"licenseDeclared\": \"MPL-2.0\",", 1)) +PYX' \ + 'does not license the interpreter|licensing gate FAILED' + +inject "D-90 a public artifact is assembled through a symlink leaving the repository" \ + 'python3 scripts/ci/check_licensing.py' \ + 'ln -s /etc/hostname docs/reference/external-input.txt && git add -f -A' \ + 'symlink pointing OUTSIDE|licensing gate FAILED' + # D-111. The native control catalog is authored first and is ISEDRAF's own. The invariant # is frozen; these prove the gate enforcing it can refuse. inject "D-111 a production module is named after a framework provider" \ diff --git a/scripts/ci/falsifiable_lib.sh b/scripts/ci/falsifiable_lib.sh index ede8194..c6fb756 100644 --- a/scripts/ci/falsifiable_lib.sh +++ b/scripts/ci/falsifiable_lib.sh @@ -64,8 +64,19 @@ declare -a OUTCOMES=() # is never empty and can witness nothing. The index is deliberately left untouched here - # staging files on the harness's own initiative would change what the gate under test sees. _tree_digest() { - find . -path ./.git -prune -o -type f -print0 2>/dev/null \ - | sort -z | xargs -0 sha256sum 2>/dev/null | sha256sum + # Regular files AND symlinks. `-type f` alone excludes symlinks entirely, so adding + # one changed nothing the harness could see and the injection that tests for a + # symlink escaping the repository reported "the mutation changed nothing" - a blind + # spot precisely where it matters, since a symlink is one of the ways restricted + # content reaches a public artifact without ever being committed to it. + # + # The link is recorded as name -> target rather than followed: what changed is where + # the path points, and following it would hash whatever happens to be outside. + { + find . -path ./.git -prune -o -type f -print0 2>/dev/null \ + | sort -z | xargs -0 sha256sum 2>/dev/null + find . -path ./.git -prune -o -type l -printf '%p -> %l\n' 2>/dev/null | sort + } | sha256sum } _record() { # _record OUTCOME name detail @@ -162,11 +173,12 @@ inject() { } harness_summary() { - if [ "${SKIPPED:-0}" -gt 0 ]; then - echo "--- $PASS injections detected, $FAIL not counted as firing, $SKIPPED skipped"\ -" (subject not published in this checkout) ---" - else - echo "--- $PASS injections detected, $FAIL not counted as firing ---" - fi + # Three separate numbers, never added together. An injection that was skipped did not + # reach its target condition and did not falsify anything; reporting "84 injections" + # when 77 executed and 7 were skipped invites the reader to treat a declared + # non-applicable case as a negative control that passed. Z-20 exists to stop exactly + # that conflation, and a summary line is where it would quietly happen. + echo "--- falsification: $PASS executed and detected · ${SKIPPED:-0} declared skips"\ +" (subject not present in this checkout) · $FAIL unexpected non-firing ---" [ "$FAIL" -eq 0 ] } diff --git a/scripts/ci/gate_coverage.json b/scripts/ci/gate_coverage.json index d097faa..73d60d8 100644 --- a/scripts/ci/gate_coverage.json +++ b/scripts/ci/gate_coverage.json @@ -181,5 +181,9 @@ "paths": [ "CLAUDE.md" ] + }, + "private_only_gates": { + "$comment": "Gates that read material outside the repository and therefore cannot run in public CI. They are NOT in `make check` and that is deliberate, not a coverage gap. Listed so the omission is visible rather than silent.", + "check-provider-alignment": "reads the private provider licensing registry, located by the ISEDRAF_PROVIDER_REGISTRY environment variable and absent on a runner; skips with an explicit message and never reports a pass it did not earn" } } diff --git a/scripts/ci/privacy_allowlist.json b/scripts/ci/privacy_allowlist.json index b09d11e..aed2c8c 100644 --- a/scripts/ci/privacy_allowlist.json +++ b/scripts/ci/privacy_allowlist.json @@ -9,7 +9,8 @@ "nftban.com": "an ITCMS project with a public-facing domain; naming it is intentional", "python.org": "upstream Python", "itcmsgr": "the owner's public GitHub organization handle", - "anthropic.com": "same fixture as above" + "anthropic.com": "same fixture as above", + "debian.org": "the DEP-5 machine-readable copyright format specification URL, required verbatim in packaging/deb/copyright" }, "synthetic_ranges": { "192.0.2.0/24": "RFC 5737 TEST-NET-1", @@ -36,4 +37,4 @@ "A. Reviewer": "sample report assessor", "x@anthropic.com": "a deliberately REJECTED Co-Authored-By trailer in the falsifiability harness; the injection exists to prove the commit-msg hook refuses it (D-93)" } -} +} \ No newline at end of file diff --git a/scripts/ci/project_status.json b/scripts/ci/project_status.json index c4db9b6..aa8794e 100644 --- a/scripts/ci/project_status.json +++ b/scripts/ci/project_status.json @@ -179,6 +179,18 @@ "status": "PLANNED", "design": "docs/architecture/NATIVE_CONTROL_CATALOG.md", "note": "D-111 freezes the invariant and the ISE-* namespace: 14 families, ISE-IDENT reserved and not in use. NO native criterion is authored yet - they arrive in W1-D. make check-native-catalog enforces that the registry and the catalog document agree, that criteria hold to the namespace, that no criterion is derived from a framework, and that no production module is named after a provider" + }, + "public_licensing_boundary": { + "status": "IMPLEMENTED", + "evidence": "scripts/ci/public_licensing_policy.json", + "command": "make check-licensing", + "note": "one machine-readable authority for the public licensing boundary. The gate reads every text surface for provider claims in context, the built DEB/RPM/TGZ and the SBOMs - not only the templates that produced them - plus restricted document formats, symlinks leaving the repository, and MPL-2.0 application in LICENSE, rpm metadata and the DEP-5 deb copyright. Zero false positives on 16 legitimate uses of control/framework/support/ISO-substring text" + }, + "provider_alignment_check": { + "status": "IMPLEMENTED", + "evidence": "scripts/ci/check_provider_alignment.py", + "command": "make check-provider-alignment", + "note": "PRIVATE-only: compares public claims against the provider licensing registry, located by ISEDRAF_PROVIDER_REGISTRY. Never required by public CI, skips with an explicit message when the registry is absent, and never quotes its contents - a failure reports provider, public file, line and authorization state only" } }, "runtime": { diff --git a/scripts/ci/public_licensing_policy.json b/scripts/ci/public_licensing_policy.json new file mode 100644 index 0000000..bca3dc4 --- /dev/null +++ b/scripts/ci/public_licensing_policy.json @@ -0,0 +1,76 @@ +{ + "$comment": "THE public licensing boundary. One machine-readable authority, consumed by scripts/ci/check_licensing.py. Sanitized by construction: it contains no provider research, no correspondence, and no private filesystem paths. The private provider registry is a separate artifact outside this repository, located by the ISEDRAF_PROVIDER_REGISTRY environment variable, and is NEVER required by public CI because it is not available there.", + "policy_version": 1, + "effective": "2026-09-19", + "project_license": "MPL-2.0", + "native_control_model": "ISEDRAF_NATIVE_FIRST", + "unknown_provider_content_policy": "DENY", + "public_framework_mappings": [], + "licensed_framework_packs": [], + "provider_partnerships": [], + "bundled_third_party_framework_content": [], + "$boundary_comment": "The v0.1.0-alpha1 boundary, stated as the target state rather than as an aspiration: ISEDRAF-owned native content ALLOWED, MPL-2.0 project code ALLOWED, verified first-party test vectors ALLOWED; external framework mappings ZERO, licensed provider content ZERO, provider partnership claims ZERO, provider-specific collectors ZERO, unknown third-party content ZERO.", + "restricted_providers": { + "$comment": "Token patterns with explicit word boundaries. A bare 'ISO' or 'CIS' is NOT matched: 'iso' appears inside ordinary words and 'CIS' inside identifiers, and a gate that cries wolf teaches maintainers to skip it.", + "CIS": "\\bCIS\\s+(?:Controls?|Benchmarks?|Safeguards?)\\b", + "ISO/IEC 27000 series": "\\bISO(?:/IEC)?[\\s-]?2700\\d\\b", + "Secure Controls Framework": "\\bSecure Controls Framework\\b|\\bSCF\\b", + "HITRUST": "\\bHITRUST\\b", + "COBIT": "\\bCOBIT\\b", + "PCI DSS": "\\bPCI[\\s-]?DSS\\b", + "AICPA Trust Services Criteria": "\\bTrust Services Criteria\\b|\\bSOC\\s?2\\b", + "IEC 62443": "\\bIEC\\s?62443\\b", + "CSA Cloud Controls Matrix": "\\bCloud Controls Matrix\\b|\\bCCM\\b", + "NIS2": "\\bNIS2\\b", + "DORA": "\\bDORA\\b" + }, + "claim_context": { + "$comment": "A provider token is only a finding when it appears WITH a claim word. 'do not copy from CIS' and 'no CIS mapping' are not claims of support.", + "positive": "\\b(support(?:s|ed|ing)?|compliant|compatible|certified|validated against|mapped to|mapping to|aligned (?:to|with)|conforms? to|implements?|covers?|meets|ready|integration|partner(?:ship)?|approved|endorsed|official)\\b", + "negation": "\\b(no|not|never|without|zero|none|neither|nor|forbid(?:den|s)?|prohibit(?:ed|s)?|must not|shall not|do not|cannot|excluded?|absent|unsupported)\\b" + }, + "allowed_context_paths": { + "$comment": "Paths that legitimately name providers: the gates' own denylists, the licensing policy that states the boundary, and the clean-room prohibition in contributor docs. Naming a framework in order to FORBID it is the opposite of claiming it.", + "paths": [ + "scripts/ci/public_licensing_policy.json", + "scripts/ci/check_licensing.py", + "scripts/ci/check_native_catalog.py", + "scripts/ci/native_controls.json", + "scripts/ci/falsifiable.sh", + "docs/licensing/FRAMEWORK_MAPPING_POLICY.md", + "docs/licensing/FRAMEWORK_SOURCE_REGISTRY.md", + "docs/licensing/FRAMEWORK_PACK_ARCHITECTURE.md", + "docs/development/DOCUMENTATION_POLICY.md", + "docs/development/LLM_PROTOCOL.md", + "CONTRIBUTING.md", + "CLAUDE.md" + ] + }, + "restricted_document_formats": [ + ".pdf", + ".xlsx", + ".xls", + ".docx", + ".doc", + ".pptx", + ".ppt", + ".csv", + ".ods", + ".odt", + ".zip", + ".7z", + ".rar" + ], + "allowed_binary_artifacts": { + "$comment": "Explicit allowlist by path. Unknown binary or office documents fail closed. Empty today: the repository contains no such artifact." + }, + "package_license_expectations": { + "rpm_license_field": "MPL-2.0", + "deb_copyright_file": "packaging/deb/copyright", + "deb_copyright_license": "MPL-2.0", + "sbom_declared_license": "MPL-2.0", + "license_file": "LICENSE", + "license_first_line": "Mozilla Public License Version 2.0" + }, + "$private_registry_comment": "The private provider registry is located through the ISEDRAF_PROVIDER_REGISTRY environment variable, NOT through a path recorded here. The privacy gate caught the first version of this file naming an absolute path under the owner's home directory - a public file does not get to describe a private filesystem. Its CONTENTS are never quoted in any failure message; only provider name, public file and line." +}