Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
# CI invokes these same targets rather than re-implementing them in YAML, which is
# what prevents a gate silently degrading into a warning. There is no warning tier.

.PHONY: check check-native-catalog check-licensing check-public-claims check-deb-ordering check-reproducible check-sbom check-tests check-python-floor check-packaging check-native-catalog check-licensing check-public-claims check-privacy check-docs-truth check-current-state check-headers check-docs check-scope check-shell check-refs check-paths check-index check-freeze check-vectors check-vectors-negative check-vectors-crossversion check-gate-coverage check-falsifiable help
.PHONY: check check-provider-alignment check-native-catalog check-licensing check-public-claims check-deb-ordering check-reproducible check-sbom check-tests check-python-floor check-packaging check-native-catalog check-licensing check-public-claims check-privacy check-docs-truth check-current-state check-headers check-docs check-scope check-shell check-refs check-paths check-index check-freeze check-vectors check-vectors-negative check-vectors-crossversion check-gate-coverage check-falsifiable help

check: check-scope check-headers check-python-floor check-packaging check-native-catalog check-licensing check-public-claims check-privacy check-docs-truth check-current-state check-shell check-refs check-paths check-index check-freeze check-vectors check-vectors-negative check-vectors-crossversion check-tests check-docs
@echo "make check: all gates passed"
Expand Down Expand Up @@ -89,6 +89,10 @@ check-packaging:
check-native-catalog:
@python3 scripts/ci/check_native_catalog.py

## check-provider-alignment PRIVATE: the public tree vs the provider registry (not in CI)
check-provider-alignment:
@python3 scripts/ci/check_provider_alignment.py

## check-licensing D-84/D-90: MPL covers what we own; unknown licensing is not distributable
check-licensing:
@python3 scripts/ci/check_licensing.py
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,9 +231,9 @@ Three artifacts, three different strengths of claim, and they are not interchang

| Artifact | Claim |
|---|---|
| source tarball | **bit-for-bit reproducible across tested builders** — observed |
| `.deb` | **bit-for-bit reproducible across tested builders** — observed |
| `.rpm` | **package semantics and payload reproducible.** *Not* claimed byte-for-byte reproducible across rpm toolchain versions |
| source tarball | **cross-builder byte reproducibility demonstrated** |
| `.deb` | **cross-builder byte reproducibility demonstrated.** It also rebuilt byte-identically after a source-tree documentation-only change that did not alter its package payload |
| `.rpm` | **payload and package semantics consistent.** Byte reproducibility across rpm 4 / rpm 6 is **not claimed** |

Tested builders: Fedora 44 / btrfs / rpm 6.0.2 and `ubuntu-latest` / ext4 / rpm 4.18.2. `BUILDTIME`
was identical on both, so `SOURCE_DATE_EPOCH` taken from the commit works across toolchains; the
Expand Down
4 changes: 3 additions & 1 deletion REUSE.toml
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,10 @@ SPDX-License-Identifier = "MPL-2.0"

# Package metadata templates: consumed verbatim by dpkg/rpm, which reject unknown fields.
# The built packages state MPL-2.0 in their own metadata and ship the licence text.
# packaging/deb/copyright is itself a DEP-5 licence declaration; an SPDX comment header
# inside it would be redundant and risks breaking the format dpkg parses.
[[annotations]]
path = ["packaging/deb/control.in", "packaging/rpm/isedraf.spec.in"]
path = ["packaging/deb/control.in", "packaging/rpm/isedraf.spec.in", "packaging/deb/copyright"]
precedence = "aggregate"
SPDX-FileCopyrightText = "2026 Antonios Voulvoulis / ITCMS <contact@itcms.gr>"
SPDX-License-Identifier = "MPL-2.0"
4 changes: 3 additions & 1 deletion docs/CURRENT_STATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@ What exists and runs today. Nothing else on this page does.
| `package_deb` | `packaging/deb/control.in` | — |
| `package_lifecycle_verified` | `scripts/compat/package_lifecycle.sh` | — |
| `package_rpm` | `packaging/rpm/isedraf.spec.in` | — |
| `provider_alignment_check` | `scripts/ci/check_provider_alignment.py` | `make check-provider-alignment` |
| `public_licensing_boundary` | `scripts/ci/public_licensing_policy.json` | `make check-licensing` |
| `report_json` | `lib/isedraf/report/render.py` | `isedraf report --json` |
| `report_markdown` | `lib/isedraf/report/render.py` | `isedraf report` |
| `reproducible_build` | `scripts/ci/check_reproducible.sh` | `make check-reproducible` |
Expand Down Expand Up @@ -106,7 +108,7 @@ Not asserted. Each number is counted at generation time.
| | |
|---|---|
| Gates | 18 |
| Falsification injections | 73 |
| Falsification injections | 83 |
| Golden vector cases | 15 |
| Frozen artifacts | 7 |
| Test files | 3 |
Expand Down
5 changes: 5 additions & 0 deletions packaging/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,11 @@ find "$STAGE/usr/lib/isedraf" -name '__pycache__' -prune -exec rm -rf {} + 2>/de
find "$STAGE" -name '*.pyc' -delete 2>/dev/null
install -m 0644 LICENSE "$STAGE/usr/share/doc/isedraf/LICENSE" 2>/dev/null || true
install -m 0644 README.md "$STAGE/usr/share/doc/isedraf/README.md"
# Debian's licence mechanism is /usr/share/doc/<pkg>/copyright in DEP-5 format, and the
# package shipped none: the RPM declared `License: MPL-2.0` in its metadata while the DEB
# said nothing anywhere a tool could read. A licence the packaging format cannot express
# is a licence a package manager cannot report.
install -m 0644 packaging/deb/copyright "$STAGE/usr/share/doc/isedraf/copyright"
install -m 0644 docs/reference/PLATFORM_COMPATIBILITY.md \
"$STAGE/usr/share/doc/isedraf/PLATFORM_COMPATIBILITY.md"
install -m 0644 docs/operator/STORAGE_AND_OUTPUTS.md \
Expand Down
19 changes: 19 additions & 0 deletions packaging/deb/copyright
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
Upstream-Name: isedraf
Upstream-Contact: Antonios Voulvoulis / ITCMS <contact@itcms.gr>
Source: https://github.com/itcmsgr/isedraf

Files: *
Copyright: 2026 Antonios Voulvoulis / ITCMS <contact@itcms.gr>
License: MPL-2.0

License: MPL-2.0
This Source Code Form is subject to the terms of the Mozilla Public License,
v. 2.0. If a copy of the MPL was not distributed with this file, You can
obtain one at https://mozilla.org/MPL/2.0/.
.
The complete licence text is installed alongside this file as
/usr/share/doc/isedraf/LICENSE.
.
No third-party framework content is bundled in this package. The package
contains only material owned by ITCMS and licensed under MPL-2.0.
205 changes: 184 additions & 21 deletions scripts/ci/check_licensing.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
"""usage: check_licensing.py [tree-root]"""
import fnmatch
import json
import os
import pathlib
import re
import subprocess
Expand All @@ -43,6 +44,7 @@
ROOT = pathlib.Path(sys.argv[1]).resolve() if len(sys.argv) > 1 else pathlib.Path(
subprocess.check_output(["git", "rev-parse", "--show-toplevel"], text=True).strip())
REGISTRY = json.loads((ROOT / "scripts" / "ci" / "framework_sources.json").read_text())
POLICY = json.loads((ROOT / "scripts" / "ci" / "public_licensing_policy.json").read_text())
FAIL = []


Expand Down Expand Up @@ -128,32 +130,193 @@ def tracked():

# --- 5. no framework support CLAIMED in public documentation ---------------------------
# A mapping that does not exist is still a claim to a reader.
RESTRICTED = [
(r"\bCIS\s+(?:Controls?|Benchmark)", "CIS"),
(r"\bISO[/ ]?IEC\s*27\d{3}", "ISO/IEC 27000 series"),
(r"\bISO\s*27001\b", "ISO 27001"),
(r"\bSCF\b", "Secure Controls Framework"),
(r"\bUCF\b", "Unified Compliance Framework"),
(r"\bNIS2\b", "NIS2"),
(r"\bDORA\b", "DORA"),
(r"\bPCI[- ]?DSS\b", "PCI DSS"),
]
SUPPORT_CLAIM = re.compile(
r"\b(support(s|ed|ing)?|compliant|compatible|certified|mapped to|coverage of|"
r"aligned (?:to|with)|conforms? to)\b", re.I)
# The policy is the single authority for WHO is restricted and WHAT counts as a claim.
# A provider token alone is not a finding: "do not copy from CIS" and "no CIS mapping"
# name a provider in order to FORBID it, which is the opposite of claiming it. A finding
# needs the token, a claim word, and no negation on the same line.
RESTRICTED = [(v, k) for k, v in POLICY["restricted_providers"].items()
if not k.startswith("$")]
CLAIM = re.compile(POLICY["claim_context"]["positive"], re.I)
NEGATION = re.compile(POLICY["claim_context"]["negation"], re.I)
ALLOWED_CTX = set(POLICY["allowed_context_paths"]["paths"])
ALLOW_MARK = "<!-- licensing:allow-framework-name -->"
public_docs = [r for r in files if r.endswith(".md")
and not r.startswith(("docs/development/", "docs/licensing/"))]
for rel in public_docs:
text = (ROOT / rel).read_text(encoding="utf-8", errors="replace")

# Every text surface, not only markdown and not only README: a claim in CLI help text or
# in a package description reaches a user just as directly as one in a document.
TEXT_EXT = (".md", ".py", ".sh", ".json", ".yml", ".yaml", ".in", ".txt", ".toml")
text_surfaces = [r for r in files
if r.endswith(TEXT_EXT) and r not in ALLOWED_CTX]
for rel in text_surfaces:
try:
text = (ROOT / rel).read_text(encoding="utf-8", errors="replace")
except OSError:
continue
for n, line in enumerate(text.splitlines(), 1):
if ALLOW_MARK in line:
if ALLOW_MARK in line or not CLAIM.search(line) or NEGATION.search(line):
continue
for pattern, label in RESTRICTED:
if re.search(pattern, line) and SUPPORT_CLAIM.search(line):
bad("%s:%d claims support for or alignment with %s. No framework "
"mapping is licensed, reviewed or bundled: %r"
if re.search(pattern, line):
bad("%s:%d CLAIM about %s — no framework mapping is licensed, reviewed "
"or bundled (public_framework_mappings is empty). Remove the claim, "
"or record an authorization in public_licensing_policy.json. Line: %r"
% (rel, n, label, line.strip()[:70]))
break

# --- 5b. the policy must still describe the state it claims to describe ---------------
for key in ("public_framework_mappings", "licensed_framework_packs",
"provider_partnerships", "bundled_third_party_framework_content"):
if POLICY[key]:
bad("public_licensing_policy.json declares %s=%r. Nothing is authorized; if this "
"changed, it changed deliberately and needs an owner decision recorded."
% (key, POLICY[key]))

# --- 5c. restricted document formats never enter the public tree ----------------------
allowed_bin = set(k for k in POLICY["allowed_binary_artifacts"] if not k.startswith("$"))
for rel in files:
if rel in allowed_bin:
continue
if any(rel.lower().endswith(ext) for ext in POLICY["restricted_document_formats"]):
bad("%s is a document/archive format that may carry restricted provider material "
"(standards, control matrices, exports). Unknown binary artifacts fail "
"closed: allowlist it by path in public_licensing_policy.json with a reason, "
"or remove it." % rel)

# --- 5d. nothing reaches the tree through a symlink -----------------------------------
# Restricted content does not have to live here to be published: a symlink or an external
# build input is enough. A path leaving the repository is a licensing boundary failure
# whatever it points at.
for rel in files:
f = ROOT / rel
if f.is_symlink():
target = os.readlink(str(f))
resolved = (f.parent / target).resolve()
try:
resolved.relative_to(ROOT)
except ValueError:
bad("%s is a symlink pointing OUTSIDE the repository (%s). Public artifacts "
"must not be assembled from external paths." % (rel, target))

# --- 5e. MPL-2.0 is actually applied where the packages claim it ----------------------
exp = POLICY["package_license_expectations"]
lic = ROOT / exp["license_file"]
if not lic.exists():
bad("%s is missing" % exp["license_file"])
elif not lic.read_text(encoding="utf-8").lstrip().startswith(exp["license_first_line"]):
bad("%s does not begin with %r — the canonical licence text may have been altered"
% (exp["license_file"], exp["license_first_line"]))

spec = ROOT / "packaging" / "rpm" / "isedraf.spec.in"
if spec.exists():
m = re.search(r"^License:\s*(\S+)", spec.read_text(), re.M)
if not m:
bad("the rpm spec declares no License field")
elif m.group(1) != exp["rpm_license_field"]:
bad("the rpm spec declares License: %s, the policy expects %s"
% (m.group(1), exp["rpm_license_field"]))

# Debian expresses a licence in /usr/share/doc/<pkg>/copyright, not in `control`. The
# package shipped no copyright file at all, so the RPM declared MPL-2.0 in its metadata
# while the DEB stated it nowhere machine-readable.
cpy = ROOT / exp["deb_copyright_file"]
if not cpy.exists():
bad("%s is missing — a .deb states its licence in a DEP-5 copyright file, and "
"without one the package declares no licence anywhere a tool can read"
% exp["deb_copyright_file"])
else:
ctext = cpy.read_text(encoding="utf-8")
if "License: %s" % exp["deb_copyright_license"] not in ctext:
bad("%s does not declare License: %s"
% (exp["deb_copyright_file"], exp["deb_copyright_license"]))
if not ctext.startswith("Format: https://www.debian.org/doc/packaging-manuals/"):
bad("%s is not in DEP-5 machine-readable format" % exp["deb_copyright_file"])

# --- 6. the ARTIFACTS, not the templates that produced them ---------------------------
# A clean source tree is not a clean package. Everything above reads the repository; this
# reads what a user actually receives, which is the only thing a licensing complaint would
# ever be about.
DIST = ROOT / "dist"
if (DIST / "packages").is_dir():
import subprocess as sp
import tarfile
import tempfile
import shutil

def listing(artifact):
"""Paths inside the artifact, without extracting more than necessary."""
name = artifact.name
if name.endswith(".rpm"):
if not shutil.which("rpm"):
return None
return sp.check_output(["rpm", "-qlp", str(artifact)], text=True,
stderr=sp.DEVNULL).split()
tmp = pathlib.Path(tempfile.mkdtemp())
try:
if name.endswith(".deb"):
(tmp / "d.tgz").write_bytes(
sp.check_output(["ar", "p", str(artifact), "data.tar.gz"]))
src = tmp / "d.tgz"
else:
src = artifact
with tarfile.open(src) as tf:
return tf.getnames()
finally:
shutil.rmtree(str(tmp), ignore_errors=True)

checked = 0
for artifact in sorted((DIST / "packages").iterdir()):
if "latest" in artifact.name or not artifact.name.endswith(
(".deb", ".rpm", ".tar.gz")):
continue
names = listing(artifact)
if names is None:
continue
checked += 1
for n in names:
low = n.lower()
if any(low.endswith(ext) for ext in POLICY["restricted_document_formats"]):
bad("%s contains %s — a document/archive format that may carry restricted "
"provider material" % (artifact.name, n))
if "PROVIDERS_LICENSE" in n or "licensed-framework-research" in n:
bad("%s contains private licensing research: %s" % (artifact.name, n))
if artifact.name.endswith(".rpm") and shutil.which("rpm"):
lic = sp.check_output(["rpm", "-qp", "--qf", "%{LICENSE}", str(artifact)],
text=True, stderr=sp.DEVNULL).strip()
if lic != exp["rpm_license_field"]:
bad("%s declares License=%r, the policy expects %r"
% (artifact.name, lic, exp["rpm_license_field"]))
if artifact.name.endswith(".deb"):
if not any(n.endswith("usr/share/doc/isedraf/copyright") for n in names):
bad("%s ships no /usr/share/doc/isedraf/copyright — the package declares "
"no licence anywhere a tool can read it" % artifact.name)
if checked:
print(" OK %d built artifacts: licence declared, no restricted document "
"format, no private research" % checked)

# --- 7. the SBOM says what the project actually knows ----------------------------------
sbom_dir = DIST / "sbom"
if sbom_dir.is_dir():
n_sbom = 0
for s in sorted(sbom_dir.glob("*.spdx.json")):
doc = json.loads(s.read_text())
n_sbom += 1
for pkg in doc.get("packages", []):
if pkg.get("SPDXID") == "SPDXRef-Package-isedraf":
for field in ("licenseConcluded", "licenseDeclared"):
if pkg.get(field) != exp["sbom_declared_license"]:
bad("%s declares %s=%r for the ISEDRAF package; the project knows "
"it is %s and the format can say so"
% (s.name, field, pkg.get(field), exp["sbom_declared_license"]))
elif pkg.get("SPDXID") == "SPDXRef-Package-cpython":
# NOASSERTION is CORRECT here: the interpreter is an external prerequisite
# whose licence this project does not determine. Claiming MPL-2.0 for it
# would be a manufactured conclusion, which is the opposite error.
if pkg.get("licenseDeclared") == exp["sbom_declared_license"]:
bad("%s declares the external Python runtime as %s — this project does "
"not license the interpreter and must not claim to"
% (s.name, exp["sbom_declared_license"]))
if n_sbom:
print(" OK %d SBOM documents: first-party licence declared, external runtime "
"not misattributed" % n_sbom)

if FAIL:
print("=== licensing gate FAILED ===")
Expand Down
8 changes: 7 additions & 1 deletion scripts/ci/check_package_payload.sh
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,14 @@ DEB="$(find "$DIST" -maxdepth 1 -name '*.deb' ! -name '*latest*' | head -1)"
RPM="$(find "$DIST" -maxdepth 1 -name '*.rpm' ! -name '*latest*' | head -1)"
if [ -n "$DEB" ] && [ -n "$RPM" ] && command -v rpm >/dev/null 2>&1; then
D="$(mktemp -d)"
# `copyright` is excluded because it is Debian's licence MECHANISM, not documentation:
# DEP-5 at /usr/share/doc/<pkg>/copyright is where a .deb states its licence, and an
# .rpm states the same thing in its `License:` metadata field instead. Requiring both
# formats to carry both mechanisms would be parity for its own sake. Everything else
# must still match exactly.
ar p "$DEB" data.tar.gz 2>/dev/null | tar tz 2>/dev/null \
| grep 'usr/share/doc/isedraf/.' | sed 's|.*/||' | sort -u > "$D/deb"
| grep 'usr/share/doc/isedraf/.' | sed 's|.*/||' | grep -v '^copyright$' \
| sort -u > "$D/deb"
rpm -qlp "$RPM" 2>/dev/null \
| grep 'usr/share/doc/isedraf/.' | sed 's|.*/||' | sort -u > "$D/rpm"
if cmp -s "$D/deb" "$D/rpm"; then
Expand Down
Loading
Loading