diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 083782c..014ad6a 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -163,7 +163,12 @@ Allowed states are `OPEN`, `ACK`, `INVESTIGATING`, `CONTAINED`, `RECOVERING`, `MONITORING`, `RESOLVED`, and `FALSE_POSITIVE`. Transitions are validated so a closed incident cannot silently return to containment. An unresolved incident gets at most three reminders (about +1 hour, +24 hours, and +72 hours); afterward -the durable open state is the reminder. A reviewed `FALSE_POSITIVE` suppresses +the durable open state is the reminder. An incident opened only by evidence the +routing gate sent to the digest (the provenance gate, low confidence, below the +floor) records why in `digest_only` and gets none: no reminder ever turns it +into a notification. It stays open, counted and listed, and becomes an ordinary +notified incident only when evidence that would itself interrupt attaches. A +reviewed `FALSE_POSITIVE` suppresses only the exact correlation key while retaining later occurrences as evidence; a changed content hash gets a new key (unless *acquired tolerance*, below, has earned the right to pre-close it). A `RESOLVED` threat that recurs opens a @@ -363,11 +368,37 @@ quiet rather than silence. verdicts, and a ledger that crossed machines would let one compromised body launder bytes into every other body's known-good set. +9. **Publisher signature** (binary-keyed findings) → **MEDIUM**, asked after + the vouch and the receipt and before `build-output`. `publisher-stable` + was the only rung that read a signature, and only on a re-sign in place, + so a valid Developer ID binary earned nothing on first sight and beaconed + HIGH from `$HOME` on the one sensor whose predicate is an OR. The binary + earns `publisher-signed` when `publisher_sig()` accepts its verdict + (apple / app-store / developer-id on macOS, a `strict: detritus` verdict + included; os-signed / signed-valid on Windows; os-managed on Linux) and + the verdict names a team or authority; the note names that signer and the + control behind it (Apple's notarization and revocation on macOS). Ad-hoc, + broken, unsigned and `signed-other` earn nothing. It reads the stat-cached + verdict the sensor already asked for, so it costs no second probe. Two + limits. The OS vendor's own signature (`apple`, Windows `os-signed`) earns + it only inside `TRUSTED_PREFIXES`, on the resolved path, and is never + carried to a copy: the same signed bytes in `/tmp`, `$HOME` or `%TEMP%` are + the living-off-the-land shape. And a caller whose evidence says the + platform's control refused the bytes withholds it (`publisher_ok=False`): + the hot-dir sensor after Gatekeeper rejects a bundle. + Guards, because grading is where an attacker would want to stand: -- **Grades, never mutes.** A downgraded finding is still created, still in the - report, still accumulates risk and joins correlation chains. Custody writes - no dismissal and cannot feed acquired tolerance. +- **Demotion never suppresses; the routing gate decides what interrupts.** A + downgraded finding is still created, still in the report, still accumulates + risk and joins correlation chains, and custody writes no dismissal and + cannot feed acquired tolerance. What keeps a proven origin quiet is the + ROUTING gate (`_provenance_gate`, asked by `route_findings` before the + notify floor): a finding below CRITICAL whose rung is in the self or + vouched tier, that is not attack-defined and not a decoy/latch/canary trip, + routes to the digest, where it remains visible (it carries + `routed: digest: provenance `, which `incident` and `report` print) + and counts toward risk at its tier weight. Weak rungs only demote. - **Attack-defined content never downgrades.** A conceal imperative stays HIGH even when self-attested — an agent prompt-injected into persisting a hostile instruction attests its own write. Custody grades *churn-shaped structure*, @@ -398,8 +429,8 @@ call sequence: bidirectional: a covered change drops to INFO, an uncovered change is promoted to at least HIGH — under enforcement, the *absence of a record* outranks provenance, which is the entire point of opting in. -4. **The notify floor** (`emit`) routes by the final severity; it never - changes one. +4. **The notify floor** (`emit`) routes by the final severity and, through + the provenance gate, by custody; it never changes a severity. 5. **The incident ratchet** (`_severity_max`) only ever steps UP: once an incident opened HIGH, a later regrade of the same subject cannot quietly lower it — de-escalation is the operator's verdict to give, not custody's. @@ -530,10 +561,13 @@ above. An allowlisted fingerprint still opened and refreshed incidents and drove reminders, because `emit` skipped it while every finding flowed into the incident tier untouched. And one genuine new HIGH marked every incident created that scan as already-notified, so a digest-routed sibling lost the -reminder that was its only path to a human. +reminder that was its only path to a human. (That reminder was itself removed +on 2026-09-23: a digest-only incident is never reminded — see Incident +workflow. The per-finding "notified" mark stands.) `route_findings` is the one place the order is written down: allowlisted → -seen → adopted → low-confidence → below-floor → tolerated/learning → new. +seen → adopted → low-confidence → provenance → below-floor → +tolerated/learning → new. The scan path computes it once with the incident tier's memory and hands the same verdicts to `emit`, to `record_security_state` (which now marks "notified" per finding and closes an allowlisted incident as `allowlisted`, @@ -568,6 +602,32 @@ observations re-rendered as though fresh. 118 of 120 dismissals were The menu-bar plugin counts OPEN INCIDENTS, so that — not findings-per-scan — is the number the operator actually reads, and it is what this tier targets. +### Ground truth: `backtest replay` + +Every earlier fix in this tier was measured by silence ("the queue got +shorter"), which is also what broken detection looks like. `aegis.py backtest +replay [--days N] [--reobserve]` scores the CURRENT code against the answer key +the store already holds: it opens the live store read-only (`mode=ro`, one +snapshot), groups the recorded `observation.finding` events into their scans +(by `observed_at`: no writer sets `scan_id`), and re-runs each batch through +`route_findings` with the live tolerance memory and the learning period OFF, +the scan's own record-and-fold (`_record_finding_events`, shared with +`record_security_state`) and `_apply_correlations` (chains, lineage, risk, +incidents) in an in-memory store. It reports per-category interrupts, lists by +id every incident the operator closed as noise (`FALSE_POSITIVE`, whatever the +resolution) whose evidence would open an interrupt again — `noise re-opened: N +of M` — counts open cases with no noise-labelled evidence (`new interrupts +from corpus`), and routes the findings the 21 assay lanes build through the +same gate (`assay recall: X/21`; predicate-only lanes and the one lane that +writes live state are named, not skipped). `--reobserve` first re-asks the +classifier and the custody ladder about every binary a process, listener, +beacon or outbound finding names that is still on disk, and re-derives its +severity from that sensor's own gate, so a classifier or ladder fix is +scoreable and not only a routing fix. The scan-tail closers are deliberately +not run: they key on what a scan re-asserted, and the event log holds only +what the live fold recorded. The command asserts its counts against the store +before printing them and puts a failed assertion at the top. + ### Identity fixes (what a finding is *about*) | Sensor | Was identified by | Now identified by | Measured | diff --git a/PRECISION-PLAN-2026-09-23.md b/PRECISION-PLAN-2026-09-23.md new file mode 100644 index 0000000..4a27a56 --- /dev/null +++ b/PRECISION-PLAN-2026-09-23.md @@ -0,0 +1,177 @@ +# Precision plan — why Aegis cannot tell the operator from an attacker, and the fix that ends the batches + +Written 2026-09-23 against main at `49e38b3` and the live store in `~/.aegis`. +Roles: Fable plans, scores and verifies; an Opus builder executes one step per worktree. + +## The numbers this plan is built on (live store, last 30 days) + +| measure | value | +|---|---| +| incidents closed as noise (FALSE_POSITIVE) | 233 | +| incidents open, all noise | 5 | +| true positives, lifetime | 0 of 538 | +| closed-noise interrupts that were HIGH with **no provenance rung** | 190 of 200 | +| hand verdicts given by the operator | 72, across 45 classes | +| new incidents opened in a class the operator had **already judged** | 109 | +| HIGH interrupts caused by one codesign misread (Zoom + Zotero "broken") | 36 | +| events on Spotify labelled `unsigned` while codesign says Developer ID | 314 (09-20 to 09-23) | + +Six false-alarm batches since 2026-08-20 each fixed four to eight mechanisms and each was followed by another batch. The +batches were correct and they did not converge, because every one of them worked *downstream* of three structural defects. + +## The three core defects + +**1. Provenance is asked last and can only whisper.** Every sensor fires on an attribute the operator's own work shares +with malware: ad-hoc signature in `$HOME` (every build product), a new listener port (Spotify picks one per launch), a +long-lived connection from a Developer ID binary under `$HOME` (the beacon rule is `suspicious_sig OR risky_location`, +the only sensor with an OR), `eval … $(` in argv (every command Claude Code runs). The custody ladder that could explain +the artifact runs *after* the finding exists, may only demote one step, and never suppresses. Measured result: 190 of +200 closed-noise interrupts carried no rung at all. The ladder exists; the interrupts never meet it. + +**2. Teaching evaporates.** A benign-positive verdict is keyed on `process::`. The operator's artifacts are +reborn under a new sha (every build), a new path (`bin.2.337.0`, a new worktree, a new App Translocation id) and so the +verdict covers nothing the operator meant. 72 verdicts, 109 re-fires in the same class. The two widest classes are the +operator's own LaunchAgents (60 re-fires) and the Zotero fork he builds (27). + +**3. There is no ground truth, so every fix is measured by silence.** The 1,900-test suite is synthetic and has never +found one of these defects; the live queue found all of them. The 233 labelled incidents and 3,747 recorded findings are +an unused corpus. "The queue got shorter" is also what broken detection looks like. + +Beneath these, one parser defect is worth naming on its own because it explains four of the five open incidents: +`_classify_mac` (aegis.py ~2696) turns *any* non-zero `codesign --verify --strict` into `broken`. The message on Zoom and +Zotero today is "resource fork, Finder information, or similar detritus not allowed" — Finder xattrs inside the bundle, +a valid Developer ID chain, an intact seal. A real tamper reads "code or signature have been modified" or "a sealed +resource is missing or invalid". `broken` is in `suspicious_sig`, so a valid signature became HIGH 36 times. + +## Done-condition and verification (fixed before any code) + +**Done** when, on this Mac, all four hold at the tip that ships: + +1. `aegis.py backtest replay --days 30` re-routes the recorded findings of every incident the operator closed as noise + and opens **0** interrupts from them. (Residual must be listed by incident id with a one-line reason each.) +2. The same command runs the 21 assay lanes and every attack fixture through the identical path and **all still + interrupt** — precision may never be bought with recall. +3. The full suite is green (`python3 -m pytest tests/ -q`, background, 10 min) and the simulated-Windows diff against the + merge base adds no failures (`PYTHONPATH=tests SIM_BODY=win python3 -m pytest tests/ -q -p simbody`, background). +4. After `aegis.py install watch` and one live scan cycle, the five open incidents are closed by the mechanism that + explains them (not by hand), and 48 hours of live operation open no new noise incident. + +**Pass cap**: six build-verify passes on Phase 2/3 combined. If the cap is hit, the residual is reported, not hidden. + +Fable runs every verification personally and reads the output; a builder's "tests pass" is a claim, not a result. + +## Phase 0 — the ground-truth harness (S0) + +`aegis.py backtest replay [--days N] [--reobserve]`: build an in-memory store, load the recorded findings (events with +`event_type='observation.finding'`, oldest first), run them through the live pipeline — `route_findings`, +`_apply_correlations`, `_accumulate_risk`, `_signal_decision` with the live tolerance memory — and print, per category: +would-interrupt count, and the ids of noise-labelled incidents that would re-open. With `--reobserve`, for every subject +still on disk, recompute `classify_signature` and the custody rung with the *current* code and patch them into the +finding before routing, so classifier and ladder fixes are scoreable, not only routing fixes. Then run the 21 assay lanes +and report recall. Rule 17: the command asserts its own counts against the store before printing them. + +Baseline is recorded on main before Phase 1 lands. Every later step is a diff against that number. + +## Phase 1 — the four defects that are wrong today (S1–S4, independent, parallel) + +- **S1 codesign parser.** Distinguish the verify failure classes. Detritus/xattr complaints with a valid authority chain + keep the chain's trust (developer-id / apple / app-store). "modified", "sealed resource missing or invalid", "invalid + signature" stay `broken`. Test against the real tool on this body (gate on `sys.platform == "darwin"` and the app + being present), plus a fixture for the CI legs. Memory: a fixture cannot test a parser. +- **S2 behavior wrapper.** The agent harness runs `bash -c "source ; eval ''"`. Strip the harness wrapper + and judge ``. A hostile `` (curl piped to sh, base64 decode+exec, osascript phish) still fires at full + severity; a `while … $(gh pr checks …)` loop does not. Not a blanket trust of agent hosts — a prompt-injected agent + running `curl | bash` is exactly what the sensor is for. +- **S3 chain legs.** `_entities` says an interpreter never joins, but `/bin/bash` was the *primary* entity of the + os-program-update finding, so an Apple OS update joined a harness eval into "persistence followed by execution". A + finding whose custody is in the self or vouched tier cannot be a chain leg; an interpreter cannot be the shared entity + from either side. +- **S4 listener identity.** A listener's identity is the program's content, not `(program, port)`; ports are evidence on + one case. Spotify's per-launch port must not be a new signal each time. Ephemeral-range ports already fold to + `#ephemeral`; the fixed port and the helper binary still minted separate signals that fed risk accumulation. + +## Phase 2 — provenance becomes a gate (S5) + +One decision, consulted before interrupt, for every sensor that names a binary: + +- **Reach**: wire `publisher-stable` so a Developer ID or App Store signature with a valid chain earns it on *first* + sight, not only on re-sign-in-place (the docstring at ~2742 already records it was structurally unreachable). Wire + `_grade_binary` into the emitters that still pass nothing (behavior, agent-surface where the subject is a binary). +- **Gate**: in `route_findings`, a finding below CRITICAL that is not attack-defined and whose custody is in + `_SELF_CUSTODY` or `_VOUCHED_CUSTODY` routes to digest, never interrupt. Weak rungs (`build-output`, `supervised`, + `copy-of-graded`) still demote one step, as today. The risk tier already weights these 0 and 0.25. +- **Invariant kept**: attack-defined evidence keeps its severity under perfect custody. A vouched binary running an + osascript password prompt is still CRITICAL. Origin is not innocence; this gate only removes *attribute-only* + findings on binaries whose origin is proven. + +## Phase 3 — verdicts teach at class width (S6, S7) + +- **S6 class tolerance.** `_signal_decision` already tries three candidates per finding (identity, endpoint class, + producer class). Add the classes the ladder can *verify*: signer team id (`8LAYR367YV` is Zotero wherever it runs), + package receipt, build-output repo, supervisor (vouched parent). A benign-positive on a finding that carries a class + records tolerance for the class. Threshold 1 for cryptographically anchored classes (team id, package receipt), 3 for + content-derived ones (build-output, supervisor). `incident benign-positive` prints what it learned: + `Learned: anything signed by team 8LAYR367YV (Zotero) — wherever it runs`. `families` shows the class it will teach. + Read PR #51 first (tolerance-follows-bytes); absorb or supersede, never duplicate. +- **S7 own LaunchAgents.** Use the harness to run `_custody_persistence` over the 60 re-fires and find why the + self-committed rung never reached a `com.charlie.*` plist whose payload is in the operator's repo. Fix what the + numbers show, not what the docstring says. + +## Phase 4 — the loop stays closed (S8) + +`aegis.py report` and the weekly digest print three lines from the harness: precision on the trailing 30 days, assay +recall, and teaching evaporation (incidents opened in an already-judged class). A rise in the third, or a drop in the +second, opens an incident against `self-protection`. The number the operator sees is the number the loop is judged on. + +## Ownership and mechanics + +- Each step: one builder, own worktree (`AIKIT_AGENT=precision-sN uv run ~/Ai/Universe/tools/aikit/worktree.py enter`), + tests first where the I/O is testable, one PR. Builder reports actual command output, never "should work". +- Fable verifies each PR: harness diff against baseline, suite, simbody, then a live scan on this Mac after install. +- Parked for the human: PRs #49 and #50 (Codex, 2026-09-20) overlap Phase 2/3 in intent; they are not merged or + rebased by this plan. Their fate is the operator's call after Phase 1 lands. + +## Assumptions stated + +- The `events` table holds enough of each finding (`path`, `trust`, `sha`, `custody`, `ancestry`, `argv preview`) to + re-route it. Checked on the five open incidents; S0 asserts it over the corpus and reports any field it lacks. +- The Zoom and Zotero detritus complaint is Finder xattrs, not a modified bundle. Both binaries print a TeamIdentifier + and a full Developer ID chain today. S1 verifies with `codesign --verify` (no `--strict`) as the discriminator. +- Spotify's `unsigned` run was the non-answer-as-verdict defect PR #52 fixed; no `unsigned` event since the 08:09 + restart today. S0's `--reobserve` will confirm rather than assume. + +## Results, 2026-09-24 (tip 4c36153) + +**Done-condition 1 was restated during the work, and why.** "0 judged-noise incidents re-alert on replay" mixed two things: +incidents the current code still alerts on after re-deriving their evidence, and incidents whose evidence cannot be +re-asked (subject gone from disk, recorded before a field existed, a clipped preview). The second group cannot reach 0 by +any code change, so the target became the re-derived count, with the as-recorded count reported beside it. + +| replay, `--days 30 --reobserve` | main | 4c36153 | +|---|---|---| +| judged-noise incidents that would alert again | 135 of 213 | 53 of 213 | +| — re-derived (the target) | not measurable | 3 | +| — as recorded (cannot be re-asked) | not measurable | 50 | +| interrupts | 138 | 52 | +| open cases | 140 | 55 | +| assay: routable lanes interrupting / predicate lanes passing | 9/9 · 11/11 | 9/9 · 11/11 | + +- **Done-condition 1:** 3 remain. #382 (Chrome app-shortcut loader, ad-hoc by design, never judged) and #432/#452 (the + operator's own RNAfold Deck build, newer bytes than the judged ones). Each needs one verdict, which now covers those + exact bytes anywhere. Not met to zero; the gap is named. +- **Done-condition 2:** met. Recall is unchanged, and the directive detector gained a hostile pole of five multi-step + exfiltration probes after a first version silenced them. +- **Done-condition 3:** full suite 2517 passed at 4c36153. The simbody win/mac diff against main runs in CI on the final PR. +- **Done-condition 4:** partly met. A real scan of 4c36153 against a sandboxed copy of the live state closed 5 of 6 open + incidents by the mechanism that explains them and opened none. #526 stays, because its subject is gone. The 48-hour live + window starts at `install watch` after merge. +- **Pass cap:** Phase 2/3 used 4 build-verify passes of the 6. + +**Found on the way, not in the plan:** +- The test suite appended real rows to `~/.aegis/custody.jsonl`. A conftest guard now refuses and counts those writes. One + row a branch-only rung leaked was removed by hand (backup beside the file). +- Two rows in that ledger, dated 2026-09-19, fail MAC verification under the real key. The live code already ignores them. + They were not removed, because they predate this work; the operator decides. +- S5 and S7 together demoted a new LaunchAgent to INFO. Demotion now never goes below LOW. +- `intent hook ` exists, but only Claude Code calls it. Codex and Hermes edits never produce receipts, which is why + some agent-config changes still alert. diff --git a/aegis.py b/aegis.py index d82a8a2..b51e412 100755 --- a/aegis.py +++ b/aegis.py @@ -204,7 +204,7 @@ ALLOWLIST = os.path.join(STATE_DIR, "allowlist.json") RUN_LOG = os.path.join(STATE_DIR, "run.log") EVENT_DB = os.path.join(STATE_DIR, "aegis.db") -BASELINE_SCHEMA_VERSION = 3 +BASELINE_SCHEMA_VERSION = 4 HOSTS_FILE = (os.path.join(os.environ.get("SystemRoot", r"C:\Windows"), "System32", "drivers", "etc", "hosts") if IS_WIN else "/etc/hosts") @@ -2243,7 +2243,11 @@ def _sig_stat(path): # alone carried the verdict; Obsidian's `broken` (#526) came out of the # same scan. Any v2 entry may be one of those, so every one re-probes # once. -_SIGCACHE_LOGIC_VERSION = 3 +# 4 strict-only detritus: a `--strict` refusal of Finder attributes was +# cached as `broken` although the seal verified (_STRICT_DETRITUS_MARKERS). +# Live, 2026-09-23: Zoom and Zotero, both Developer ID. Any v3 `broken` +# may be one of those, so every v3 entry re-probes once. +_SIGCACHE_LOGIC_VERSION = 4 def classify_signature(path): @@ -2294,6 +2298,8 @@ def classify_signature(path): # reading them describes neither version. Either way the answer is still # returned for this call; it is only not remembered. failed = result.pop("probe_failed", False) + if failed and stat_sig is not None: + _SIG_UNANSWERED[path] = stat_sig if (stat_sig is not None and not failed and _sig_stat(path) == stat_sig): _sigcache.pop(path, None) # overwrite any prior entry for this path @@ -2363,6 +2369,13 @@ def _classify_linux(path): # a DEGRADED sensor rather than letting the silence read as "everything signed". _SIG_PROBE_FAILURES = 0 +# {path: stat signature} for every probe that did not answer this scan. A +# non-answer is never cached (see classify_signature), which is right for the +# verdict and would make the custody ladder's publisher rung ask codesign (or +# a cold PowerShell) a SECOND time about a binary its own sensor just failed +# to classify. Read only by _publisher_signer; reset with the count above. +_SIG_UNANSWERED = {} + # Every PowerShell-backed probe in this file is sized against MEASUREMENTS of a # cold powershell.exe, never a guess. A cap at or near the boot cost is not a # timeout, it is a coin flip on whether the interpreter finished booting -- and @@ -2628,6 +2641,27 @@ def _is_apple_os_signing(leaf, authorities): return _APPLE_OS_SIGNING_CA in (authorities or ()) +# `codesign --verify --strict` refuses more than a broken seal. It also refuses +# Finder extended attributes left inside a bundle, with +# +# resource fork, Finder information, or similar detritus not allowed +# +# and the plain verify, which checks the same seal, passes. Every non-zero +# strict exit used to be filed `broken`, which is in suspicious_sig(). Live, +# 2026-09-23: Zoom and Zotero, full Developer ID chains and intact seals, read +# `broken` and raised 36 HIGH interrupts in 30 days. A real tamper reads +# "invalid signature (code or signature have been modified)" or "a sealed +# resource is missing or invalid" and fails without --strict too. +# +# So a strict failure that names detritus is asked once more without --strict, +# and the plain verify decides: it passes -> the chain's trust stands and the +# verdict records `strict: detritus`; it fails with a reason -> `broken`. The +# text gate only limits the second probe to the one class known to be benign; +# it is never the verdict, so detritus added to a modified binary cannot +# launder it. Every other strict failure is `broken` exactly as before. +_STRICT_DETRITUS_MARKERS = ("detritus", "resource fork", "finder information") + + def _classify_mac(path): global _SIG_PROBE_FAILURES out, err, rc = run(["codesign", "-dv", "--verbose=4", path], timeout=12) @@ -2696,7 +2730,22 @@ def _classify_mac(path): result["trust"] = "unknown" result["probe_failed"] = True elif vrc != 0 and "not signed" not in (verr or "").lower(): - result["trust"] = "broken" + if not any(m in (verr or "").lower() + for m in _STRICT_DETRITUS_MARKERS): + result["trust"] = "broken" + else: + # The plain verify's non-answer is the strict one's: a timeout, + # or a failing exit that says nothing, is not a verdict. + _, perr, prc = run(["codesign", "--verify", path], timeout=20) + if (_probe_timed_out(perr, prc) + or (prc != 0 and not (perr or "").strip())): + _SIG_PROBE_FAILURES += 1 + result["trust"] = "unknown" + result["probe_failed"] = True + elif prc == 0: + result["strict"] = "detritus" + else: + result["trust"] = "broken" return result @@ -2955,7 +3004,8 @@ def _epoch(value=None): last_notified_at INTEGER, resolution TEXT, subject_json TEXT, - last_novel_at INTEGER + last_novel_at INTEGER, + digest_only TEXT ); CREATE INDEX IF NOT EXISTS idx_incidents_active ON incidents(status, next_reminder_at); @@ -3015,7 +3065,7 @@ def _epoch(value=None): # Bumped ONLY when _EVENT_SCHEMA_SQL or the ALTER migrations below change. # Stamped into the store's own `PRAGMA user_version`, which is what stops the # schema being re-executed on every connection open (see _event_connection). -_EVENT_SCHEMA_VERSION = 1 +_EVENT_SCHEMA_VERSION = 2 # Two retention buckets, not one. `events` used to keep the newest 50,000 rows # full stop, and _record_health writes ONE row per sensor per scan (~45). At the @@ -3112,6 +3162,10 @@ def _event_connection(): # new is eligible immediately, the correct verdict on it. db.execute("UPDATE incidents SET last_novel_at=created_at " "WHERE last_novel_at IS NULL") + if "digest_only" not in cols: + # NULL for every existing row: an incident opened before the + # mark existed keeps the reminders it was promised. + db.execute("ALTER TABLE incidents ADD COLUMN digest_only TEXT") db.execute("PRAGMA user_version=%d" % _EVENT_SCHEMA_VERSION) db.commit() except sqlite3.DatabaseError as e: @@ -3446,8 +3500,21 @@ def _mark_novelty(db, incident_id, event_ids, now): def _upsert_incident(db, key, title, severity, kind, now, event_ids, - initially_notified=False, subject=None): + initially_notified=False, subject=None, route=None): + """Open or update the incident for `key`; returns its id. + + `route` is the routing verdict (route_findings) for the evidence being + attached, when the caller has one. It carries the one reminder rule: an + incident OPENED by evidence the gate sent to the digest -- the provenance + gate, low confidence, anything below the interrupt tier -- records why in + `digest_only`, is never claimed by a reminder, and so is never turned into + a notification later by the clock. Evidence that would itself interrupt + clears the mark when it attaches, and the incident becomes an ordinary + notified one from that moment. Callers without a routing (the correlation + and risk tiers, legacy callers) pass None and keep the old behaviour.""" subject_json = json.dumps(subject, sort_keys=True) if subject else None + routed = (route or {}).get("route") + digest_why = (route or {}).get("why") if routed == ROUTE_DIGEST else None marks = ",".join("?" for _ in _ACTIVE_INCIDENT_STATES) row = db.execute( "SELECT * FROM incidents WHERE correlation_key=? AND status IN (%s) " @@ -3465,6 +3532,13 @@ def _upsert_incident(db, key, title, severity, kind, now, event_ids, "updated_at=?, subject_json=COALESCE(subject_json,?) " "WHERE id=?", (new_sev, new_status, now, now, subject_json, incident_id)) + if row["digest_only"] and routed == ROUTE_INTERRUPT: + # The one way out: this evidence interrupted on its own merits, + # so the operator has just been told, and the case is an ordinary + # notified incident from here on. + db.execute("UPDATE incidents SET digest_only=NULL, " + "last_notified_at=?, next_reminder_at=? WHERE id=?", + (now, now + _REMINDER_DELAYS[0], incident_id)) _mark_novelty(db, incident_id, event_ids, now) else: # FALSE_POSITIVE is a reviewed verdict on the SIGNALS that were seen, not @@ -3501,10 +3575,11 @@ def _upsert_incident(db, key, title, severity, kind, now, event_ids, "INSERT INTO incidents(kind,correlation_key,title,severity,status," "created_at,first_seen,last_seen,updated_at,reminder_count," "next_reminder_at,last_notified_at,subject_json," - "last_novel_at) " - "VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?)", + "last_novel_at,digest_only) " + "VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)", (kind, key, title, severity, "OPEN", now, now, now, now, 0, - now + _REMINDER_DELAYS[0], last_notified, subject_json, now)) + None if digest_why else now + _REMINDER_DELAYS[0], + last_notified, subject_json, now, digest_why)) incident_id = cur.lastrowid for event_id in event_ids: db.execute("INSERT OR IGNORE INTO incident_events(incident_id,event_id) " @@ -3547,12 +3622,37 @@ def _canon_entity_key(value): if os.path.isabs(value) else value +def _is_shared_interpreter(value): + """True when an entity names an interpreter (`/bin/bash`, `python3`, ...): + a program half the machine runs, so two findings that both name it share + nothing but the shell.""" + return os.path.basename(str(value)) in _INTERPRETERS + + +def _join_entities(f): + """`_entities(f)` minus every interpreter -- what a CHAIN may join on. + + `_entities()` keeps interpreters out of the secondary identities only; its + primary is returned unconditionally, because display, dedup and path + lineage all read it as the finding's own object. A chain cannot: live + incident #538 joined an OS update that replaced /bin/bash (primary + `/bin/bash`, custody os-vendor) to a harness command bash ran (primary + `/bin/bash`) into "Persistence followed by execution" -- the exact join + `_entities()` says must never happen, reached through the one side it did + not guard. #517 was the same hash under `chain:clickfix`. Excluded here + from BOTH sides, primary or not, so every chain rule inherits it; the + payload an interpreter runs is still carried as `script_target` and still + joins.""" + return [v for v in _entities(f) if not _is_shared_interpreter(v)] + + def _shared_entity(a, b): """The identity two findings have in common, or None. Returns the value as the finding REPORTED it (not the comparison form) so callers keep deriving - correlation keys exactly as they did when only the primary entity joined.""" - eb = {_canon_entity_key(v) for v in _entities(b)} - for v in _entities(a): + correlation keys exactly as they did when only the primary entity joined. + Never an interpreter, from either side: see `_join_entities`.""" + eb = {_canon_entity_key(v) for v in _join_entities(b)} + for v in _join_entities(a): if _canon_entity_key(v) in eb: return v return None @@ -3651,6 +3751,9 @@ def _category_dismissal_weights(db, now, window=90 * 86400): # facts — only hash churn generalizes. New endpoint = new incident. # · It takes REPEATED EXPOSURE: >= _TOLERANCE_MIN_VERDICTS distinct dismissed # incidents inside _TOLERANCE_WINDOW, so one hasty dismissal teaches nothing. +# (That floor is for identities that reach bytes nobody reviewed. The +# exact bytes, a publisher's team id and a package receipt need one; +# _TOLERANCE_FLOOR is the table.) # · INFLAMMATION OVERRIDES: never CRITICAL, never above the severity the # operator actually reviewed, never for attack-defined fingerprints, and # never while any incident on that identity is active (a `reopen` is a @@ -3661,6 +3764,18 @@ def _category_dismissal_weights(db, now, window=90 * 86400): # --------------------------------------------------------------------------- # _TOLERANCE_MIN_VERDICTS = 3 +# Verdicts an identity needs before it tolerates, by what it generalizes over +# -- the ONE table: exact bytes = 1 · signer / package = 1 · build repo / +# supervisor / producer = 3 · path identities = 3. The floor of three exists +# because a path- or class-shaped identity reaches bytes nobody reviewed. An +# exact-bytes identity (#51's content identity) reaches nothing the operator +# did not judge, so asking three times about one fact only made the teaching +# evaporate; a publisher chain or an install receipt is anchored outside this +# uid, so one verdict is the claim. `derived` classes are built from things +# the operator's own tools move, and keep three. +_TOLERANCE_FLOOR = {"exact": 1, "anchored": 1, + "derived": _TOLERANCE_MIN_VERDICTS, + "path": _TOLERANCE_MIN_VERDICTS} _TOLERANCE_WINDOW = 180 * 86400 # A trailing : component of this shape is a content hash, not identity. _TOLERANCE_HASH_RE = re.compile(r"^[0-9a-f]{12,64}$", re.I) @@ -3769,7 +3884,18 @@ def _tolerance_identity(fingerprint): changed = changed or (versionless != part) or ("#" in versionless) part = versionless normalized.append(part) - if not changed or len(normalized) < 2: + # What survives the strip must still NAME A SUBJECT, not just a category + # and a structural literal. `process:sha:` — the content-keyed shape + # the process sensor moved to — strips to the two components + # `process:sha`, which is not an identity at all: it is every content-keyed + # process incident on the machine sharing one bucket, so three + # benign-positive verdicts on three UNRELATED binaries would have tolerized + # the sensor outright. The floor of three exists to make tolerance + # antigen-specific and that string defeats it by naming no antigen. Three + # components is the real requirement (`persistence:changed:`, + # `process::`, `behavior:bash:` all clear it); the + # bytes keep their own identity via _fingerprint_content_identity. + if not changed or len(normalized) < 3: return None return ":".join(normalized) @@ -3932,7 +4058,15 @@ def _subject_identity(sub): return "persistence:%s:%s" % (sub.get("op") or "changed", path) if kind == "process" and generalizes: return "process:%s:%s" % (path, sub.get("trust") or "") - if kind == "beacon" and generalizes: + # A beacon subject now carries `content` so the content-keyed identity can + # be rendered from it — but content must NOT be what qualifies the + # PATH-keyed rendering below, or adding that field would silently grant + # every stable-path beacon the generalization _tolerance_identity still + # refuses it (see its `changed` guard: tolerating a never-normalized path + # hands a binary replaced IN PLACE at a reused endpoint the operator's + # verdicts). Path churn stays the only qualifier here, so the two + # derivations keep agreeing; the bytes get their own identity instead. + if kind == "beacon" and (path != sub.get("raw_path") or "#" in path): # net-beacon is named in the comment above as one of the three sensors # that declare a subject, and it was the one kind this function never # handled -- every beacon subject fell through to None. That was @@ -3949,6 +4083,59 @@ def _subject_identity(sub): return None +# A trailing full-length sha256 in a fingerprint is the subject's CONTENT. +_CONTENT_SHA_RE = re.compile(r"^[0-9a-f]{64}$", re.I) + + +def _content_identity(kind, content, ip=None, port=None): + """The content-keyed tolerance identity for a process or beacon, or None. + + This is the SECOND identity a verdict on these two sensors accumulates + under, never a replacement for the path-keyed one above — see + _finding_content_identity for why it had to be additive. + """ + content = str(content or "") + if not _CONTENT_SHA_RE.match(content): + return None + if kind == "process": + return "process:content:%s" % content.lower() + if kind == "beacon": + return "beacon:content:%s:%s:%s" % (content.lower(), ip or "", + port or "") + return None + + +def _subject_content_identity(sub): + """Content-keyed identity from a subject that carries its content hash.""" + if not isinstance(sub, dict): + return None + return _content_identity(sub.get("kind"), sub.get("content"), + sub.get("ip"), sub.get("port")) + + +def _fingerprint_content_identity(fp): + """The same identity recovered from a fingerprint string, for rows written + before their sensor declared a subject — and for BOTH process key shapes, + the pre-custody `process:::` and the content-keyed + `process:sha:` that replaced it. Recovering it from the old shape is + what lets verdicts the operator already gave carry forward on their own, + with no migration and no closer. + + It also closes a hole the new shape opened. `process:sha:` fed to + _tolerance_identity strips its trailing hash as churn and returns the + two-component identity `process:sha` — one identity shared by every + content-keyed process incident on the machine, so three benign-positive + verdicts on three UNRELATED binaries would have tolerized the sensor + outright. Claiming that string here means it never reaches that stripper. + """ + parts = str(fp or "").split(":") + if len(parts) < 3 or parts[0] != "process": + return None + if not _CONTENT_SHA_RE.match(parts[-1]): + return None + return _content_identity("process", parts[-1]) + + def _subject_endpoint_classes(sub): """Beacon endpoint classes from a subject's structured endpoint. A hostname is a fact and refuses to generalize, exactly as in the string @@ -3970,7 +4157,9 @@ def _subject_endpoint_classes(sub): ("beacon:%s:#ip:#port" % path, "%s:%s" % (ip, port))] -_PRODUCER_MIN_SIBLINGS = 3 +# A producer class is derived (from launcher bytes and a payload path the +# operator's tools write), so it keeps the derived floor (_TOLERANCE_FLOOR). +_PRODUCER_MIN_SIBLINGS = _TOLERANCE_FLOOR["derived"] def _producer_class(sub): @@ -4029,6 +4218,41 @@ def _finding_identity(f): return _tolerance_identity(f.get("fingerprint")) +def _finding_content_identity(f): + """The finding's content-keyed identity, ADDED to the path-keyed one above + rather than replacing it. + + Additive because the two cover opposite churn and neither covers both. A + vendor app updates IN PLACE or under a versioned directory: its path is + stable (or normalizes to one) while its bytes change every release, so + only the path-keyed identity can accumulate. The operator's own build + output does the reverse — one binary reproduced into staging dirs, release + dirs, per-agent worktrees, DMG scratch mounts and Downloads — so the bytes + are the stable thing and only a content-keyed identity can accumulate. + Keying on one of them alone means the other sensor population never + converges, which is exactly what the live queue showed: 30 process + verdicts spread over 28 path identities, every one of them below the + floor of three, so the process sensor had learned nothing at all. + + Content is also the only part of a process subject that is STABLE. The + same uv binary at ~/.local/bin/uv graded 'broken' when the + operator dismissed it and 'adhoc' when it reopened as #514, so the trust + class in the path-keyed identity manufactures churn of its own. + + This is strictly narrower than the identity it joins: it pins the exact + bytes, so replacing a vouched binary mints a new identity that has earned + nothing, and every existing guard still applies (never CRITICAL, never + above the reviewed severity, never a disputed identity, never + attack-defined). Because it reaches nothing the operator did not judge, + it tolerates on ONE verdict, not the three a path needs + (_TOLERANCE_FLOOR["exact"], 2026-09-23). + """ + sub = f.get("subject") + if sub: + return _subject_content_identity(sub) + return _fingerprint_content_identity(f.get("fingerprint")) + + def _finding_endpoint_classes(f): sub = f.get("subject") if sub: @@ -4164,6 +4388,23 @@ def _incident_identity(row): return _tolerance_identity(fp), _beacon_endpoint_classes(fp) +def _incident_content_identity(row): + """The stored incident's content-keyed identity, derived exactly as + _finding_content_identity derives a finding's — subject first, then the + correlation key — so a verdict and the finding it should later match + render the same string.""" + raw = row["subject_json"] if "subject_json" in row.keys() else None + if raw: + try: + ident = _subject_content_identity(json.loads(raw)) + if ident: + return ident + except Exception: + pass + return _fingerprint_content_identity( + (row["correlation_key"] or "")[len("signal:"):]) + + def _rotating_endpoint_memory(db, now): """{endpoint_class: (verdicts, max_reviewed_sev)} for classes the operator has dismissed across enough DISTINCT endpoints to establish rotation. @@ -4243,11 +4484,249 @@ def _producer_memory(db, now): return memory -def _tolerance_memory(db, now): - """{identity: (distinct_verdicts, max_reviewed_sev_order)} from the - operator's own benign-positive dismissals of signal incidents inside the - window. Only identities past the verdict floor are returned.""" - memory = {} +# --- class tolerance: a verdict teaches at the width the ladder can VERIFY --- +# +# Measured on the live store, 2026-09-23: 72 hand verdicts across 45 classes, +# then 109 new incidents opened in a class the operator had already judged. +# Every identity above names a PLACE or exact BYTES, and the operator's own +# work mints both on every rebuild, new worktree, runner self-update and +# translocated copy, so each re-sighting arrived as a stranger. The custody +# ladder already knows more than that about a binary: who signed it, which +# package transaction installed it, which repo built it, which vouched program +# started it. Each is a class a verdict can name, and a class does not move +# when the path or the bytes do. +# +# Width is set by what anchors the class, not by how useful it would be, and +# its floor is read from _TOLERANCE_FLOOR: +# anchored a publisher's team id (a signing chain this uid cannot mint) or +# a package receipt (an install transaction the operator ran; the +# honest limits written above _homebrew_receipt apply unchanged) +# -- ONE verdict; +# derived a build repo or a supervisor, derived from things the operator's +# own tools move and an attacker at this uid can reach -- the +# floor of three the path identities keep. +# Every other guard is unchanged: never CRITICAL, never attack-defined, never +# above the severity the operator reviewed, never a disputed class, never a +# _NEVER_TOLERATE_PREFIXES fingerprint. +# +# Two refusals are specific to classes. Only a finding ABOUT A BINARY carries +# one (process, net-beacon): a persistence job signed by team X can run anything +# X ships with arguments nobody reviewed. And an interpreter carries none: a +# signed python, node or pwsh beaconing is whatever script it runs, and its +# signer, receipt or repo says nothing about that script -- the rule S3 wrote +# for chain legs, applied to the widest join there is. +_CLASS_CATEGORIES = frozenset(("process", "net-beacon")) +# The finding fields a class is read from. _class_facts writes them at +# emission; _incident_classes pre-filters the store on them, which is sound +# only because _finding_classes reads nothing else to name a class. +_CLASS_TRIGGER_KEYS = ("team", "package", "build_repo", "supervisor") +# Display-only companions: what a lesson prints, never what a key is made of. +_CLASS_FACT_KEYS = _CLASS_TRIGGER_KEYS + ("authority", "supervisor_path") +_TEAM_ID_RE = re.compile(r"^[A-Z0-9]{10}$") +# Script hosts outside _INTERPRETERS that a publisher signs and that run +# whatever they are handed. Not a detection list; only a refusal to let one +# binary's signature stand for everything it can be told to do. +_CLASS_SCRIPT_HOSTS = frozenset(("pwsh", "powershell", "cmd", "wscript", + "cscript", "mshta", "java", "deno", "bun")) + + +def _class_interpreter(path): + """True when `path` is a program that runs what it is handed.""" + name = os.path.basename(str(path or "").replace("\\", "/")).lower() + if name.endswith(".exe"): + name = name[:-4] + return (name in _INTERPRETERS or name in _CLASS_SCRIPT_HOSTS + # python3.13, node18: the versioned name of the same host. + or name.rstrip("0123456789.") in _INTERPRETERS) + + +def _package_class(receipt): + """`:` a package receipt names, without its version, or + None when it names no package (a winget shim is a link, not a package). + + Homebrew receipts read `formula@version` and a formula's own name can + carry a version (`python@3.14`), so only the trailing `@version` goes; + every other manager spells the version into the name and it is folded + with the tolerance layer's own version regex.""" + manager, sep, name = str(receipt or "").partition(":") + if not (sep and manager and name): + return None + if manager == "homebrew": + name = name.rsplit("@", 1)[0] + elif manager == "winget" and name == "link": + return None + else: + name = _TOLERANCE_VERSION_RE.sub("#", name) + return "%s:%s" % (manager, name) if name else None + + +def _finding_classes(f): + """[(class_key, width)] a finding carries, in the order _signal_decision + consults them: signer, then package, build repo, supervisor. + + The ONE spelling of a class key. The memory a verdict builds and the + lookup a later finding makes both call this, so the two can never + disagree on what a class is called -- the drift _subject_identity + documents, where a subject and its fingerprint rendered different + identities and the verdicts split between two memories. Pure: it reads + the facts the emitter attached (_class_facts) and asks the machine + nothing.""" + if not isinstance(f, dict) or f.get("category") not in _CLASS_CATEGORIES: + return [] + if f.get("attack_defined") or str(f.get("fingerprint") or "").startswith( + _NEVER_TOLERATE_PREFIXES): + return [] + path = f.get("path") or f.get("program") + if not path or _class_interpreter(path): + return [] + out = [] + team = str(f.get("team") or "") + if _TEAM_ID_RE.match(team) and publisher_sig(f.get("trust")): + out.append(("signer:%s" % team, "anchored")) + custody = f.get("custody") + if custody == "package-managed": + name = _package_class(f.get("package")) + if name: + out.append(("package:%s" % name, "anchored")) + elif custody == "build-output": + repo = str(f.get("build_repo") or "") + if repo and os.path.isabs(repo): + out.append(("buildrepo:%s" % repo, "derived")) + elif custody == "supervised": + sup = str(f.get("supervisor") or "") + if _CONTENT_SHA_RE.match(sup): + out.append(("supervisor:%s" % sup.lower(), "derived")) + return out + + +def _class_facts(path, trust, custody, parents=None): + """The class facts a binary sensor attaches to its finding, as fields. + + Asked right after grading, so every answer here is one the ladder has + just established: the team only under a publisher verdict the classifier + repeats, the receipt only on the package rung, the repo only on the + build-output rung (and only when git said yes, never on a non-answer), the + supervisor's bytes only on the supervised rung. Each call is a cache hit + for the scan that graded it. Never raises: a fact that cannot be read + costs that fact, never the scan.""" + facts = {} + if not path: + return facts + try: + if publisher_sig(trust): + sig = classify_signature(path) or {} + team = str(sig.get("team") or "") + if sig.get("trust") == trust and _TEAM_ID_RE.match(team): + facts["team"] = team + if sig.get("authority"): + facts["authority"] = sig["authority"] + if custody == "package-managed": + receipt = _package_receipt(path) + if receipt: + facts["package"] = receipt + elif custody == "build-output": + git = _git_bin() + selfness = _repo_is_self_committed( + git, os.path.dirname(os.path.abspath(path))) if git else None + if selfness and selfness[0] and selfness[1] is True: + facts["build_repo"] = selfness[0] + elif custody == "supervised": + parent = _supervising_parent(path, parents) + sha = _graded_sha(parent) if parent else None + if sha: + facts["supervisor"] = sha + facts["supervisor_path"] = parent + except Exception: + pass + return facts + + +def _incident_classes(db, incident_id): + """[(class_key, width, finding)] named by an incident's recorded evidence, + one entry per class. `finding` is the newest evidence that names the + class, for a lesson to describe it by. + + Read from the NEWEST observation of each signal only -- the latest word + on its subject -- and a row is parsed only if it carries one of + _CLASS_TRIGGER_KEYS, without which it names no class by construction. + This runs every scan over every judged incident, and a judged incident + holds up to thousands of re-observations of the same few signals (the + live store: ~11,000 evidence rows over ~150 signals); pattern-matching + them all in SQL cost ~70 ms of CPU a scan to learn what the newest row + per signal already says.""" + try: + newest = sorted((r[0] for r in db.execute( + "SELECT MAX(x.id) FROM incident_events ie " + "JOIN events x ON x.id=ie.event_id WHERE ie.incident_id=? " + "AND x.event_type='observation.finding' " + # A row with no signal id is its own signal, never one shared + # bucket of every such row. + "GROUP BY COALESCE(x.signal_id, -x.id)", (incident_id,))), + reverse=True) + raws = [] + for i in range(0, len(newest), 500): + chunk = newest[i:i + 500] + raws.extend(r[0] for r in db.execute( + "SELECT data_json FROM events WHERE id IN (%s) " + "ORDER BY id DESC" % ",".join("?" * len(chunk)), chunk)) + except sqlite3.Error: + # A store with no evidence links names no class: the memory built + # from it teaches none, so there is none for a dispute to revoke. + return [] + marks = tuple('"%s":' % k for k in _CLASS_TRIGGER_KEYS) + out = {} + for raw in raws: + if not any(m in (raw or "") for m in marks): + continue + try: + f = json.loads(raw) + except Exception: + continue + for klass, width in _finding_classes(f): + out.setdefault(klass, (width, f)) + return [(k, w, f) for k, (w, f) in sorted(out.items())] + + +def _class_verdicts(db, now): + """{class_key: {"inc", "sev", "width", "finding"}} over the operator's + benign-positive verdicts on signal incidents inside the window, before + any floor is applied. Distinct INCIDENTS, as for every tolerance tier: + one verdict writes a dismissal row per category.""" + seen = {} + try: + rows = db.execute( + "SELECT DISTINCT d.incident_id, i.severity " + "FROM dismissals d JOIN incidents i ON i.id=d.incident_id " + "WHERE d.reason_code='benign-positive' AND d.dismissed_at>=? " + "AND d.correlation_key LIKE 'signal:%'", + (now - _TOLERANCE_WINDOW,)).fetchall() + for row in rows: + for klass, width, f in _incident_classes(db, row["incident_id"]): + bucket = seen.setdefault(klass, {"inc": set(), "sev": -1, + "width": width, "finding": f}) + bucket["inc"].add(row["incident_id"]) + bucket["sev"] = max(bucket["sev"], + SEV_ORDER.get(row["severity"], -1)) + except Exception: + return {} + return seen + + +def _class_memory(db, now): + """{class_key: (verdicts, max_reviewed_sev)} for classes past the floor + their width sets (_TOLERANCE_FLOOR).""" + return {k: (len(b["inc"]), b["sev"]) + for k, b in _class_verdicts(db, now).items() + if len(b["inc"]) >= _TOLERANCE_FLOOR.get( + b["width"], _TOLERANCE_MIN_VERDICTS)} + + +def _tolerance_verdicts(db, now): + """{identity: {"incidents", "sev", "width"}} from the operator's own + benign-positive dismissals of signal incidents inside the window, before + any floor is applied. `width` is "path" for the path-keyed identity and + "exact" for the content-keyed one (_TOLERANCE_FLOOR).""" + seen = {} try: rows = db.execute( "SELECT d.incident_id, d.correlation_key, i.severity, " @@ -4257,20 +4736,31 @@ def _tolerance_memory(db, now): "AND d.correlation_key LIKE 'signal:%'", (now - _TOLERANCE_WINDOW,)).fetchall() except Exception: - return memory - seen = {} + return seen for row in rows: - ident = _incident_identity(row)[0] - if not ident: - continue - bucket = seen.setdefault(ident, {"incidents": set(), "sev": -1}) - bucket["incidents"].add(row["incident_id"]) - bucket["sev"] = max(bucket["sev"], - SEV_ORDER.get(row["severity"], -1)) - for ident, bucket in seen.items(): - if len(bucket["incidents"]) >= _TOLERANCE_MIN_VERDICTS: - memory[ident] = (len(bucket["incidents"]), bucket["sev"]) - return memory + # Both identities a verdict counts toward: the path-keyed one and, + # where the subject or key names the bytes, the content-keyed one. + # A single verdict contributes one incident to each bucket, so a + # floor always counts DISTINCT incidents either way. + for ident, width in ((_incident_identity(row)[0], "path"), + (_incident_content_identity(row), "exact")): + if not ident: + continue + bucket = seen.setdefault(ident, {"incidents": set(), "sev": -1, + "width": width}) + bucket["incidents"].add(row["incident_id"]) + bucket["sev"] = max(bucket["sev"], + SEV_ORDER.get(row["severity"], -1)) + return seen + + +def _tolerance_memory(db, now): + """{identity: (distinct_verdicts, max_reviewed_sev_order)} for the + identities past the floor their width sets: one verdict for the exact + bytes, three for a path (_TOLERANCE_FLOOR).""" + return {ident: (len(b["incidents"]), b["sev"]) + for ident, b in _tolerance_verdicts(db, now).items() + if len(b["incidents"]) >= _TOLERANCE_FLOOR[b["width"]]} def _disputed_identities(db): @@ -4305,6 +4795,13 @@ def _disputed_identities(db): ident, classes = _incident_identity(row) if ident: idents.add(ident) + # A dispute is about a SUBJECT, so it has to reach every identity that + # subject accumulates under — otherwise a `reopen` revokes the + # path-keyed tolerance while the content-keyed one keeps closing the + # very incidents the operator just reopened. + content_ident = _incident_content_identity(row) + if content_ident: + idents.add(content_ident) for klass, _observed in classes: idents.add(klass) raw = row["subject_json"] if "subject_json" in row.keys() else None @@ -4314,6 +4811,11 @@ def _disputed_identities(db): idents.add(klass) except Exception: pass + # A class is the widest thing a verdict teaches, so a dispute on ANY + # incident in it revokes it for all of them -- including the ones it + # has already closed. + for klass, _width, _f in _incident_classes(db, row["id"]): + idents.add(klass) return idents @@ -4367,13 +4869,38 @@ def _accumulate_risk(db, now, new_ids): "SELECT id, observed_at, data_json FROM events " "WHERE event_type='observation.finding' AND observed_at>=?", (now - RISK_WINDOW,)).fetchall() - demote = _category_dismissal_weights(db, now) - by_entity = {} + findings = [] for row in rows: try: - f = json.loads(row["data_json"]) + findings.append((row["id"], json.loads(row["data_json"]))) except Exception: continue + by_entity = _risk_buckets(findings, _category_dismissal_weights(db, now), + new_ids) + for ek, b in by_entity.items(): + score, min_signals = _risk_score(b) + if b["new"] and len(b["fps"]) >= min_signals \ + and score >= RISK_THRESHOLD: + # Severity follows the score instead of a hardcoded "HIGH": + # barely past threshold is a MEDIUM worth a look, not an + # interrupt-grade verdict the number never supported. + sev = "HIGH" if score >= 2 * RISK_THRESHOLD else "MEDIUM" + _upsert_incident( + db, "risk:%s" % ek, + "Accumulated risk on %s (%d signals across %d sensor%s, score %.1f)" + % (b["entity"][:80], len(b["fps"]), len(b["cats"]), + "" if len(b["cats"]) == 1 else "s", score), + sev, "risk", now, sorted(b["ids"])) + + +def _risk_buckets(findings, demote, new_ids=()): + """{entity key: {weight, fps, ids, cats, entity, new}} for [(event id, + finding)]: what _accumulate_risk sums per entity before it scores, with + `demote` the per-category dismissal weights. One function, so a pile + re-scored later (_rejudge_open_incidents) is scored the way it was + opened.""" + by_entity = {} + for event_id, f in findings: # Single primary entity on purpose, unlike the chain rules: accumulation # counts weak signals piling up on ONE object, so bucketing a finding # under its every identity would both double-count it and pool unrelated @@ -4449,29 +4976,22 @@ def _accumulate_risk(db, now, new_ids): continue # count each distinct signal once, not once per rescan b["fps"].add(fp) b["weight"] += w - b["ids"].add(row["id"]) + b["ids"].add(event_id) b["cats"].add(_RISK_SENSOR_GROUP.get(category, category)) - if row["id"] in new_ids: + if event_id in new_ids: b["new"] = True - for ek, b in by_entity.items(): - # Corroboration across sensors is the higher-precision evidence: it needs - # fewer signals AND scores higher against the same constant threshold. - # One sensor keeps the original bar, so no existing detection regresses. - multi = len(b["cats"]) >= 2 - min_signals = RISK_MIN_SIGNALS_MULTI_SENSOR if multi else RISK_MIN_SIGNALS - score = b["weight"] * (RISK_CORROBORATION_BONUS if multi else 1.0) - if b["new"] and len(b["fps"]) >= min_signals \ - and score >= RISK_THRESHOLD: - # Severity follows the score instead of a hardcoded "HIGH": - # barely past threshold is a MEDIUM worth a look, not an - # interrupt-grade verdict the number never supported. - sev = "HIGH" if score >= 2 * RISK_THRESHOLD else "MEDIUM" - _upsert_incident( - db, "risk:%s" % ek, - "Accumulated risk on %s (%d signals across %d sensor%s, score %.1f)" - % (b["entity"][:80], len(b["fps"]), len(b["cats"]), - "" if len(b["cats"]) == 1 else "s", score), - sev, "risk", now, sorted(b["ids"])) + return by_entity + + +def _risk_score(b): + """(score, signals needed) for one _risk_buckets entry.""" + # Corroboration across sensors is the higher-precision evidence: it needs + # fewer signals AND scores higher against the same constant threshold. + # One sensor keeps the original bar, so no existing detection regresses. + multi = len(b["cats"]) >= 2 + min_signals = RISK_MIN_SIGNALS_MULTI_SENSOR if multi else RISK_MIN_SIGNALS + return (b["weight"] * (RISK_CORROBORATION_BONUS if multi else 1.0), + min_signals) # --------------------------------------------------------------------------- # @@ -4499,13 +5019,15 @@ def _accumulate_risk(db, now, new_ids): def _suppression_memory(db, now): """Everything the incident tier knows that can quiet a signal, read once - per scan: (tolerance, rotating, disputed, learning).""" + per scan: (tolerance, rotating, disputed, learning, producer, classes).""" tolerance = _tolerance_memory(db, now) rotating = _rotating_endpoint_memory(db, now) producer = _producer_memory(db, now) - disputed = _disputed_identities(db) if (tolerance or rotating or producer) \ - else frozenset() - return tolerance, rotating, disputed, _in_learning_period(now), producer + classes = _class_memory(db, now) + disputed = _disputed_identities(db) \ + if (tolerance or rotating or producer or classes) else frozenset() + return (tolerance, rotating, disputed, _in_learning_period(now), producer, + classes) def _signal_decision(f, memory): @@ -4518,6 +5040,7 @@ def _signal_decision(f, memory): return None, 0 tolerance, rotating, disputed, learning = memory[:4] producer = memory[4] if len(memory) > 4 else {} + classes = memory[5] if len(memory) > 5 else {} sev = SEV_ORDER.get(f.get("severity"), -1) if sev >= SEV_ORDER["CRITICAL"]: return None, 0 @@ -4530,6 +5053,14 @@ def _signal_decision(f, memory): ident = _finding_identity(f) if ident: candidates.append((ident, tolerance)) + content_ident = _finding_content_identity(f) + if content_ident: + candidates.append((content_ident, tolerance)) + # Then the classes the ladder verified (signer, package, build repo, + # supervisor), each already past the floor its width sets. + if classes: + for klass, _width in _finding_classes(f): + candidates.append((klass, classes)) if rotating: for klass, _observed in _finding_endpoint_classes(f): candidates.append((klass, rotating)) @@ -4552,14 +5083,44 @@ def _signal_decision(f, memory): return None, 0 -def route_findings(findings, first_run=False, adopt=frozenset(), memory=None): +def _provenance_gate(f): + """The custody rung that keeps `f` out of the interrupt tier, or None. + + Custody used to be asked last and could only demote, one step; nothing + consulted it before an interrupt, and 190 of the 200 HIGH interrupts the + operator closed as noise (30 days to 2026-09-23) carried no rung at all. + A finding whose ORIGIN is proven -- a self-tier rung (the operator + authored it) or a vouched-tier rung (a signed publisher, a package + receipt, an OS update) -- is an attribute-only observation about a known + thing, and belongs in the digest, where it stays visible and still counts + toward risk at its tier weight. + + Three things are never gated, whatever their custody: CRITICAL, evidence + that is attack-DEFINED (a hostile argv, an IOC hit, a conceal imperative + -- origin is not innocence), and a tripped decoy/latch/canary, which no + provenance explains. Weak rungs (`build-output`, `supervised`, ...) + explain nothing here; they only ever buy the grader's one step.""" + if SEV_ORDER.get(f.get("severity"), -1) >= SEV_ORDER["CRITICAL"]: + return None + if str(f.get("fingerprint") or "").startswith(_NEVER_TOLERATE_PREFIXES): + return None + if not _custody_explained(f): + return None + return f.get("custody") or f.get("provenance") + + +def route_findings(findings, first_run=False, adopt=frozenset(), memory=None, + seen=None): """{fingerprint: {"route", "why", "decision", "verdicts"}} for a batch. `route` is the interrupt-tier outcome. `decision` is the incident-tier outcome ("allowlisted", "tolerated", "learning" or None), carried separately because a finding the seen-ledger already knows still needs its incident decided. `memory` is _suppression_memory(...) or None, in - which case only the interrupt-tier checks apply. + which case only the interrupt-tier checks apply. `seen` is the seen-ledger + to consult, read from SEEN when None; `backtest replay` passes its own, + built as it goes, because the live ledger already holds every fingerprint + the replay is asking about. First-run silence is the KnockKnock "trust what's already installed" rule — it applies to PERSISTENCE and SHELL-HISTORY only, the two surfaces made @@ -4570,8 +5131,13 @@ def route_findings(findings, first_run=False, adopt=frozenset(), memory=None): on the very first scan. `adopt` is the same rule for a surface an upgrade sees for the first time. Confidence is the second routing axis: a high-impact-but-noisy hit (explicit confidence='low') is logged and - correlated but routed to the digest instead of interrupting.""" - seen = load_json(SEEN, {}) + correlated but routed to the digest instead of interrupting. Provenance + is the third: a finding whose origin custody has proven routes to the + digest at any severity below CRITICAL (see _provenance_gate), and says so + on the finding itself as `routed`, which the incident view and the report + print.""" + if seen is None: + seen = load_json(SEEN, {}) allow = set(load_json(ALLOWLIST, [])) out = {} for f in findings: @@ -4579,6 +5145,8 @@ def route_findings(findings, first_run=False, adopt=frozenset(), memory=None): if fp in out: continue decision, verdicts = _signal_decision(f, memory) + gate = _provenance_gate(f) + f.pop("routed", None) if fp in allow: route, why, decision = ROUTE_SILENT, "allowlisted", "allowlisted" elif fp in seen: @@ -4588,6 +5156,9 @@ def route_findings(findings, first_run=False, adopt=frozenset(), memory=None): route, why = ROUTE_DIGEST, "adopted" elif CONFIDENCE_ORDER.get(f.get("confidence", "medium"), 1) <= 0: route, why = ROUTE_DIGEST, "low-confidence" + elif gate: + route, why = ROUTE_DIGEST, "provenance:%s" % gate + f["routed"] = "digest: provenance %s" % gate elif SEV_ORDER[f["severity"]] < SEV_ORDER[NOTIFY_MIN_SEV]: route, why = ROUTE_DIGEST, "below-floor" elif decision: @@ -5090,6 +5661,24 @@ def _chain_severity(leg_pairs, attack_defined=False): return best +def _custody_explained(f): + """True when custody has already explained where this finding came from: + its rung is in the self tier (the operator authored it) or the vouched + tier (an OS update, a package receipt, a same-publisher re-sign). + + Such a finding cannot TRIGGER a chain -- see `_apply_correlations`. It may + still be the other leg of one an unexplained finding triggers, and + attack-defined evidence is never explained, the same refusal `_demote()` + makes: knowing who wrote a payload is not a reason to stop calling it one. + Weak rungs (`build-output`, `worktree`, ...) explain nothing here; they + only ever buy one step of demotion. The sensors carry the rung as + `custody`; the agent-surface diff calls it `provenance`.""" + if f.get("attack_defined"): + return False + rung = f.get("custody") or f.get("provenance") or "" + return rung in _SELF_CUSTODY + _VOUCHED_CUSTODY + + def _apply_correlations(db, new_events, now, initially_notified=False, suppressed_categories=frozenset(), routing=None): """Run a deliberately tiny set of high-precision, versioned chain rules.""" @@ -5130,6 +5719,15 @@ def _apply_correlations(db, new_events, now, initially_notified=False, new_ids = {event_id for event_id, f in new_events if f.get("category") not in suppressed_categories and event_id not in quieted} + # Provenance, by the same pattern: a finding custody has already explained + # leaves the TRIGGER set and stays in `observations`, so it can still be + # the other leg of a chain an unexplained finding triggers. #538's left + # leg was an OS update custody had graded os-vendor, LOW, in its own + # detail. A set of its own rather than a narrower `new_ids`: the risk + # tier reads `new_ids` too, and already weights these rungs itself + # (_RISK_CUSTODY_WEIGHT). + chain_triggers = {event_id for event_id, f in new_events + if event_id in new_ids and not _custody_explained(f)} attached = set() # (entity_key, incident_id, evidence_ids) for every chain raised this scan, # so overlapping rules can be reconciled once they have all run. See @@ -5142,7 +5740,7 @@ def correlate(base_key, title, left_pred, right_pred, window=900, legs_by_entity = {} for left_id, right_id, left, right in _correlation_pairs( observations, left_pred, right_pred, window): - if left_id in new_ids or right_id in new_ids: + if left_id in chain_triggers or right_id in chain_triggers: entity = _canon_entity_path(_shared_entity(left, right)) entity_key = hashlib.sha256( entity.encode("utf-8", "replace")).hexdigest()[:16] @@ -5285,7 +5883,8 @@ def has_marker(f, values): incident_id = _upsert_incident( db, "signal:" + (f.get("case_fingerprint") or f["fingerprint"]), f["title"], f["severity"], "signal", now, [event_id], - notified or bool(decision), subject=f.get("subject")) + notified or bool(decision), subject=f.get("subject"), + route=verdict) if decision == "allowlisted": _close_allowlisted(db, incident_id, now) elif decision == "tolerated": @@ -5735,6 +6334,233 @@ def _close_removed_drop_incidents(db, observed, now): return len(closed) +# Re-judging open incidents with the current code (_rejudge_open_incidents). +# Bump _REJUDGE_LOGIC_VERSION when what may close changes; a new aegis.py is a +# new logic too (its code sha is part of the key), so an install re-judges on +# its first scan instead of an hour later. +_REJUDGE_LOGIC_VERSION = 1 +_REJUDGE_INTERVAL = 3600 +_REJUDGE_MAX_INCIDENTS = 25 # per run; the rest wait for the next one +_REJUDGE_BUDGET = 30.0 # seconds of re-derivation per run +_REJUDGE_KINDS = ("signal", "risk", "correlation") + + +def _rejudge_logic(): + """What judged: this closer's version and the code it ran in.""" + return "%d:%s" % (_REJUDGE_LOGIC_VERSION, + (_running_code_sha() or "unknown")[:12]) + + +def _rejudge_subject(f): + """` `, the way a resolution names a finding.""" + return ("%s %s" % (f.get("category") or "finding", os.path.basename( + str(f.get("path") or f.get("program") or "")))).strip() + + +def _rejudge_change(f, g): + """What the current code says differently about `f`, re-derived as `g`.""" + parts = [] + if g.get("trust") and (g.get("trust") or None) != (f.get("trust") or None): + parts.append("now classifies %s" % g["trust"]) + rung = g.get("custody") or g.get("provenance") + if rung and rung != (f.get("custody") or f.get("provenance")): + parts.append("custody %s" % rung) + if g["severity"] != f["severity"]: + parts.append("%s -> %s" % (f["severity"], g["severity"])) + return "%s: %s" % (_rejudge_subject(f), ", ".join(parts) or "as recorded") + + +def _rejudge_summary(reasons, limit=4): + distinct = [] + for why in reasons: + if why not in distinct: + distinct.append(why) + extra = len(distinct) - limit + return "; ".join(distinct[:limit]) + ("; +%d more" % extra + if extra > 0 else "") + + +def _rejudge_incident(db, row, memory, demote, memo, stats): + """Why the current code would not raise incident `row` today, as one + line, or None when it would, or when that cannot be shown. + + Every observation finding it holds is re-derived by `_reobserve`, the + path `backtest replay --reobserve` scores with. Any finding replayed as + recorded -- a subject gone from disk, a sensor it does not model, a field + the record never carried -- and the incident stands: absence of an answer + is not an answer. So does any CRITICAL, attack-defined or never-tolerate + evidence, whatever the current grade. Then, by kind: a signal case needs + every finding dropped or routed below the interrupt tier; a risk pile is + scored again (_risk_buckets, _risk_score) the way _accumulate_risk scores + it -- over the findings inside one RISK_WINDOW, at each moment one of them + was observed -- and no window may still cross; a co-occurrence chain must + be one the current join rules would not form + (_unjoinable_chain_resolution).""" + evidence = [] + for ev in db.execute( + "SELECT e.id, e.observed_at, e.data_json FROM incident_events ie " + "JOIN events e ON e.id=ie.event_id WHERE ie.incident_id=? AND " + "e.event_type='observation.finding' ORDER BY e.id", (row["id"],)): + try: + f = json.loads(ev["data_json"]) + except (ValueError, TypeError): + return None + if not isinstance(f, dict) or not f.get("fingerprint") \ + or f.get("severity") not in SEV_ORDER: + return None + if SEV_ORDER[f["severity"]] >= SEV_ORDER["CRITICAL"] \ + or f.get("attack_defined") \ + or str(f["fingerprint"]).startswith(_NEVER_TOLERATE_PREFIXES): + return None + evidence.append((ev["id"], ev["observed_at"] or 0, f)) + if not evidence: + return None + judged = [] + for event_id, at, f in evidence: + g, as_recorded = _reobserve(dict(f), memo, stats) + if as_recorded: + return None + judged.append((event_id, at, f, g)) + kind, key = row["kind"], row["correlation_key"] or "" + if kind == "signal": + reasons, routed = [], set() + for _event_id, _at, f, g in judged: + if g is None: + reasons.append("%s: no longer emitted" % _rejudge_subject(f)) + continue + if g["fingerprint"] in routed: + continue # one fact, re-derived once per scan held + routed.add(g["fingerprint"]) + verdict = route_findings([dict(g)], memory=memory, + seen={})[g["fingerprint"]] + if verdict["route"] == ROUTE_INTERRUPT: + return None + reasons.append("%s (%s: %s)" % (_rejudge_change(f, g), + verdict["route"], verdict["why"])) + return _rejudge_summary(reasons) + kept = [(event_id, at, g) for event_id, at, _f, g in judged + if g is not None] + if kind == "risk" and key.startswith("risk:"): + # Each scan scores the findings of the last RISK_WINDOW, so the + # evidence of an incident that re-crossed for days is many piles, + # never one: summed whole, a program that picks a new port per launch + # is 68 distinct signals (#527). Every window the evidence holds is + # asked, and the highest is reported. + peak = (0.0, 0, RISK_MIN_SIGNALS) + for anchor in sorted({at for _event_id, at, _g in kept}): + b = _risk_buckets( + [(event_id, g) for event_id, at, g in kept + if anchor - RISK_WINDOW <= at <= anchor], + demote).get(key[len("risk:"):]) + if not b: + continue + score, needed = _risk_score(b) + if len(b["fps"]) >= needed and score >= RISK_THRESHOLD: + return None + peak = max(peak, (score, len(b["fps"]), needed)) + changes = [_rejudge_change(f, g) if g is not None + else "%s: no longer emitted" % _rejudge_subject(f) + for _event_id, _at, f, g in judged] + return _rejudge_summary(changes + [ + "the pile now peaks at %.1f from %d signal(s) in any %d-minute " + "window, under the risk threshold (%.1f from %d)" + % (peak[0], peak[1], RISK_WINDOW // 60, RISK_THRESHOLD, peak[2])]) + if kind == "correlation" and key.startswith("chain:") \ + and not key.startswith("chain:lineage:"): + why = _unjoinable_chain_resolution(_chain_entity_key(key), + [g for _event_id, _at, g in kept]) + if not why: + return None + return why.replace("superseded: ", "", 1).replace( + " — reopens on new evidence", "") + return None + + +def _rejudge_open_incidents(db, now): + """Close OPEN/ACK incidents the CURRENT code would not raise; the number + closed, or None when this run was throttled. + + Every other exit waits for the sensor to say something: re-grade reads + the newest evidence, re-verify re-asks one signature, cleared-state and + removed-file need the sensor to look again. An incident opened by code + that has since been fixed hears none of that unless the same subject is + re-observed, so it waited out the age-out clock. Live, 2026-09-23: #527, + a Spotify risk case built from listener findings recorded `unsigned` + while codesign was not answering (Spotify is Developer ID), and #537, a + staging plugin-container the current code grades MEDIUM on its + build-output rung. The harness already re-derives recorded evidence with + the current code; this uses the same path to heal, not only to score. + + Judged with the live teaching (the suppression memory, the dismissal + weights), the learning period off -- the question is whether the code + would raise this, not whether the machine is still learning -- and the + seen-ledger empty, since having told the operator once is not a verdict. + Read-only on everything but the incident row: the custody ledger is not + written while the ladder is asked. See _rejudge_incident for what stands. + + The discipline is the other machine exits': FALSE_POSITIVE with a + resolution that says why and which logic judged it, no dismissals row + (a machine verdict never feeds precision or tolerance), and new evidence + opens the case again. Never an incident created by the scan that runs it. + + Hourly at most, and at once when the logic changes (a new version or a + new aegis.py). Bounded per run by _REJUDGE_MAX_INCIDENTS and + _REJUDGE_BUDGET; a run that stops early says how many it left, and the + next resumes after the last incident it examined.""" + logic = _rejudge_logic() + meta = {r["key"]: r["value"] for r in db.execute( + "SELECT key, value FROM meta WHERE key IN " + "('rejudge_logic','rejudge_at','rejudge_cursor')")} + try: + last, cursor = int(meta.get("rejudge_at") or 0), \ + int(meta.get("rejudge_cursor") or 0) + except ValueError: + last, cursor = 0, 0 + if meta.get("rejudge_logic") == logic and now - last < _REJUDGE_INTERVAL: + return None + marks = ",".join("?" for _ in _REJUDGE_KINDS) + rows = db.execute( + "SELECT id, kind, correlation_key FROM incidents WHERE status IN " + "('OPEN','ACK') AND kind IN (%s) AND severity<>'CRITICAL' AND " + "created_at cursor] + \ + [r for r in rows if r["id"] <= cursor] + memory = _suppression_memory(db, now) + memory = tuple(memory[:3]) + (False,) + tuple(memory[4:]) + demote = _category_dismissal_weights(db, now) + memo, stats = {}, _reobserve_stats() + deadline = time.monotonic() + _REJUDGE_BUDGET + closed, examined = [], [] + with _replay_overrides(_custody_remember=lambda *_a, **_k: False): + for row in rows: + if len(examined) >= _REJUDGE_MAX_INCIDENTS \ + or time.monotonic() > deadline: + break + examined.append(row["id"]) + why = _rejudge_incident(db, row, memory, demote, memo, stats) + if why: + closed.append((row["id"], why)) + for incident_id, why in closed: + db.execute( + "UPDATE incidents SET status='FALSE_POSITIVE',resolution=?," + "updated_at=?,next_reminder_at=NULL WHERE id=? AND status IN " + "('OPEN','ACK')", + ("re-judged by current code (logic %s): %s — reopens on new " + "evidence" % (logic, why), now, incident_id)) + left = len(rows) - len(examined) + resume = (examined[-1] if examined else cursor) if left else 0 + for key, value in (("rejudge_logic", logic), ("rejudge_at", now), + ("rejudge_cursor", resume)): + db.execute("INSERT INTO meta(key,value) VALUES(?,?) ON CONFLICT(key) " + "DO UPDATE SET value=excluded.value", (key, str(value))) + if rows: + log_run("re-judged %d open incident(s) with current code (logic %s), " + "closed %d%s" % (len(examined), logic, len(closed), + "; %d left for the next run" % left + if left else "")) + return len(closed) + + def _age_out_incidents(db, now, days=_AGE_OUT_DAYS): """Close OPEN incidents that stopped producing evidence, and say so. @@ -6312,6 +7138,144 @@ def _retire_orphaned_behavior_incidents(db, now): return len(aged) +def _unjoinable_chain_resolution(entity_key, legs): + """The superseded resolution for a chain keyed on `entity_key` that the + current join rules would not form from `legs`, or None when they still + would -- or when this cannot tell which entity the chain joined on.""" + def keyed(value): + return hashlib.sha256(_canon_entity_path(value).encode( + "utf-8", "replace")).hexdigest()[:16] == entity_key + joined = next((v for f in legs for v in _entities(f) if keyed(v)), None) + if joined is None: + return None + if _is_shared_interpreter(joined): + return ("superseded: a chain cannot join on an interpreter (%s) — " + "reopens on new evidence" % joined) + holders = [f for f in legs if any(keyed(v) for v in _join_entities(f))] + if len(holders) < 2 or not all(_custody_explained(f) for f in holders): + return None + rungs = sorted({str(f.get("custody") or f.get("provenance")) + for f in holders}) + return ("superseded: every leg of this chain is provenance-explained " + "(custody %s), and a chain needs a leg custody cannot explain — " + "reopens on new evidence" % ", ".join(rungs)) + + +def _retire_unjoinable_chain_incidents(db, now): + """One-time: close the chain incidents the 2026-09-23 join rules would not + have formed. + + Two rules shipped together in the correlation tier: an interpreter is + never the shared entity of a chain (`_join_entities`), and a finding + custody has explained cannot trigger one (`_custody_explained`). Both are + forward-only, and a chain is an event, not a state: an incident they would + have prevented -- #538, an OS update joined to a harness command on + /bin/bash -- receives no evidence that could re-grade it, so it could only + wait out the age-out clock. + + Judged on the incident's own EVIDENCE, the way the forward rules judge a + pair. The entity the chain was keyed on is recovered by hashing each leg's + identities as correlate() does; an interpreter there is rule 1. Otherwise + the chain stands while two legs hold that entity and at least one of them + is unexplained -- the forward code would form it again, the explained leg + as its other leg -- and only a chain whose every leg on it is explained is + rule 2. An entity no leg reproduces is left alone: a chain this cannot + re-derive is not one it may retire. + + Path lineage (`chain:lineage:`) is out of scope: it joins a remembered + drop, not a co-occurrence, and neither rule changed it. Incidents created + in the scan that runs the migration are out of scope by construction. + Closed as SUPERSEDED with evidence intact, as the other identity + migrations are, and no dismissals row is written. + """ + retired = 0 + for row in db.execute( + "SELECT id, correlation_key FROM incidents WHERE " + "kind='correlation' AND status IN ('OPEN','ACK') AND " + "correlation_key LIKE 'chain:%' AND " + "correlation_key NOT LIKE 'chain:lineage:%' AND created_at: payload bytes -> ` and nothing else. +# Any other part of a change (program, args, env) is joined on with "; " and +# the anchors refuse it. +_MIG_PAYLOAD_ONLY_DETAIL_RE = re.compile( + r"^.*?: payload (.+) bytes [0-9a-f?]{1,12} -> [0-9a-f?]{1,12}$") + + +def _fold_payload_update_cases(db, now): + """One-time: fold the per-job incidents about one payload's bytes into + the one case per payload that check_persistence now mints. + + Before the 2026-09-23 payload case, a rewrite of a script several jobs + run was one incident per referring plist -- #383 and #392-#397 were one + edit to aikit's run.py. The forward fix changes which key a NEW incident + is minted under and does nothing for the rows already open, so they are + folded here: the newest is re-keyed to `persistence:payload-update:` + and inherits every sibling's evidence, and the siblings close as + superseded. Folding rather than retiring, unlike the OS-program + migration, because the new case IS about the same fact -- the payload + and its bytes are in the evidence, so nothing is guessed. + + Matched on EVIDENCE, never on key shape: `changed:` is still the + live key for every change that is not payload-only. An incident folds only + when its newest evidence is a payload-only detail naming the same path as + its recorded script_target, on the plist its key names. Incidents created + in the scan that runs the migration are out of scope by construction, and + adjudicated rows are never touched. + """ + groups = {} + for row in db.execute( + "SELECT id,correlation_key FROM incidents WHERE status IN " + "('OPEN','ACK') AND correlation_key LIKE " + "'signal:persistence:changed:%' AND created_at < ? ORDER BY id", + (now,)).fetchall(): + key = _MIG_PERSIST_CHANGED_CASE_RE.match(row["correlation_key"] or "") + if not key: + continue + ev = _mig_latest_event(db, row["id"]) + head = str(ev.get("detail") or "").split("\n", 1)[0] + hit = _MIG_PAYLOAD_ONLY_DETAIL_RE.match(head) + if not hit or hit.group(1) != ev.get("script_target"): + continue + if ev.get("path") not in (None, key.group(1)): + continue + groups.setdefault(hit.group(1), []).append(row["id"]) + folded = 0 + for payload, ids in sorted(groups.items()): + folded += _fold_incidents( + db, now, "signal:persistence:payload-update:%s" % payload, ids, + "superseded: a change to a payload script's BYTES is now one case " + "per payload, not one per persistence item that runs it — one " + "edit to a shared run.py minted seven HIGH incidents. Folded into " + "the payload's case with its evidence") + return folded + + _STORE_MIGRATIONS = ( ("exec_identity_migrated", _retire_legacy_exec_incidents, "retired %d incident(s) keyed on the old positional exec identity"), @@ -6333,6 +7297,17 @@ def _retire_orphaned_behavior_incidents(db, now): ("behavior_case_identity_20260922", _retire_orphaned_behavior_incidents, "retired %d behavior incident(s) keyed on the exact command of one " "session"), + # The 2026-09-23 chain-leg rules (an interpreter never joins, an explained + # finding never triggers) are forward-only, and a chain is an event: the + # ones they would not have formed receive nothing that could close them. + ("chain_legs_20260923", _retire_unjoinable_chain_incidents, + "retired %d chain incident(s) the current join rules would not form"), + # The 2026-09-23 payload case: a change confined to a payload script's + # bytes is one case per payload, and the per-job cases minted before it + # fold into it, evidence and all. + ("persistence_payload_case_20260923", _fold_payload_update_cases, + "folded %d per-job persistence incident(s) into one case per rewritten " + "payload"), ) @@ -6357,6 +7332,76 @@ def _run_store_migrations(db, now): return ran +def _record_finding_events(db, findings, now, routing=None, folded=None): + """Record one scan's findings as signals and finding events in `db`, and + return [(event_id, finding)] for the events actually written. + + One function for the scan and for `backtest replay`, which re-records + history into a scratch store through this same fold — so the two can never + disagree about which re-observation is news. A finding folded into its + active case writes no event; when `folded` is a dict it receives + {index in findings: incident id folded into}, which is how the replay + attributes a folded finding to the case that absorbed it.""" + marks = ",".join("?" for _ in _ACTIVE_INCIDENT_STATES) + new_events = [] + for index, original in enumerate(findings): + f = _redact_value(dict(original)) + occurred = _epoch(f.get("occurred_at") or f.get("ts") or now) + attrs = _event_attributes(f) + prev = db.execute( + "SELECT severity FROM signals WHERE fingerprint=?", + (f["fingerprint"],)).fetchone() + db.execute( + "INSERT INTO signals(fingerprint,rule_id,rule_version,category," + "severity,title,detail,first_seen,last_seen,occurrence_count," + "attributes_json) VALUES(?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT(" + "fingerprint) DO UPDATE SET last_seen=excluded.last_seen," + "severity=excluded.severity,title=excluded.title,detail=excluded.detail," + "occurrence_count=signals.occurrence_count+1," + "attributes_json=excluded.attributes_json", + (f["fingerprint"], f.get("rule_id") or "aegis.legacy", + int(f.get("rule_version") or 1), f["category"], f["severity"], + f["title"], f["detail"], now, now, 1, + json.dumps(attrs, sort_keys=True))) + signal_id = db.execute("SELECT id FROM signals WHERE fingerprint=?", + (f["fingerprint"],)).fetchone()[0] + # A persisting condition is one fact with a count, not an + # event per scan. When this signal's OWN case is already an + # ACTIVE incident holding its evidence, at the same severity, + # the re-observation adds nothing an event row can say — + # occurrence_count (incremented above) and last_seen carry it. + # Live cost of the old behaviour: 41-64 byte-identical rows + # per open signal incident, feeding the 50k retention budget. + # Three deliberate bounds: a SEVERITY change still records (a + # custody re-grade is exactly the news the incident needs); a + # finding with a routing verdict still records (an allowlist + # or tolerance decision must reach the loop that closes the + # case); and only kind='signal' incidents fold — a risk + # incident's corroborating facts must stay visible in the + # accumulator's event window. + rv = (routing or {}).get(f["fingerprint"]) + if prev and prev["severity"] == f["severity"] \ + and not (rv and rv.get("decision")): + case = db.execute( + "SELECT i.id FROM incidents i JOIN incident_events ie " + "ON ie.incident_id=i.id JOIN events e ON e.id=ie.event_id " + "WHERE i.status IN (%s) AND i.kind='signal' " + "AND e.signal_id=? LIMIT 1" % marks, + _ACTIVE_INCIDENT_STATES + (signal_id,)).fetchone() + if case: + if folded is not None: + folded[index] = case[0] + continue + cur = db.execute( + "INSERT INTO events(occurred_at,observed_at,source,event_type," + "signal_id,data_json) VALUES(?,?,?,?,?,?)", + (occurred, now, f.get("sensor_id") or f["category"], + "observation.finding", signal_id, + json.dumps(f, sort_keys=True))) + new_events.append((cur.lastrowid, f)) + return new_events + + def record_security_state(findings, sensor_health=(), now=None, initially_notified=False, suppressed_categories=frozenset(), routing=None): @@ -6365,57 +7410,7 @@ def record_security_state(findings, sensor_health=(), now=None, new_events = [] try: with db: - marks = ",".join("?" for _ in _ACTIVE_INCIDENT_STATES) - for original in findings: - f = _redact_value(dict(original)) - occurred = _epoch(f.get("occurred_at") or f.get("ts") or now) - attrs = _event_attributes(f) - prev = db.execute( - "SELECT severity FROM signals WHERE fingerprint=?", - (f["fingerprint"],)).fetchone() - db.execute( - "INSERT INTO signals(fingerprint,rule_id,rule_version,category," - "severity,title,detail,first_seen,last_seen,occurrence_count," - "attributes_json) VALUES(?,?,?,?,?,?,?,?,?,?,?) ON CONFLICT(" - "fingerprint) DO UPDATE SET last_seen=excluded.last_seen," - "severity=excluded.severity,title=excluded.title,detail=excluded.detail," - "occurrence_count=signals.occurrence_count+1," - "attributes_json=excluded.attributes_json", - (f["fingerprint"], f.get("rule_id") or "aegis.legacy", - int(f.get("rule_version") or 1), f["category"], f["severity"], - f["title"], f["detail"], now, now, 1, - json.dumps(attrs, sort_keys=True))) - signal_id = db.execute("SELECT id FROM signals WHERE fingerprint=?", - (f["fingerprint"],)).fetchone()[0] - # A persisting condition is one fact with a count, not an - # event per scan. When this signal's OWN case is already an - # ACTIVE incident holding its evidence, at the same severity, - # the re-observation adds nothing an event row can say — - # occurrence_count (incremented above) and last_seen carry it. - # Live cost of the old behaviour: 41-64 byte-identical rows - # per open signal incident, feeding the 50k retention budget. - # Three deliberate bounds: a SEVERITY change still records (a - # custody re-grade is exactly the news the incident needs); a - # finding with a routing verdict still records (an allowlist - # or tolerance decision must reach the loop that closes the - # case); and only kind='signal' incidents fold — a risk - # incident's corroborating facts must stay visible in the - # accumulator's event window. - rv = (routing or {}).get(f["fingerprint"]) - if prev and prev["severity"] == f["severity"] and not (rv and rv.get("decision")) and db.execute( - "SELECT 1 FROM incidents i JOIN incident_events ie " - "ON ie.incident_id=i.id JOIN events e ON e.id=ie.event_id " - "WHERE i.status IN (%s) AND i.kind='signal' " - "AND e.signal_id=? LIMIT 1" % marks, - _ACTIVE_INCIDENT_STATES + (signal_id,)).fetchone(): - continue - cur = db.execute( - "INSERT INTO events(occurred_at,observed_at,source,event_type," - "signal_id,data_json) VALUES(?,?,?,?,?,?)", - (occurred, now, f.get("sensor_id") or f["category"], - "observation.finding", signal_id, - json.dumps(f, sort_keys=True))) - new_events.append((cur.lastrowid, f)) + new_events = _record_finding_events(db, findings, now, routing) _record_health(db, sensor_health, now) _apply_correlations(db, new_events, now, initially_notified, frozenset(suppressed_categories), routing) @@ -6455,6 +7450,10 @@ def record_security_state(findings, sensor_health=(), now=None, log_run("closed %d removed-file incident(s)" % removed) except Exception as e: log_run("removed-file close skipped: %s" % e) + try: + _rejudge_open_incidents(db, now) + except Exception as e: + log_run("re-judge skipped: %s" % e) try: global _LAST_AGED_OUT aged = _age_out_incidents(db, now) @@ -6697,6 +7696,10 @@ def claim_due_incident_reminders(now=None): rows = db.execute( "SELECT * FROM incidents WHERE status IN (%s) AND " "next_reminder_at IS NOT NULL AND next_reminder_at<=? " + # The reminder rule's enforcement point: a digest-only + # incident is never claimed, whoever armed its timer (a + # hand `reopen` does) -- see _upsert_incident. + "AND digest_only IS NULL " "ORDER BY severity DESC,next_reminder_at" % marks, _ACTIVE_INCIDENT_STATES + (now,)).fetchall() for row in rows: @@ -7895,10 +8898,128 @@ def _os_program_update(old, rec, prog_changed, return program +def _payload_update(old, rec, prog_changed, env_changed, args_changed, + target_changed, attack_defined): + """The payload path when this persistence change is confined to the bytes + of the script the job runs. + + The payload twin of _os_program_update, for the same reason. On + 2026-09-10 one edit to ~/Ai/Universe/tools/aikit/schedule/run.py + (4a1646366adf -> 25ee9a4593c2) minted seven HIGH incidents, #383 and + #392-#397 -- one per `com.aikit.*` plist that runs it, each keyed on the + referring PLIST, none of them about the plist, which was untouched. The + fact was one script's bytes; the plists were where it was seen. + + Every conjunct is load-bearing: + + confined to payload bytes a program, argv or env change alongside is + a repointed or injected job -- the attack + this sensor exists for -- and keeps its own + per-job case. + same payload PATH a job told to run a DIFFERENT script is a + config edit, not a rewrite of this one. + not attack-defined a hostile argv or a loader-injection env on + the job is never folded, so the payload's + custody can never be what quiets it. + + Unlike the OS case this makes no severity claim of its own: the payload's + custody, asked once for all the jobs, is the only thing that may demote + it. + """ + if not target_changed or attack_defined: + return None + if prog_changed or env_changed or args_changed: + return None + payload = rec.get("script_target") + if not payload or payload != old.get("script_target"): + return None + return payload + + +# Programs whose whole job is to start ANOTHER program that their argv names +# without a path: `open -b `, `powershell -enc `, +# `rundll32 x.dll,Entry`. Their own receipt says nothing about what they start. +_PERSIST_LAUNCHER_NAMES = frozenset(( + "open", "launchctl", "xargs", "nice", "chroot", "cmd", "cmd.exe", + "powershell", "powershell.exe", "pwsh", "pwsh.exe", "rundll32", + "rundll32.exe", "mshta", "mshta.exe", "wscript", "wscript.exe", "cscript", + "cscript.exe", "regsvr32", "regsvr32.exe", "msiexec", "msiexec.exe", + "explorer", "explorer.exe", "conhost", "conhost.exe", +)) + + +def _persistence_exec_extra(args, program, payload): + """Why this job's argv can run code other than its program and its + payload, or None when those two are all it runs. + + A persistence item is graded by what it EXECUTES: the program and, when + it has one, the payload script. That is a complete account only when the + argv names nothing else that runs, so the rule is stated in one place and + it fails closed -- any of these and the item earns no custody rung: + + * a URL anywhere in the argv (`uv run --from git+https://...`, a + fetched payload): the code is somewhere this cannot hash; + * a launcher wrapper in front (`caffeinate -i ...`): the plist's program + is not the thing that runs the payload; + * an interpreter or runner with no payload FILE (`python -m pkg`, + `node -e`, `npx some-package`): the code is named, not graded; + * an interpreter or runner given anything before its payload other than + its one declared subcommand (`uv run --with evil run.py`, + `uv run --directory D python -m mod`, `python -X ...`): an + interpreter option can pull in code, and a list of the harmless ones + is a list that rots, so there is no such list; + * a program that is not an interpreter, given a path (`/` or `\\`, + including `--opt=`), or one of the launchers above given any + argument: it names another thing to run. + + Arguments AFTER the payload are the payload's own input, and the payload + is graded, so they are not held against it. + """ + if not isinstance(args, list) or not args: + return None + flat = [str(a) for a in args if a is not None] + if any("://" in a for a in flat[1:]): + return "a URL in its arguments" + if _effective_argv(args, program)[0] != flat: + return "a launcher wrapper in front of what it runs" + names = {os.path.basename(str(c)).lower() for c in (program, flat[0]) if c} + if _interp_fronted(flat, program): + if not payload: + return "an interpreter that runs no script file" + try: + at = flat.index(payload, 1) + except ValueError: + return "a script target that is not one of its arguments" + before = flat[1:at] + subs = set() + for name in names: + subs.update(_RUNNER_SUBCOMMANDS.get(name) or ()) + if before and before[0] in subs: + before = before[1:] + if before: + return "interpreter options before its script (%s)" % ( + " ".join(before)[:80]) + return None + if len(flat) > 1 and names & _PERSIST_LAUNCHER_NAMES: + return "a launcher that starts what its arguments name" + for a in flat[1:]: + value = a.split("=", 1)[1] if a.startswith("-") and "=" in a else a + if "/" in value or "\\" in value: + return "a path in its arguments (%s)" % a[:80] + return None + + def check_persistence(baseline_snap, current_snap): findings = [] # (program, new_sha) -> [labels of the jobs that reference it] os_updates = {} + # (payload, new_sha) -> [(path, old, rec, severity)] of the jobs whose + # only change is that payload's bytes + payload_updates = {} + # One custody answer per program and per payload for this call: seven + # jobs sharing uv and run.py ask git and the receipts once, not seven + # times (see _custody_executes). + custody_memo = {} base = baseline_snap or {} for path, rec in current_snap.items(): if path not in base: @@ -7907,19 +9028,41 @@ def check_persistence(baseline_snap, current_snap): # by content -- so an edited copy is NOT ours and lands below. continue sev = _persistence_severity(rec) + # Graded by what it EXECUTES. Until 2026-09-23 a NEW item was + # never asked who made its program or its script, so the + # operator's own scheduler kit (#287 #288 #295 #296 #300 #302 + # #307) and a Homebrew menu-bar app (#399) interrupted HIGH on + # every sighting. Attack-defined jobs get no rung here, as + # everywhere. + attack_defined = _env_attack_defined(rec.get("env")) \ + or _hostile_args(rec.get("args"), rec.get("program")) + prov, note = (None, None) if attack_defined \ + else _custody_executes(rec, sev, custody_memo) + detail = "%s -> %s [%s]" % (rec["label"], + rec.get("program") or "?", + rec.get("trust")) + if prov and note: + detail = "%s\n%s" % (detail, note) findings.append(finding( - sev, "persistence", "New persistence item", - "%s -> %s [%s]" % (rec["label"], rec.get("program") or "?", - rec.get("trust")), + _demote(sev, prov, attack_defined=attack_defined), + "persistence", "New persistence item", detail, "persistence:new:%s:%s" % (path, rec.get("sha256")), subject=_subject( "persistence", path, op="new", content=rec.get("sha256"), program_sha=rec.get("sha256"), trust=rec.get("trust"), target=_script_target(rec.get("args"), - rec.get("program"))), + rec.get("program")), + target_sha=rec.get("target_sha")), path=path, program=rec.get("program"), trust=rec.get("trust"), script_target=_script_target(rec.get("args"), rec.get("program")), + # The bytes of both halves of what the job executes, as the + # snapshot hashed them -- the pair a CHANGED diff compares. + # The recorded evidence named neither, so a NEW finding read + # back from the store could not say which producer it was. + program_sha=rec.get("sha256"), + target_sha=rec.get("target_sha"), + custody=prov, run_at_load=rec.get("run_at_load"))) else: old = base[path] @@ -7971,6 +9114,17 @@ def check_persistence(baseline_snap, current_snap): # payload can never be quieted by proving who moved it. attack_defined = _env_attack_defined(rec.get("env")) \ or _hostile_args(rec.get("args"), rec.get("program")) + # One payload rewritten is ONE fact, however many jobs run + # it: collected here, graded once after the loop, keyed on + # the payload. See _payload_update. + payload = _payload_update( + old, rec, prog_changed, env_changed, args_changed, + target_changed, attack_defined) + if payload: + payload_updates.setdefault( + (payload, rec.get("target_sha")), []).append( + (path, old, rec, sev)) + continue prov = (None if attack_defined else _custody_persistence(old, rec) or (_custody(path, rec.get("sha256"))[0] @@ -7980,8 +9134,11 @@ def check_persistence(baseline_snap, current_snap): # so grading only `path` can never see who did it and every # payload update reads as a swap. Ask the ledger about the # payload that actually changed. Same rung, same demote-only - # ladder, same attack-defined refusal above. - if prov is None and target_changed: + # ladder, same attack-defined refusal above -- asked here too, + # or a DYLD-injected job whose payload is committed would + # record custody=self-committed on an attack-defined finding + # that the grader downstream cannot tell apart. + if prov is None and target_changed and not attack_defined: tgt = rec.get("script_target") or _script_target( rec.get("args"), rec.get("program")) if tgt and rec.get("target_sha") and _intent_worthy(tgt): @@ -8042,6 +9199,42 @@ def check_persistence(baseline_snap, current_snap): path=program, program=program, trust="apple", custody="os-vendor", referrer_count=len(labels), referrers=sorted(labels))) + # One finding per rewritten payload, carrying every job that runs it, so + # the operator adjudicates the edit once. Custody is asked ONCE, about + # the payload: seven jobs sharing run.py asked git seven times a scan, + # and every probe that timed out re-minted a case at HIGH. Severity is + # the worst any job scores (each already floored at HIGH, exactly as a + # single job's is) before the payload's rung demotes it. The signal key + # carries the new sha, so the NEXT edit is a new finding rather than a + # silenced recurrence. + for (payload, sha), jobs in sorted(payload_updates.items()): + jobs.sort(key=lambda job: job[0]) + worst = max(jobs, key=lambda job: SEV_ORDER[job[3]]) + key = ("payload", payload, sha) + if key not in custody_memo: + custody_memo[key] = _custody_payload(payload, sha) + prov, note = custody_memo[key] + labels = sorted(str(rec.get("label") or path) + for path, _old, rec, _sev in jobs) + olds = sorted({(old.get("target_sha") or "?")[:12] + for _path, old, _rec, _sev in jobs}) + detail = ("payload %s bytes %s -> %s; %d persistence item(s) run it: " + "%s" % (payload, ",".join(olds), sha[:12], len(labels), + ", ".join(labels[:12]) + + (" …" if len(labels) > 12 else ""))) + if note: + detail = "%s\n%s" % (detail, note) + findings.append(finding( + _demote(worst[3], prov), "persistence", + "Persistence item CHANGED", detail, + "persistence:payload-update:%s:%s" % (payload, sha), + case_fingerprint="persistence:payload-update:%s" % payload, + subject=_subject("persistence", payload, content=sha), + path=payload, program=worst[2].get("program"), + trust=worst[2].get("trust"), custody=prov, + script_target=payload, target_sha=sha, + referrer_count=len(jobs), referrers=labels, + referrer_paths=[path for path, _old, _rec, _sev in jobs])) return findings @@ -8455,14 +9648,9 @@ def check_processes(): warm_signature_cache([c for _p, _o, c, _a in procs if c and not _is_trusted_prefix(c)]) # Who started each process, for the one rung that asks (_supervised_rung). - # The ancestry table is a second read of the process table, so it is built - # at most once per scan, on the first finding that could use it, and never - # on a host with no vouch to earn that rung from -- the common case, which - # pays nothing. A tampered vouch store earns nothing either, so it is - # skipped the same way. - exe_by_pid = {str(p): c for p, _o, c, _a in procs if c} - vouched_any = bool(load_vouches()[0]) - table = None + # Built lazily and at most once per scan, never on a host with no vouch + # (_parents_by_pid). + parents_of = _parents_by_pid({str(p): c for p, _o, c, _a in procs if c}) for pid, _owner, comm, _argv in procs: if not comm: continue @@ -8499,15 +9687,7 @@ def check_processes(): # later reusing the same path is a new finding (and not silently # covered by an allowlist entry made for the earlier one). sha = sha256(comm) - parents = [] - if vouched_any: - if table is None: - table = _process_ancestry_table() - # "?" for an ancestor with no exe path this user can read: it - # is still a link in the chain, and _supervised_rung ends the - # walk there rather than stepping over it. - parents = [exe_by_pid.get(p) or "?" - for p in _ancestry(pid, table)] + parents = parents_of(pid) graded, rung, note = _grade_binary(sev, comm, sha=sha, parents=parents) f = finding( @@ -8530,7 +9710,8 @@ def check_processes(): else "process:%s" % _program_subject(comm)), subject=_subject("process", comm, trust=sig["trust"], content=sha), - path=comm, trust=sig["trust"], sha256=sha, custody=rung) + path=comm, trust=sig["trust"], sha256=sha, custody=rung, + **_class_facts(comm, sig["trust"], rung, parents)) if parents: # The programs it ran under, by exe path -- the operator's # answer to "why supervised?", and to why not. Paths rather @@ -8719,11 +9900,119 @@ def _obfuscated_payload_signals(argv, idioms): return out +# The agent harness's own command wrapper. Claude Code runs every command the +# agent issues as ONE `-c` string (live `ps`, reference Mac, 2026-09-23): +# +# /bin/bash -c source ~/.claude/shell-snapshots/snapshot-bash-.sh +# 2>/dev/null || true && shopt -u extglob 2>/dev/null || true && +# { \builtin unalias -- 'unsetenv'; \builtin unset -f -- 'unsetenv'; } +# >/dev/null 2>&1 || true && eval '' < /dev/null && +# pwd -P >| /tmp/claude--cwd +# +# `eval-subshell` is `eval … $(`, so ANY with a command substitution +# matched on the harness's `eval` (incident #538: a `while … s=$(gh pr checks +# …)` poll loop), and that same `eval` was the exec half that turned a benign +# value capture `v=$(curl -s https://…)` into fileless-fetch-exec at HIGH. +# The sensor was judging the harness, not the command. +# +# So the wrapper is unwrapped and is judged — with the full ruleset, +# exactly as the same line typed at a prompt. This is NOT trust of agent +# hosts: a prompt-injected agent running `curl | bash` is what the sensor +# exists for, and it still fires. What makes it safe is that the wrapper is +# matched against a FIXED grammar: every character outside the payload must +# be one of these clauses, with no shell metacharacter in any free field (a +# snapshot path or cwd file that could carry `;curl${IFS}…` never matches). +# Anything the harness does not write means it is not the harness, and the +# whole argv is judged as before. Only the shapes in the recorded corpus are +# known: the Claude Code prologue with and without the unalias clause. Codex +# runs `bash -lc ` with no eval to unwrap; no Hermes argv is on record. +_HARNESS_SHELL_RE = re.compile(r"(?:[\w./+-]*/)?(?:bash|zsh|sh) -c ") +_HARNESS_SNAPSHOT_RE = re.compile( + r"source [\w./+-]*/shell-snapshots/snapshot-(?:bash|zsh|sh)-[\w-]+\.sh" + r" 2>/dev/null \|\| true && ") +_HARNESS_OPTIONAL_CLAUSE_RES = ( + re.compile(r"shopt -u extglob 2>/dev/null \|\| true && "), + re.compile(r"\{ \\builtin unalias -- '[\w.:-]+'; \\builtin unset -f -- " + r"'[\w.:-]+'; \} >/dev/null(?: 2>&1)?(?: \|\| true)? && "), +) +_HARNESS_EPILOGUE_RE = re.compile( + r"(?: < /dev/null)?(?: && pwd -P >\| [\w./+-]*/claude-[0-9a-f]+-cwd)?\s*") +_HARNESS_WRAPPER = "claude-code-snapshot" + + +def _shell_single_quoted(text, i): + """(value, end) of the shell word at text[i] built from single-quoted runs + joined by the two escapes a quoter emits for an embedded quote — `'"'"'` + (Claude Code) and `'\\''` — or None when a run never closes. The word ends + at the first character that is neither; the caller decides whether what + follows is allowed.""" + out = [] + n = len(text) + while i < n and text[i] == "'": + j = text.find("'", i + 1) + if j < 0: + return None + out.append(text[i + 1:j]) + i = j + 1 + if text.startswith("\"'\"", i): + out.append("'") + i += 3 + elif text.startswith("\\'", i): + out.append("'") + i += 2 + return "".join(out), i + + +def _unwrap_agent_harness(argv): + """ when argv is exactly the Claude Code wrapper around it, else + None. One layer; see _agent_harness_payload.""" + m = _HARNESS_SHELL_RE.match(argv) + if not m: + return None + m = _HARNESS_SNAPSHOT_RE.match(argv, m.end()) + if not m: + return None + pos = m.end() + for rx in _HARNESS_OPTIONAL_CLAUSE_RES: + m = rx.match(argv, pos) + while m: + pos = m.end() + m = rx.match(argv, pos) + if not argv.startswith("eval '", pos): + return None + word = _shell_single_quoted(argv, pos + len("eval ")) + if word is None or not _HARNESS_EPILOGUE_RE.fullmatch(argv, word[1]): + return None + return word[0] + + +def _agent_harness_payload(argv): + """(text to judge, wrapper name or None). + + Unwrapped to a fixed point, so every helper that judges or displays an + argv — _argv_signals, _argv_match_spans, _argv_evidence_preview, + _argv_case_identity — sees the same text whichever of them runs first, + even for a payload that is itself a harness line. Terminates: each layer + is strictly shorter than the one around it.""" + if not argv or "shell-snapshots/snapshot-" not in argv: + return argv, None + wrapper = None + while True: + payload = _unwrap_agent_harness(argv) + if payload is None: + return argv, wrapper + argv, wrapper = payload, _HARNESS_WRAPPER + + def _argv_signals(argv): """Return [(name, severity)] for hostile patterns in a live process's argv (empty = clean). Structural signals keep their assigned severity; the shared shell idioms notify (HIGH) only as a fetch+exec COMBINATION, else stay MEDIUM; - anti-VM gates are MEDIUM corroborators below the notify floor.""" + anti-VM gates are MEDIUM corroborators below the notify floor. + + An agent-harness wrapper is unwrapped first and the command inside it is + what gets judged (_agent_harness_payload).""" + argv = _agent_harness_payload(argv)[0] if not argv: return [] best = {} @@ -8778,7 +10067,11 @@ def _argv_match_spans(argv): Re-runs the same regexes `_argv_signals` ran and keeps the spans it throws away. Deliberately a second pass rather than a changed return contract: this runs only for an argv that ALREADY matched — a rare path — while - `_argv_signals` is called for every watched process on the box.""" + `_argv_signals` is called for every watched process on the box. + + Same text `_argv_signals` judged, so for a harness-wrapped argv the spans + index the unwrapped payload (_agent_harness_payload), not the argv.""" + argv = _agent_harness_payload(argv)[0] or "" spans = [] for rx, _name, _sev in _HOSTILE_ARGV_RES: m = rx.search(argv) @@ -8829,8 +10122,13 @@ def _argv_case_identity(argv, length=16): acquired tolerance needs, and the same command shape opened a fresh HIGH incident forever — #450 and #503 are the same finding twice, minted eight days apart under two nonces. + + A harness-wrapped argv is keyed on the command inside the wrapper + (_agent_harness_payload), so a verdict attaches to what ran — the + wrapper also carries a per-command cwd-file nonce the rules above do not + name, which made every wrapped command its own case. """ - flat = argv or "" + flat = _agent_harness_payload(argv)[0] or "" for rx, repl in _ARGV_NONCE_RES: flat = rx.sub(repl, flat) flat = _program_subject(flat) @@ -8857,7 +10155,11 @@ def _argv_evidence_preview(argv, budget=_ARGV_PREVIEW_BUDGET): the rule it already followed — only the hostile verdict and the evidence that earned it are written, and everything still passes through `redact_sensitive` first. + + A harness-wrapped argv previews the command inside the wrapper — what + was judged, and what ran; the finding names the wrapper separately. """ + argv = _agent_harness_payload(argv)[0] flat = re.sub(r"\s+", " ", argv or "").strip() if not flat: return "" @@ -8963,16 +10265,22 @@ def check_behavior(): # its command line the same way. The fingerprint stays on the hash so # identity does not move when the redaction regexes do. preview = _argv_evidence_preview(argv) + # A harness-wrapped command is judged, previewed and keyed on the + # command inside the wrapper; the wrapper itself is not evidence, but + # the fact that it was there is, so the operator is told. + wrapper = _agent_harness_payload(argv)[1] + extra = {"wrapper": wrapper} if wrapper else {} findings.append(finding( top, "behavior", "Suspicious process behavior", - "%s triggered [%s]; command sha256=%s; command: %s" % - (base, names, command_sha[:16], preview), + "%s triggered [%s]; command sha256=%s; %scommand: %s" % + (base, names, command_sha[:16], + "wrapper: %s; " % wrapper if wrapper else "", preview), fp, case_fingerprint="behavior:%s:%s:%s" % ( base, "|".join(sorted(n for n, _ in signals)), _argv_case_identity(argv)), program=argv.split(None, 1)[0] if argv else "", pid=pid, markers=[n for n, _ in signals], command_sha256=command_sha, - command_preview=preview)) + command_preview=preview, **extra)) _annotate_ancestry(findings) return findings @@ -9334,7 +10642,13 @@ def _check_hot_app(path, st, cutoff): # "Verify you built/trust it" is a question custody can often answer # outright: an app this machine built, or a copy of one, is the ordinary # reading of signed-but-un-notarized on a developer's Mac. - graded, rung, rung_note = _grade_binary("MEDIUM", exe, sha=sha) + # + # Gatekeeper just refused this bundle, and Gatekeeper's notarization check + # is the control the publisher rung leans on: the platform said no, so the + # signature earns nothing here (publisher_ok=False). The same bytes, once + # notarized, earn it from every other sensor. + graded, rung, rung_note = _grade_binary("MEDIUM", exe, sha=sha, + publisher_ok=False) return [finding( graded, "hot-dir", "Un-notarized app in watched folder", "%s [%s] is signed but NOT notarized (Gatekeeper: %s%s), modified %s — " @@ -12683,9 +13997,11 @@ def _outbound_candidate_trust(path): return trust -def _outbound_findings(rows): +def _outbound_findings(rows, ancestry=None): """One MEDIUM/medium-confidence finding per PROGRAM holding live outbound - sockets, over `rows` of (path, remote_ip, remote_port). + sockets, over `rows` of (path, remote_ip, remote_port). `ancestry`, when + the caller has one, is (path, ip, port) -> the ancestor exe list of the + process holding that socket, for the `supervised` rung. Below the notify floor on purpose (ad-hoc dev binaries talk to the network routinely — must not page alone): logged, rendered, and fed to correlation. @@ -12709,8 +14025,10 @@ def _outbound_findings(rows): for path, rip, rport, trust in sorted(by_subject[subject]): endpoint = "%s:%s" % (rip, rport) endpoints.append(endpoint) + parents = ancestry(path, rip, rport) if ancestry else [] graded, rung, note = _grade_binary("MEDIUM", path, - endpoint=endpoint) + endpoint=endpoint, + parents=parents) dev_case, dev_note = _vouch_endpoint_deviation(path, endpoint) # Rank: severity first, then a vouch deviation (the fact the # operator must actually adjudicate), then an ungraded rung — all @@ -12719,9 +14037,9 @@ def _outbound_findings(rows): 0 if rung else 1) if worst is None or rank > worst[0]: worst = (rank, path, rip, rport, trust, graded, rung, note, - dev_case, dev_note) - _r, path, rip, rport, trust, graded, rung, note, dev_case, dev_note \ - = worst + dev_case, dev_note, parents) + _r, path, rip, rport, trust, graded, rung, note, dev_case, dev_note, \ + parents = worst if dev_note: note = (note + "\n" + dev_note) if note else dev_note shown = endpoints[:_OUTBOUND_DETAIL_ENDPOINTS] @@ -12743,7 +14061,8 @@ def _outbound_findings(rows): # own attribute rather than a replacement. remote=rip, port=rport, endpoints=endpoints, endpoint_count=len(endpoints), trust=trust, confidence="medium", - custody=rung, markers=["outbound-exfil"])) + custody=rung, markers=["outbound-exfil"], + **({"ancestry": parents} if parents else {}))) return findings @@ -12786,7 +14105,9 @@ def _decode_proc_hex_addr(addr_hex): def _outbound_rows(): - """[(path, remote_ip, remote_port)] of live outbound TCP, per platform.""" + """[(path, remote_ip, remote_port, pid)] of live outbound TCP, per + platform. The pid is the one the row was attributed from; it is kept so + the grader can ask who STARTED the program holding the socket.""" if IS_LINUX: rows = [] for pf in ("/proc/net/tcp", "/proc/net/tcp6"): @@ -12809,7 +14130,7 @@ def _outbound_rows(): unexamined("pid %s -> %s:%s" % (pid, rip, rport), "its executable could not be read", e) continue - out.append((path, rip, rport)) + out.append((path, rip, rport, pid)) return out if IS_WIN: text, rc = _netstat_tcp_rows() @@ -12829,7 +14150,7 @@ def _outbound_rows(): continue path = pid_exe.get(parts[4]) if path: - out.append((path, host, port)) + out.append((path, host, port, parts[4])) return out text, _, rc = run(NETSTAT_CMD, timeout=15) if rc in (124, 127) or not text: @@ -12851,7 +14172,7 @@ def _outbound_rows(): comm_cache[pid] = pout.strip() if prc == 0 else "" comm = comm_cache[pid] if comm: - out.append((comm, rip, rport)) + out.append((comm, rip, rport, pid)) return out @@ -12868,11 +14189,18 @@ def check_outbound(): seen = set() snap_rows = [] generic_rows = [] - for path, rip, rport in _outbound_rows(): + pid_of = {} + for row in _outbound_rows(): + path, rip, rport = row[:3] key = "%s:%s:%s" % (path, rip, rport) if key in seen: continue seen.add(key) + # Kept beside the rows, never in them: the stored beacon history is + # (path, ip, port, trust), and a pid would make every restart a new + # row in a snapshot whose whole detection is that nothing moved. + pid_of[(str(path), str(rip), str(rport))] = \ + row[3] if len(row) > 3 else None # Trust captured at observation time (sigcache makes the re-ask free) # so the stored rows grade without re-classifying long-gone binaries. resolvable = path.startswith("/") or (IS_WIN and ":" in path[:3]) @@ -12888,11 +14216,18 @@ def check_outbound(): findings.append(intel) else: generic_rows.append((path, rip, rport)) - findings += _outbound_findings(generic_rows) + # Who started the program holding each socket, asked only of a row that + # reaches the grader and never on a host with no vouch (_parents_by_pid). + parents_of = _parents_by_pid() + + def ancestry(path, rip, rport): + return parents_of(pid_of.get((str(path), str(rip), str(rport)))) + findings += _outbound_findings(generic_rows, ancestry) if snap_rows: record_observation(BEACON_SENSOR_ID, sorted(snap_rows)) findings += _beacon_recurrence( - _load_observations(BEACON_SENSOR_ID, BEACON_WINDOW_DAYS), snap_rows) + _load_observations(BEACON_SENSOR_ID, BEACON_WINDOW_DAYS), snap_rows, + ancestry) return findings @@ -12915,7 +14250,7 @@ def check_outbound(): r"(?:\.exe)?(?: Helper(?: \([^/\\]*\))?)?$", re.I) -def _beacon_recurrence(history, current_rows): +def _beacon_recurrence(history, current_rows, ancestry=None): """HIGH findings for the beacon residue shape: a (path, remote ip:port) pair live in THIS scan and already observed in >= BEACON_MIN_SCANS distinct stored scans spanning >= BEACON_MIN_SPAN_SECS, from a non-browser binary @@ -12932,7 +14267,7 @@ def _beacon_recurrence(history, current_rows): sightings = {} for ts, rows in history: _beacon_add_sighting(sightings, ts, rows) - return _beacon_from_sightings(sightings, current_rows) + return _beacon_from_sightings(sightings, current_rows, ancestry) def _beacon_add_sighting(sightings, ts, rows): @@ -13005,8 +14340,14 @@ def _beacon_dispersion(sightings): return disp -def _beacon_from_sightings(sightings, current_rows): - """The recurrence DECISION, over an already-built sightings map.""" +def _beacon_from_sightings(sightings, current_rows, ancestry=None): + """The recurrence DECISION, over an already-built sightings map. + + `ancestry`, when the caller has one (check_outbound; never `rehunt`, + whose stored rows carry no pid), is (path, ip, port) -> the ancestor exe + list of the process holding that socket. It is what the `supervised` + rung reads, and it is recorded on the finding as `ancestry` so the + replay can re-derive the grade (#352).""" findings = [] rows = sorted(set(tuple(r) for r in current_rows)) dispersion = _beacon_dispersion(sightings) @@ -13031,7 +14372,10 @@ def _beacon_from_sightings(sightings, current_rows): if not (suspicious_sig(trust) or is_risky_location(path)): continue endpoint = "%s:%s" % (rip, rport) - graded, rung, note = _grade_binary("HIGH", path, endpoint=endpoint) + parents = ancestry(path, rip, rport) if ancestry else [] + graded, rung, note = _grade_binary("HIGH", path, endpoint=endpoint, + parents=parents) + lineage = {"ancestry": parents} if parents else {} dev_case, dev_note = _vouch_endpoint_deviation(path, endpoint) if dev_note: note = (note + "\n" + dev_note) if note else dev_note @@ -13065,10 +14409,12 @@ def _beacon_from_sightings(sightings, current_rows): "beacon:rotating:%s:%s" % (_program_subject(path), rport), case_fingerprint=dev_case or ( "beacon:rotating:%s:%s" % (_program_subject(path), rport)), - subject=_subject("beacon", path, port=rport), + subject=_subject("beacon", path, port=rport, + content=_graded_sha(path)), path=path, program=path, port=rport, trust=trust, endpoint_count=len(fleet), endpoints=shown, - custody=rung, markers=["outbound-exfil", "beacon"])) + custody=rung, markers=["outbound-exfil", "beacon"], + **dict(_class_facts(path, trust, rung, parents), **lineage))) continue findings.append(finding( graded, "net-beacon", @@ -13088,11 +14434,13 @@ def _beacon_from_sightings(sightings, current_rows): "beacon:%s:%s:%s" % (_program_subject(path), rip, rport), case_fingerprint=dev_case or ("beacon:%s:%s:%s" % ( _program_subject(path), rip, rport)), - subject=_subject("beacon", path, ip=rip, port=rport), + subject=_subject("beacon", path, ip=rip, port=rport, + content=_graded_sha(path)), path=path, program=path, remote=rip, port=rport, trust=trust, scan_count=len(stamps), span_secs=span, custody=rung, - markers=["outbound-exfil", "beacon"])) + markers=["outbound-exfil", "beacon"], + **dict(_class_facts(path, trust, rung, parents), **lineage))) return findings @@ -14333,8 +15681,11 @@ def diff_agent_skills(prior, cur): # _accumulate_risk, and 'agent-skill' is in no correlate() rule — the # durable record is real, but the "auto-correlates with a later osascript # phish" chain is not yet wired. The phish itself still fires CRITICAL alone. - def _graded(key, base_sev, base_conf): - """(severity, confidence, path, markers, why) for one skill. + deadline = time.monotonic() + _SKILL_CUSTODY_BUDGET + + def _graded(key, base_sev, base_conf, sig, old_sig=None): + """(severity, confidence, path, markers, why, provenance) for one + skill. Until 2026-09-03 both tiers were hardcoded MEDIUM, which is BELOW NOTIFY_MIN_SEV — so a skill the operator did not author could appear in @@ -14343,10 +15694,20 @@ def _graded(key, base_sev, base_conf): risk accumulation, which the old comment here admitted. Reading the skill's own instructions fixes both: concealment is attack-defined (nothing legitimate tells an agent to hide what it did), and - credential+egress together is the stealer shape.""" + credential+egress together is the stealer shape. + + Custody grades the CHANGE and never the content, the same split + diff_agent_surface makes: it is asked only when the instructions carry + no directive at all. A skill whose text names a secret, a channel or + concealment is judged on that text whoever committed it.""" d = _AGENT_SKILL_DIRS.get(key) marks = _skill_instruction_markers(d) - sev, conf, why = base_sev, base_conf, "" + sev, conf, why, prov = base_sev, base_conf, "", None + if not marks: + prov, note = _skill_custody(d, sig, old_sig, deadline) + sev = _demote(sev, prov) + if note: + why = " " + note if "conceal" in marks: sev, conf = "HIGH", "high" why = (" Its instructions tell the agent to CONCEAL its actions — " @@ -14368,20 +15729,24 @@ def _graded(key, base_sev, base_conf): "legitimate skills mention both.") elif marks: why = (" Its instructions mention: %s." % ", ".join(marks)) - return sev, conf, d, ["agent-skill"] + ["imperative:" + m for m in marks], why + markers = ["agent-skill"] + ["imperative:" + m for m in marks] + if prov in _SELF_CUSTODY: + markers.append("self-custody") + return sev, conf, d, markers, why, prov def new_fn(key, sig): - sev, conf, path, marks, why = _graded(key, "MEDIUM", "medium") + sev, conf, path, marks, why, prov = _graded(key, "MEDIUM", "medium", sig) return finding( sev, "agent-skill", "New AI-agent skill installed", "%s appeared — AI-agent skills run with your full privileges and are " "a live 2026 stealer channel (a malicious SKILL.md can drive a fake " "password dialog). Verify you installed it.%s" % (key, why), "agent-skill:new:%s" % key, skill=key, confidence=conf, - path=path, markers=marks) + path=path, provenance=prov, markers=marks) def changed_fn(key, sig, old): - sev, conf, path, marks, why = _graded(key, "MEDIUM", "low") + sev, conf, path, marks, why, prov = _graded(key, "MEDIUM", "low", + sig, old) return finding( sev, "agent-skill", "AI-agent skill changed", "%s was modified — its SKILL.md or a shipped script changed. Routine " @@ -14389,11 +15754,117 @@ def changed_fn(key, sig, old): "hijack.%s" % (key, why), "agent-skill:changed:%s:%s" % (key, hashlib.sha256(sig.encode()).hexdigest()[:12]), - skill=key, confidence=conf, path=path, markers=marks) + skill=key, confidence=conf, path=path, provenance=prov, + markers=marks) return _diff_map(prior, cur, new_fn, changed_fn) +# A skill whose change touches more files than this is not graded at all: a +# bulk rewrite is not the shape custody was built to explain, and asking git +# about each file every scan would cost more than the answer is worth. +_SKILL_CUSTODY_FILES = 8 + +# Custody is asked on EVERY scan for every skill that differs from the +# baseline, and a changed skill is never re-baselined until the operator +# accepts it (the anti-laundering rule). Measured on the reference machine the +# day this was wired: 113 changed and 7 new skills out of 168, 236 files, 37 s +# of git per scan -- for answers that change only when the bytes do. +# +# So the answer is kept, keyed on the bytes' real path and content hash, and +# bounded three ways. Only an ANSWER is kept: _git_provenance returns None +# both for "in no repository" and for a git that did not answer in time, and +# a timeout remembered as a verdict is the defect the 2026-09-22 batch +# removed, so a None is asked again next scan. An entry expires after +# _SKILL_CUSTODY_TTL, so an aged-out intent record or an expired reflog is +# seen within hours rather than for the life of the watch daemon. And new +# bytes are a new key. What goes stale in between is an untracked file later +# committed, which reads at its OLD, weaker rung -- toward suspicion. +_SKILL_CUSTODY_MEMO = {} +_SKILL_CUSTODY_TTL = 6 * 3600 +_SKILL_CUSTODY_MEMO_CAP = 4096 +# Seconds one diff may spend asking git. A skill it does not reach stays +# ungraded at full severity (the fail-toward-suspicion outcome), and the memo +# lets the next scan continue from there. +_SKILL_CUSTODY_BUDGET = 10.0 + + +def _skill_file_custody(path, deadline=None): + """(rung, note) for one skill file: from the memo while fresh, else asked + at its real path -- or (None, "") once `deadline` has passed.""" + sha = sha256(path) + key = (os.path.realpath(path), sha) + now = time.time() + hit = _SKILL_CUSTODY_MEMO.get(key) + if hit and now - hit[0] < _SKILL_CUSTODY_TTL: + return hit[1], hit[2] + if deadline is not None and time.monotonic() > deadline: + return None, "" + rung, note = _bytes_custody(path, sha) + if rung is not None: + if len(_SKILL_CUSTODY_MEMO) >= _SKILL_CUSTODY_MEMO_CAP: + _SKILL_CUSTODY_MEMO.clear() + _SKILL_CUSTODY_MEMO[key] = (now, rung, note) + return rung, note + + +def _skill_sig_parts(sig): + """{file name: content hash} from a _skill_signature string, or {} when it + is not one (a test placeholder, or a format this reader predates).""" + out = {} + for part in str(sig or "").split("|"): + name, sep, h = part.partition("=") + if not sep: + continue + if name == "exec": + for item in h.split(","): + n, _at, hh = item.partition("@") + if n: + out[n] = hh + else: + out[name] = h + return out + + +def _skill_custody(skill_dir, sig, old_sig=None, deadline=None): + """(rung, note) for the files that CHANGED in one agent skill, asked where + they live, and graded by the WEAKEST of them. + + Where they live: ~/.codex/skills/ are symlinks into the operator's + skills repositories, and git answers nothing about a path under a symlink, + so a committed skill change graded as authorless (#329-#332). + + The weakest, and only the files that changed: a committed SKILL.md says + nothing about a shipped script swapped underneath it, so a skill is only + as vouched-for as its least vouched-for changed file, and a file the + signature names but the disk no longer holds is no answer at all.""" + if not skill_dir: + return None, "" + new, old = _skill_sig_parts(sig), _skill_sig_parts(old_sig) + changed = sorted(n for n, h in new.items() if old.get(n) != h) + if not changed or len(changed) > _SKILL_CUSTODY_FILES: + return None, "" + worst, seen = None, set() + for name in changed: + p = os.path.join(skill_dir, name) + try: + st = os.stat(p) + except OSError: + return None, "" + # One file, asked once: on a case-insensitive volume the signature + # lists SKILL.md and skill.md for the same inode, and git, asked about + # the spelling it does not track, answers nothing. + if (st.st_dev, st.st_ino) in seen: + continue + seen.add((st.st_dev, st.st_ino)) + rung, note = _skill_file_custody(p, deadline) + if worst is None or _custody_strength(rung) < _custody_strength(worst[0]): + worst = (rung, note) + if not _custody_strength(rung): + break + return worst + + # --- Timestomp detection (T1070.006) ----------------------------------------- def timestomp_signal(path, st=None): """A reason string if a file's timestamps look tampered, else None. `touch`/ @@ -15134,6 +16605,19 @@ def _credential_surface_present(): return found +# Path components in CREDENTIAL_SURFACE that name a CONTAINER every tool uses, +# not a secret: `.kube/config` contributed "config", `.docker/config.json` +# "config.json", every XDG entry ".config", and `.mozilla/firefox` "firefox". +# As substrings of instruction text those matched "configure", +# "configuration", any ~/.config path and the browser's name, so on the +# reference machine the credential marker fired on nearly every instruction +# file, the operator's own included. Each entry stays reachable through its +# distinctive component (.kube, .docker, bitwarden, hosts.yml, .mozilla, +# cookies.sqlite), so this table loses no secret it could name. +_CREDENTIAL_CONTAINER_WORDS = frozenset(("config", "config.json", ".config", + "firefox")) + + def _credential_path_tokens(): """Distinctive path fragments used by the instruction-file imperative detector to decide whether a line NAMES a secret. Derived from the one @@ -15141,10 +16625,11 @@ def _credential_path_tokens(): toks = set() for rel, _, _, _ in CREDENTIAL_SURFACE: base = rel.split("/")[-1] - if len(base) > 3: + if len(base) > 3 and base.lower() not in _CREDENTIAL_CONTAINER_WORDS: toks.add(base.lower()) first = rel.split("/")[0] - if first.startswith(".") and len(first) > 3: + if first.startswith(".") and len(first) > 3 and \ + first.lower() not in _CREDENTIAL_CONTAINER_WORDS: toks.add(first.lower()) toks.update({"id_rsa", "id_ed25519", "credentials", "secret", "api_key", "api-key", "access_token", "private key", ".env", "keychain"}) @@ -15286,10 +16771,22 @@ def _credential_path_tokens(): # "Do not tell the user TO RUN `codex plugin marketplace add`" is guidance # about what to recommend, not an instruction to deceive. Concealment # continues with "about/that/of"; advice continues with "to ". + # + # The lookahead used to require a WORD character after "to", and was found + # wanting on a second real file (canvas-lms, 2026-09-04, still firing HIGH + # on every scan three weeks later): 'Never tell the user to "generate a new + # token"' quotes the verb, and a quote is not \w. "Tell someone to" is an + # instruction to direct them, whatever punctuation follows; nothing that + # conceals is phrased that way. r"\b(?:do\s*not|don'?t|never)\s+(?:tell|inform|notify|alert|warn|mention\s+" r"(?:this|it)\s+to)\s+(?:the\s+)?(?:user|operator|owner|human|dev(?:eloper)?)" - r"\b(?!\s+to\s+\w)", - r"\bwithout\s+(?:telling|informing|notifying|alerting|asking)\s+" + r"\b(?!\s+to\b)", + # Not "asking": acting without ASKING is autonomy, not concealment -- the + # person is not kept in the dark, only not consulted. It fired on the + # operator's own gsd-discuss-phase skill ("downstream agents can act + # without asking the user again"), and the pattern this table exists for, + # doing something the operator never learns of, is "without telling". + r"\bwithout\s+(?:telling|informing|notifying|alerting)\s+" r"(?:the\s+)?(?:user|operator|owner|human|dev(?:eloper)?)\b", r"\b(?:hide|conceal|suppress)\s+(?:\w+\s+){0,3}from\s+(?:the\s+)?" r"(?:user|operator|owner|human|dev(?:eloper)?)\b", @@ -15301,32 +16798,88 @@ def _credential_path_tokens(): )) _IMPERATIVE_EGRESS = tuple(re.compile(p, re.I) for p in ( + # An egress DIRECTIVE names what leaves or where it goes. The object used + # to be optional, so "send to" matched as a NOUN -- "drive the gated real + # send to the original sender", a row of the operator's own mail tool's + # AGENTS.md (#335). Without an object, only an address (a URL or a mail + # address) makes it a directive. r"\b(?:send|post|upload|transmit|exfiltrate|forward|email)\s+" - r"(?:it|them|this|the\s+\w+|contents?)?\s*to\b", + r"(?:(?:it|them|this|the\s+\w+|contents?)\s+to\b|" + r"to\s+(?:https?://|[\w.+-]+@[\w-]+\.\w))", r"\binclude\s+(?:it|them|the\s+\w+|the\s+contents?)?\s*in\s+" r"(?:your|the)\s+(?:next\s+)?(?:commit|message|response|reply|answer|PR|" r"pull\s*request)\b", r"\bcommit\s+(?:it|them|the\s+\w+)\s+to\b", r"\bcurl\s+-[A-Za-z]*[dF]\b", - r"https?://(?!(?:localhost|127\.0\.0\.1|github\.com|gitlab\.com|" - r"docs\.\w+|developer\.\w+))[\w.-]+\.[a-z]{2,}/\S*", + # A URL is a DESTINATION only when something is sent to it. Bare, it is a + # citation: the rule matched every documentation link in every instruction + # file (#336 was a markdown link to a vendor's structured-outputs docs), + # and a config line's API base. A transmit verb earlier on the same line + # is what turns a reference into a channel. + r"\b(?:send|post|upload|transmit|exfiltrate|forward|email|submit|beacon)" + r"\b[^\n]{0,120}?https?://(?!(?:localhost|127\.0\.0\.1|github\.com|" + r"gitlab\.com|docs\.\w+|developer\.\w+))[\w.-]+\.[a-z]{2,}/\S*", )) +# One unit of instruction text: a paragraph, a list item, a table row or a +# heading. Wrapped prose lines join the unit they continue. +_DIRECTIVE_BREAK_RE = re.compile( + r"\n[ \t]*\n|\n(?=[ \t]*(?:[-*+][ \t]|\d+[.)][ \t]|[|#>]))") + +# The verbs that ACCESS a file. A secret after one of these, in the same unit, +# is the verb's object: something the agent is told to read, load or copy. +# Base forms only, so the third person of documentation ("the tool reads its +# key from the keychain") describes and does not direct. The second row is the +# transmit verbs that take a FILE as their object -- "Upload ~/.aws/ +# credentials at https://..." reads the secret by sending it, and was HIGH on +# main. `post` and `email` are left out: before a token they are usually +# nouns ("the email credentials"). +_ACCESS_VERB_RE = re.compile( + r"\b(?:read|cat|copy|cp|open|load|print|dump|include|source|export|" + r"base64|get|fetch|grab|collect|" + r"send|upload|transmit|exfiltrate|forward|submit)\b", re.I) + + +def _directive_units(text): + return [u for u in _DIRECTIVE_BREAK_RE.split(text) if u.strip()] + + +def _accesses_secret(unit, toks): + """True when `unit` directs the agent to ACCESS a secret: an access verb + with a credential token after it.""" + low = unit.lower() + verb = _ACCESS_VERB_RE.search(low) + return bool(verb) and any(tok in low[verb.end():] for tok in toks) + def _imperative_signals(text): """Semantic markers present in a blob of instruction text. Returns a sorted list drawn from {'conceal', 'egress', 'credential'}. Deliberately narrow: three tables, no scoring, no model. An empty list is - the overwhelmingly common answer and costs one pass.""" + the overwhelmingly common answer. + + 'credential' means the file directs the agent to ACCESS a secret -- an + access verb with the secret as its object, in one unit (_accesses_secret) + -- AND directs egress somewhere. It used to mean a secret was mentioned + anywhere, which made credential+egress ("together they are an exfil + instruction") true of nearly every long instruction file: the operator's + mail tool (#335/#336) paired "No secrets." with a docs link three sections + away, and two of his skills (#330/#332) paired a keychain note with an API + base URL. A mention is not an instruction to touch the secret. + + The access and the egress are deliberately NOT required to share a unit. + Multi-step injections are written as steps -- "1. Read ~/.aws/credentials + / 2. Upload the file to https://..." -- and a same-unit rule silenced + exactly those (found by review before merge). What the rule needs is two + directives, one that touches the secret and one that sends, not proximity. + + Every table here is narrower than the one it replaced, so a marker can + only disappear from a stored snapshot, never appear: a baseline written by + the old tables diffs against these without a single `gained` marker.""" if not text: return [] hits = set() - low = text.lower() - for tok in _credential_path_tokens(): - if tok in low: - hits.add("credential") - break for rx in _IMPERATIVE_CONCEAL: if rx.search(text): hits.add("conceal") @@ -15335,6 +16888,10 @@ def _imperative_signals(text): if rx.search(text): hits.add("egress") break + if "egress" in hits: + toks = _credential_path_tokens() + if any(_accesses_secret(u, toks) for u in _directive_units(text)): + hits.add("credential") return sorted(hits) @@ -15342,8 +16899,10 @@ def _imperative_severity(markers): """Map semantic markers to a severity. conceal alone is HIGH because it is attack-defined. credential+egress is - HIGH because together they are an exfil instruction. credential alone is a - MEDIUM record — plenty of legitimate instruction files mention .env.""" + HIGH because together they are an exfil instruction (credential is only + reported when the text directs an access to a secret AND an egress -- see + _imperative_signals). A bare credential marker, from a caller that builds + its own, is a LOW record.""" if not markers: return None if "conceal" in markers: @@ -15556,6 +17115,12 @@ def _reset_custody_probes(): _REPO_ROOT_CACHE.clear() _REPO_SELFNESS_CACHE.clear() _BUILD_OUTPUT_CACHE.clear() + # Not a git probe, but the same per-scan lifetime: the parsed installer + # receipts (see _cargo_dist_receipts, _rustup_toolchain_files and + # _cargo_install_records). + _CARGO_DIST_CACHE.clear() + _RUSTUP_CACHE.clear() + _CARGO_INSTALL_CACHE.clear() def _repo_root_of(git, d): @@ -15742,6 +17307,13 @@ def _supervised_rung(path, parents): is not re-conferred, `_demote` moves one step and never suppresses, and `_RISK_CUSTODY_WEIGHT` still lets the finding corroborate at half weight. """ + return "supervised" if _supervising_parent(path, parents) else None + + +def _supervising_parent(path, parents): + """The vouched ancestor that earns `path` the `supervised` rung, or None. + The walk _supervised_rung describes, returning WHO rather than whether, + because the supervisor class (_finding_classes) is keyed on its bytes.""" if not (path and parents): return None real = os.path.realpath(path) @@ -15754,7 +17326,7 @@ def _supervised_rung(path, parents): if (real.startswith(home) and all(p.startswith(home) for p in between) and _vouch_covers(parent)): - return "supervised" + return parent between.append(resolved) return None @@ -16026,6 +17598,14 @@ def _custody_carry_note(carried): "publisher can ship a bad build, and a stolen cert " "signs too, so this quiets it rather than clearing " "it.)"), + "publisher-signed": ("The binary carries a valid signature from a " + "publisher the platform itself trusts, and the " + "signer names itself — the ORIGIN of these bytes is " + "a known vendor, not a drop. Origin, not innocence: " + "a publisher can ship a bad build and a stolen " + "certificate signs cleanly until it is revoked, so " + "this demotes one step and routes to the digest; " + "attack-defined behaviour keeps its full severity."), "package-managed": ("This binary is owned by a package-manager transaction " "on this machine, proven by its receipt on disk — it " "arrived through an install you ran, not a drop. " @@ -16063,13 +17643,16 @@ def _custody_carry_note(carried): # authority as its baseline -> a vendor update # package-managed the binary is owned by a package-manager transaction on # this machine, proven by its RECEIPT on disk +# publisher-signed the binary carries a valid signature from a publisher the +# platform trusts (publisher_sig) that names its signer +# -> a vendor's build, on FIRST sight # # They are deliberately weaker than _SELF_CUSTODY: those three are claims of # AUTHORSHIP, these are claims of ORIGIN. So they demote one step (HIGH -> # MEDIUM), never straight to LOW, except `relocated`, which is a proof that # the executed content is byte-identical and therefore carries no new code. _VOUCHED_CUSTODY = ("relocated", "publisher-stable", "package-managed", - "os-vendor") + "os-vendor", "publisher-signed") # Recognised-but-weak: git knows the edit is local and unpushed. The note has # always read "routine if you made it" while the finding stayed HIGH anyway — @@ -16118,10 +17701,13 @@ def _demote(severity, provenance, attack_defined=False): The three invariants this function exists to hold, all inherited from the delegate-surface grader that came before it: - * grading DEMOTES, it never suppresses — every finding stays in the - report at its new level, no finding is dropped, no incident is - auto-closed, and nothing here ever writes a dismissal, so a demotion - can never feed acquired tolerance; + * demotion never suppresses — every finding stays in the report at + its new level, no finding is dropped, no incident is auto-closed, and + nothing here ever writes a dismissal, so a demotion can never feed + acquired tolerance. (Keeping a proven-origin finding out of the + interrupt tier is the ROUTING gate's decision, _provenance_gate, not + this function's: the finding still reaches the digest and the risk + tier at its custody weight.); * attack-DEFINED evidence is never demoted, whoever authored it. A hostile argv, an IOC hit, a dylib-injection env, a conceal imperative keeps its severity even under perfect custody: knowing who wrote a @@ -16139,6 +17725,12 @@ def _demote(severity, provenance, attack_defined=False): target = _step_down(severity) else: return severity + # Demotion never suppresses, so it never goes below LOW: INFO is what a + # sensor says when it has nothing to report, not a grade custody hands + # out. Without this a vouched or weak rung stepped a LOW finding -- a new + # LaunchAgent running an Apple-signed program -- down to INFO. + if SEV_ORDER[target] < SEV_ORDER["LOW"] <= SEV_ORDER[severity]: + target = "LOW" return target if SEV_ORDER[target] < SEV_ORDER[severity] else severity @@ -16291,6 +17883,328 @@ def _choco_receipt(real): return None +# --- installer receipts: installers that are not package managers ------------ +# +# The cargo-dist installer (`curl ... | sh` for uv and most Rust CLIs) and +# Playwright's browser download both leave a receipt as readable as Homebrew's, +# and their binaries were the residue of the process sensor's queue: an ad-hoc +# `uv` in ~/.local/bin and a Playwright Firefox whose seal reads broken. (That +# `uv` is also the PROGRAM of LaunchAgents a uv-run toolkit installs; the +# persistence ladder does not consult package receipts, so this does not +# reach those findings.) +# +# Doctrine, the same as every receipt above: a receipt is a same-uid file, so +# anything already running as the operator can forge one — exactly as it can +# drop an INSTALL_RECEIPT.json beside a Cellar file. That is why these answer +# only the vouched tier's `package-managed` rung: one severity step, never to +# LOW, a 0.25 weight in the risk tier, and `_grade_binary` never consults them +# for attack-defined evidence. Origin is not innocence; a receipt quiets the +# identity half of a finding and nothing else. + +# Parsed receipts per config root, for one scan: {root: {binary key: label}}. +# Cleared by _reset_custody_probes, so a receipt the next install rewrites is +# read again on the next scan and not before. +_CARGO_DIST_CACHE = {} + + +def _cargo_dist_config_roots(): + """Where a cargo-dist installer writes `/-receipt.json`. + + Read from the installers themselves (uv 0.11.6, cargo-dist 0.31.0): + install.sh uses ${XDG_CONFIG_HOME:-$HOME/.config} (or %LOCALAPPDATA% from + a Windows posix shell), install.ps1 uses %XDG_CONFIG_HOME% else + %LOCALAPPDATA%. XDG_CONFIG_HOME is asked first and the default as well, + because the environment Aegis runs under is not the shell that installed. + """ + roots = [] + xdg = os.environ.get("XDG_CONFIG_HOME") + if xdg and os.path.isabs(xdg): + roots.append(xdg) + if IS_WIN: + default = os.environ.get("LOCALAPPDATA") + else: + default = os.path.join(HOME, ".config") + if default and default not in roots: + roots.append(default) + return roots + + +def _binary_key(path): + """One spelling of a file for comparing two paths to it: links resolved, + and case folded where the filesystem folds it.""" + return os.path.normcase(os.path.realpath(path)) + + +def _parse_cargo_dist_receipt(receipt_path, dirname): + """{binary key: label} for one receipt, or {} when it is not one. + + Only a plain file name in `binaries` is honoured: an entry with a + separator in it would turn a same-uid JSON file into a vouch for any path + on the disk. The hierarchical layouts record the root and install one + level down in bin/; flat (and a receipt that predates the field) installs + into the prefix itself.""" + try: + data = json.loads(_read_text(receipt_path, limit=64 * 1024) or "") + except ValueError: + return {} + if not isinstance(data, dict): + return {} + binaries, prefix = data.get("binaries"), data.get("install_prefix") + if (not isinstance(binaries, list) or not isinstance(prefix, str) + or not os.path.isabs(prefix)): + return {} + if data.get("install_layout") in ("hierarchical", "cargo-home"): + prefix = os.path.join(prefix, "bin") + source = data.get("source") if isinstance(data.get("source"), dict) else {} + name = source.get("name") or source.get("app_name") or dirname + label = "cargo-dist:%s" % name + if source.get("owner"): + label = "cargo-dist:%s/%s" % (source["owner"], name) + if data.get("version"): + label += "@%s" % data["version"] + found = {} + for b in binaries: + if (isinstance(b, str) and b not in ("", ".", "..") + and os.path.basename(b) == b): + found[_binary_key(os.path.join(prefix, b))] = label + return found + + +def _cargo_dist_receipts(): + """{binary key: label} over every cargo-dist receipt on this body, read + once per scan. Bounded to `//-receipt.json`: the installer + names the receipt after its own directory, and nothing else is read.""" + out = {} + for root in _cargo_dist_config_roots(): + if root not in _CARGO_DIST_CACHE: + found = {} + try: + names = sorted(os.listdir(root)) + except OSError: + names = [] + for d in names: + rp = os.path.join(root, d, d + "-receipt.json") + if os.path.isfile(rp): + found.update(_parse_cargo_dist_receipt(rp, d)) + _CARGO_DIST_CACHE[root] = found + out.update(_CARGO_DIST_CACHE[root]) + return out + + +def _cargo_dist_receipt(real): + """A binary a cargo-dist installer put on disk: `real` resolves to + `/` for a binary its receipt lists. Same-uid + forgeable like the Homebrew receipt, hence vouched-tier only (see the + section comment above).""" + if not real: + return None + return _cargo_dist_receipts().get(_binary_key(real)) + + +# `-`; the browser part may carry `_` and a host-platform +# tag (`webkit_ubuntu20.04-x64_special-2092`), the revision is digits. +_PLAYWRIGHT_BROWSER_DIR = re.compile(r"^[^.].*-\d+$") + + +def _playwright_roots(): + """The browser caches Playwright installs into, as its registry resolves + them: PLAYWRIGHT_BROWSERS_PATH when it names an absolute directory, and + the platform default as well (Aegis's environment is not the installer's). + "0" (node_modules/.../.local-browsers) and a relative value (resolved + against the installing process's cwd) are not knowable here and are not + guessed at.""" + roots = [] + env = os.environ.get("PLAYWRIGHT_BROWSERS_PATH") + if env and env != "0" and os.path.isabs(env): + roots.append(env) + if IS_MAC: + default = os.path.join(HOME, "Library", "Caches", "ms-playwright") + elif IS_WIN: + base = (os.environ.get("LOCALAPPDATA") + or os.path.join(HOME, "AppData", "Local")) + default = os.path.join(base, "ms-playwright") + else: + base = os.environ.get("XDG_CACHE_HOME") or os.path.join(HOME, ".cache") + default = os.path.join(base, "ms-playwright") + if default not in roots: + roots.append(default) + return roots + + +def _playwright_receipt(real): + """A file inside a `-` directory directly under a + Playwright cache root that holds INSTALLATION_COMPLETE — the marker + Playwright's download worker writes after the extract finished, never + before. The marker is only evidence where Playwright writes it: the same + tree anywhere else answers nothing. Same-uid forgeable like the Homebrew + receipt, hence vouched-tier only (see the section comment above).""" + if not real: + return None + target = os.path.normcase(os.path.abspath(real)) + for root in _playwright_roots(): + for base in {os.path.normcase(os.path.abspath(root)), + os.path.normcase(os.path.realpath(root))}: + if not target.startswith(base.rstrip(os.sep) + os.sep): + continue + rest = target[len(base.rstrip(os.sep)) + 1:].split(os.sep) + if len(rest) < 2 or not _PLAYWRIGHT_BROWSER_DIR.match(rest[0]): + continue + if os.path.isfile(os.path.join(base, rest[0], + "INSTALLATION_COMPLETE")): + return "playwright:%s" % rest[0] + return None + + +# The Rust toolchain installers: rustup, and `cargo install`. Both keep a +# record that names every file they wrote, which is a stronger receipt than a +# marker beside a directory, and both are same-uid-forgeable exactly like the +# Homebrew receipt — so they answer only the vouched tier (see the section +# comment above). The rustup PROXIES in /bin (`cargo`, `rustc`, +# ... are links to, or copies of, the rustup binary) get nothing: rustup-init +# leaves no receipt for rustup itself. settings.toml names the default +# toolchain and update-hashes/ holds a channel-manifest hash; neither +# records rustup's own bytes. + +# Parsed per toolchain / per cargo home, for one scan; cleared by +# _reset_custody_probes like _CARGO_DIST_CACHE. +_RUSTUP_CACHE = {} +_CARGO_INSTALL_CACHE = {} + + +def _rustup_homes(): + """RUSTUP_HOME when it names an absolute directory, and ~/.rustup as well + (Aegis's environment is not the shell that ran rustup).""" + roots = [] + env = os.environ.get("RUSTUP_HOME") + if env and os.path.isabs(env): + roots.append(env) + default = os.path.join(HOME, ".rustup") + if default not in roots: + roots.append(default) + return roots + + +def _rustup_toolchain_files(toolchain): + """{normcased path relative to `toolchain`: component} for every `file:` + line of every installed component's manifest. + + Format read from a real rustup install (rust-installer-version 3): + lib/rustlib/components lists the installed components one per line, and + lib/rustlib/manifest- lists what that component wrote, as + `file:` or `dir:` relative to the toolchain root. Only a + component `components` names is read, and only `file:` lines count: a + `dir:` line names a directory, and a file dropped into it afterwards was + never part of the install.""" + if toolchain in _RUSTUP_CACHE: + return _RUSTUP_CACHE[toolchain] + rustlib = os.path.join(toolchain, "lib", "rustlib") + files = {} + listed = _read_text(os.path.join(rustlib, "components"), limit=64 * 1024) + for comp in (listed or "").splitlines(): + comp = comp.strip() + if comp in ("", ".", "..") or os.path.basename(comp) != comp: + continue + manifest = _read_text(os.path.join(rustlib, "manifest-" + comp), + limit=4 * 1024 * 1024) + for line in (manifest or "").splitlines(): + if line.startswith("file:"): + rel = line[len("file:"):].strip() + if rel: + rel = os.path.normpath(rel.replace("/", os.sep)) + files[os.path.normcase(rel)] = comp + _RUSTUP_CACHE[toolchain] = files + return files + + +def _rustup_receipt(real): + """A file rustup installed: `real` resolves inside + /toolchains// and a manifest of an installed component + of that toolchain lists it. Resolved first, so a listed path that has been + swapped for a link out of the toolchain answers nothing.""" + if not real: + return None + target = _binary_key(real) + for home in _rustup_homes(): + base = os.path.join(_binary_key(home), "toolchains") + if not target.startswith(base + os.sep): + continue + rest = target[len(base) + 1:].split(os.sep, 1) + if len(rest) < 2 or not rest[0] or not rest[1]: + continue + comp = _rustup_toolchain_files(os.path.join(base, rest[0])).get(rest[1]) + if comp: + return "rustup:%s:%s" % (rest[0], comp) + return None + + +def _cargo_homes(): + """CARGO_HOME when it names an absolute directory, and ~/.cargo as well.""" + roots = [] + env = os.environ.get("CARGO_HOME") + if env and os.path.isabs(env): + roots.append(env) + default = os.path.join(HOME, ".cargo") + if default not in roots: + roots.append(default) + return roots + + +def _listed_file_key(path): + """The comparison key for a file a record lists: its directory resolved, + its own name kept. Unlike _binary_key it does not follow a link AT the + listed path: the record names the file the installer wrote there, not + wherever a link put there later points.""" + return os.path.normcase(os.path.join( + os.path.realpath(os.path.dirname(path)), os.path.basename(path))) + + +def _cargo_install_records(cargo_home): + """{listed file key: label} from /.crates2.json, cargo's own + record of `cargo install`: `installs` maps " ()" + to a record whose `bins` are the files it put in /bin. Only a + plain file name counts, as for cargo-dist.""" + if cargo_home in _CARGO_INSTALL_CACHE: + return _CARGO_INSTALL_CACHE[cargo_home] + found = {} + try: + data = json.loads(_read_text(os.path.join(cargo_home, ".crates2.json"), + limit=1024 * 1024) or "") + except ValueError: + data = None + installs = data.get("installs") if isinstance(data, dict) else None + bindir = os.path.join(cargo_home, "bin") + for key, rec in (installs.items() if isinstance(installs, dict) else ()): + parts = key.split(" ") + if not parts[0] or not isinstance(rec, dict): + continue + bins = rec.get("bins") + if not isinstance(bins, list): + continue + label = "cargo-install:%s" % parts[0] + if len(parts) > 1 and parts[1]: + label += "@%s" % parts[1] + for b in bins: + if (isinstance(b, str) and b not in ("", ".", "..") + and os.path.basename(b) == b): + found[_listed_file_key(os.path.join(bindir, b))] = label + _CARGO_INSTALL_CACHE[cargo_home] = found + return found + + +def _cargo_install_receipt(real): + """A binary `cargo install` put in /bin and recorded in + .crates2.json. Same-uid forgeable like the Homebrew receipt, hence + vouched-tier only (see the section comment above).""" + if not real: + return None + target = _binary_key(real) + for home in _cargo_homes(): + hit = _cargo_install_records(home).get(target) + if hit: + return hit + return None + + def _os_package_receipt(real): """The OS-NATIVE package manager's claim on `real`. @@ -16310,8 +18224,36 @@ def _os_package_receipt(real): # _os_package_receipt is LAST on purpose: the probes above it are pure path # arithmetic, while it can cost up to three subprocesses on Linux. Cheap # questions first, so the expensive one is only asked when no cheap answer won. +# +# Roster: installers on the reference Mac (2026-09-23) whose binaries land in +# user-writable paths, and whether a receipt here names them. A receipt is +# implemented only where the installer writes a record that names the file. +# covered Homebrew /Cellar INSTALL_RECEIPT.json +# covered pipx .../pipx/venvs/ pipx_metadata.json +# covered uv Pythons ~/.local/share/uv/python BUILD +# covered uv standalone ~/.local/bin/uv, uvx cargo-dist receipt +# covered Playwright ms-playwright/- INSTALLATION_COMPLETE +# covered rustup ~/.rustup/toolchains manifest- +# covered cargo install ~/.cargo/bin/ .crates2.json +# none rustup proxies ~/.cargo/bin/cargo, ... no receipt for rustup +# none mise ~/.local/share/mise/installs: .mise.backend.toml +# names the backend, not files (its one install here, node, is +# Developer ID signed and never reaches the ladder) +# none bun ~/.bun/bin/bun: no record (Developer ID signed) +# n/a uv tools uv-receipt.toml names the ~/.local/bin entry +# points, but they are scripts; the interpreter they run is covered +# as a uv Python +# n/a npm global prefix ~/.local has no lib/node_modules here; +# package bins are scripts run by node +# absent pyenv, go (~/go/bin), deno: not installed here; no receipt +# format checked +# Seen here with no installer record: vendor-installed Claude Code, Codex and +# a Node under an agent's home (all Developer ID signed), and an ad-hoc +# ~/.local/bin/herdr, which stays uncovered. _PACKAGE_RECEIPTS = (_homebrew_receipt, _vscode_receipt, _pipx_receipt, _uv_python_receipt, _winget_receipt, _choco_receipt, + _playwright_receipt, _cargo_dist_receipt, + _rustup_receipt, _cargo_install_receipt, _os_package_receipt) @@ -16346,8 +18288,88 @@ def _package_receipt(path): return None +# The OS vendor's own platform signature, per body. Its origin claim is only +# as good as WHERE the bytes run: inside the platform's protected install tree +# (TRUSTED_PREFIXES -- the sealed system volume on macOS; SystemRoot and the +# admin-only Program Files trees on Windows) it is the OS; a copy of the same +# signed bytes in /tmp, $HOME or %TEMP% is the living-off-the-land shape, and +# the signature says nothing about who put them there. A third-party vendor's +# signature (developer-id, app-store, signed-valid) names that vendor wherever +# it runs; Linux's os-managed is already a claim about the package's own path. +_PLATFORM_SIGNATURES = ("apple", "os-signed") + + +def _publisher_signer(path): + """The signature verdict for `path` when a publisher the platform trusts + vouches for these bytes AND names itself, else None. + + `publisher-stable` was the only rung that read a signature, and only on a + re-sign in place, so a valid Developer ID binary earned nothing on first + sight and beaconed HIGH from $HOME. This is the first-sight half. It reads + classify_signature, whose verdict is stat-cached per path, so a binary the + sensor has already classified costs no second probe. `publisher_sig` is + the per-body vocabulary (developer-id / app-store / apple on macOS; a + `strict: detritus` verdict is still developer-id); an ad-hoc, broken, + unsigned or unanchored (`signed-other`) signature is never it. A verdict + with no team and no authority names nobody the operator could check, so + it earns nothing either. Never raises: this runs inside a sensor's + grading path, where an error must cost a rung, never a scan. + + A probe that already failed this scan on these exact bytes is not asked + again: the answer would be the same non-answer, bought with a second + codesign run (or a second cold PowerShell on Windows). + + A platform signature (_PLATFORM_SIGNATURES) earns the rung only inside + the system tree, judged on the resolved path so a symlink or a `..` + cannot walk a copy in.""" + if not path: + return None + failed_on = _SIG_UNANSWERED.get(path) + if failed_on is not None and failed_on == _sig_stat(path): + return None + try: + sig = classify_signature(path) + except Exception: + return None + if not (publisher_sig(sig.get("trust")) + and (sig.get("team") or sig.get("authority"))): + return None + if sig.get("trust") in _PLATFORM_SIGNATURES and \ + not _is_trusted_prefix(os.path.realpath(path)): + return None + return sig + + +def _publisher_line(sig): + """The one line that names who signed and which control stands behind + it, so the operator can check the claim rather than take it.""" + trust, team = sig.get("trust"), sig.get("team") + authority = sig.get("authority") or "" + if trust == "os-managed": + return ("Owned by the distro package %s; the distribution's signed " + "package pipeline is the control that stands behind this " + "rung." % authority) + if trust in ("os-signed", "signed-valid"): + return ("Authenticode-signed by %s; the certificate chain and its " + "revocation are the control that stands behind this rung." + % authority) + if trust == "developer-id" and team: + name = authority.split(": ", 1)[-1] + if name.endswith(" (%s)" % team): + name = name[:-len(" (%s)" % team)] + who = "Developer ID team %s (%s)" % (team, name) + elif trust == "app-store": + who = "the Mac App Store%s" % (" for team %s" % team if team else "") + elif trust == "apple": + who = "Apple (%s)" % (authority or "platform binary") + else: + who = authority or "team %s" % team + return ("Signed by %s; Apple's notarization/revocation is the control " + "that stands behind this rung." % who) + + def _grade_binary(severity, path, attack_defined=False, endpoint=None, - sha=None, parents=None): + sha=None, parents=None, publisher_ok=True): """(graded_severity, rung, note) for a finding keyed on a BINARY's identity. process / net-listener / net-outbound / net-beacon all raise on the same @@ -16371,6 +18393,12 @@ def _grade_binary(severity, path, attack_defined=False, endpoint=None, the caller has one (check_processes, and only on a host with a vouch). It is the only input here that is not about the file: the `supervised` rung asks who STARTED it, which no path, receipt or repo can answer. + + `publisher_ok=False` is a caller saying the platform's own control has + already refused these bytes -- the hot-dir sensor after Gatekeeper + rejected the bundle. The publisher rung leans on exactly that control + (notarization and revocation), so it is not asked; the rest of the ladder + is. """ if attack_defined: return severity, None, None @@ -16379,9 +18407,17 @@ def _grade_binary(severity, path, attack_defined=False, endpoint=None, # the caller names an endpoint, the vouch must cover that exact endpoint — # an identity vouch never widens into "may talk to anywhere". if _vouch_covers(path, endpoint): - rung = "operator-vouched" + rung, signer = "operator-vouched", None elif _package_receipt(path): - rung = "package-managed" + rung, signer = "package-managed", None + else: + # A publisher the platform itself trusts, on FIRST sight. Asked only + # when neither stronger rung answered, so a vouched or receipted + # binary never costs a signature lookup here. + rung, signer = None, (_publisher_signer(path) if publisher_ok + else None) + if rung or signer: + rung = rung or "publisher-signed" # Last, and weakest: a generated artifact of a repo this machine commits # to. Before this rung existed, the two questions above were the ONLY ones # asked of a binary, and a developer's machine answers no to both for @@ -16416,9 +18452,15 @@ def _grade_binary(severity, path, attack_defined=False, endpoint=None, _custody_carry_note(carried)) # A rung earned HERE is what a later copy elsewhere will inherit. Recorded # after the grading decision, never before it, so the ledger only ever - # holds rungs that were actually awarded. - _custody_remember(sha or _graded_sha(path), rung, path) - return _demote(severity, rung), rung, _PROVENANCE_NOTE.get(rung) + # holds rungs that were actually awarded. Except a platform signature: it + # earned the rung only BECAUSE of where it runs, so a copy of those bytes + # elsewhere must not inherit even `copy-of-graded` from it. + if not (signer and signer.get("trust") in _PLATFORM_SIGNATURES): + _custody_remember(sha or _graded_sha(path), rung, path) + note = _PROVENANCE_NOTE.get(rung) + if signer: + note = "%s %s" % (_publisher_line(signer), note) + return _demote(severity, rung), rung, note # --- the vouch tier: a workload the operator signed for, by hand ------------- @@ -17014,6 +19056,101 @@ def _custody_persistence(old, rec): return None +def _custody_payload(path, sha): + """(rung, note) for the payload script of a CHANGED persistence item. + + The two questions _custody asks -- a signed intent receipt for these + exact bytes at this path, then git -- with two differences the live store + showed were needed (2026-09-23): + + * no `_intent_worthy` prefilter. That exists to keep HOOK mode from + recording every file an agent touches; a script a persistence item + runs is worth grading by definition. #319's payload, + ~/.local/bin/improver, has no extension, so it was never asked. + * when git gives NO answer -- not in a repo, or a probe timed out -- + the bytes are asked what _grade_binary asks last: were they already + explained somewhere else? By the custody ledger (a rung this payload + earned on an earlier scan, which is what stops a git timeout flipping + a graded case back to HIGH: 23 of the 217 scans that recorded run.py + did exactly that, taking 11-190 s per job against 0-1 s when git + answered), or by an intent receipt for the same sha at another path + (#319: the agent wrote improver.py, an install step copied the + bytes). Either carries as `copy-of-graded` -- the weakest rung, one + step, never a re-conferral. + + A carried rung fills a non-answer and never argues with an answer: + `untracked` and `remote-foreign` stand as git gave them. A rung earned + here is remembered against the bytes, so the next non-answer can carry it. + """ + rung, note = _custody(path, sha) + if rung in _SELF_CUSTODY or rung in _VOUCHED_CUSTODY \ + or rung in _WEAK_CUSTODY: + _custody_remember(sha, rung, path) + return rung, note + if rung is not None: + return rung, note + carried = _custody_carried(sha) + if carried is None: + receipt = _intent_receipt(sha) + if receipt: + carried = ("self-attested",) + receipt + if carried: + return "copy-of-graded", _custody_carry_note(carried) + return None, "" + + +def _weaker_rung(severity, first, second): + """The rung that explains LESS: the one the risk tier weighs heavier + (_RISK_CUSTODY_WEIGHT; no rung, or one that does not demote, weighs 1.0), + then the one that leaves the higher severity. Ties keep `first`.""" + def weakness(item): + rung = item[0] + return (_RISK_CUSTODY_WEIGHT.get(rung, 1.0), + SEV_ORDER.get(_demote(severity, rung), 0)) + return max((first, second), key=weakness) + + +def _custody_executes(rec, severity, graded=None): + """(rung, note) for a NEW persistence item, graded by what it executes. + + The program through the binary ladder (_grade_binary: vouched, package + receipts, build output, carried copies) and the payload script, when + there is one, through _custody_payload. The item's rung is the WEAKER of + the two: a strong program running an unexplained script explains + nothing, and neither does an explained script run by an unexplained + program. No payload means the program's rung alone. + + No rung at all when the argv can run anything else + (_persistence_exec_extra), when the program is not on disk, or when the + payload has no hash to grade -- an unhashed script is not an explained + one. The caller has already refused attack-defined jobs. + + `graded` memoizes answers for one check_persistence call, keyed on the + program's and the payload's bytes. + """ + graded = {} if graded is None else graded + program, args = rec.get("program"), rec.get("args") + if not program or not rec.get("sha256"): + return None, None + payload = _script_target(args, program) + if _persistence_exec_extra(args, program, payload): + return None, None + if payload and not rec.get("target_sha"): + return None, None + key = ("program", program, rec.get("sha256")) + if key not in graded: + _sev, rung, note = _grade_binary(severity, program, + sha=rec.get("sha256")) + graded[key] = (rung, note) + result = graded[key] + if payload: + key = ("payload", payload, rec.get("target_sha")) + if key not in graded: + graded[key] = _custody_payload(payload, rec.get("target_sha")) + result = _weaker_rung(severity, result, graded[key]) + return result + + def cmd_vouch(argv): @@ -17235,6 +19372,26 @@ def _custody(path, content_sha): return prov, _PROVENANCE_NOTE.get(prov, "") +def _custody_strength(rung): + """3 authorship, 2 origin, 1 weak local evidence, 0 none: the order + _demote already applies, as a number, for choosing between two rungs.""" + if rung in _SELF_CUSTODY: + return 3 + if rung in _VOUCHED_CUSTODY: + return 2 + if rung in _WEAK_CUSTODY: + return 1 + return 0 + + +def _bytes_custody(path, content_sha): + """_custody asked where the bytes LIVE. A symlink is answered for by + nobody: git refuses a pathspec under one ("outside repository"), so + ~/.codex/skills/ -- a link into the operator's skills repo -- graded + None for content that repo had committed (#329-#332).""" + return _custody(os.path.realpath(path), content_sha) + + # --- the intent ledger: supervised writes attest themselves ------------------- # # Git answers "how did this arrive" only for tracked files. The intent ledger @@ -17343,6 +19500,41 @@ def _intent_attested(path, sha): return False +def _intent_receipt(sha): + """(path, ts) of the newest valid intent record for content `sha` at ANY + path, or None. + + _intent_attested binds bytes to the path the agent wrote. This asks about + the bytes alone, for the one caller that must recognise a copy: an agent + writes a script in its repo and an install step copies it to where a + persistence item runs it (#319: improver.py -> ~/.local/bin/improver, + same sha a588f96d11b4). What that proves is carried, never re-conferred + -- see _custody_payload. Same validity rules: newest first, and a bad MAC + or a stale timestamp is a non-match.""" + if not sha: + return None + try: + with open(INTENT_FILE, "rb") as f: + blob = f.read(_INTENT_MAX_BYTES).decode("utf-8", "replace") + except OSError: + return None + cutoff = _epoch() - _INTENT_MAX_AGE_DAYS * 86400 + for ln in reversed(blob.splitlines()): + try: + rec = json.loads(ln) + except Exception: + continue + if not isinstance(rec, dict) or rec.get("sha256") != sha: + continue + if _epoch(rec.get("ts")) < cutoff: + continue + expect = _intent_mac(rec.get("ts"), rec.get("path"), + rec.get("sha256"), rec.get("tool")) + if hmac.compare_digest(expect, str(rec.get("mac") or "")): + return rec.get("path"), rec.get("ts") + return None + + def _intent_worthy(path): """Is `path` something the delegate surface could ever grade? Cheap prefilter for hook mode so the ledger holds agent-surface writes, not a @@ -17456,6 +19648,44 @@ def cmd_intent(argv): # --- discovery + snapshot ---------------------------------------------------- +# Fields only a RUNNING process has. A registration names what WILL run when +# the agent starts; it cannot carry the pid and start time of a process that +# already ran. +_PROCESS_PID_KEYS = ("osPid", "pid") +_PROCESS_START_KEYS = ("startedAtMs", "startedAt", "started_at", "startTime") + + +def _is_process_table(obj): + """True when a parsed agent file is a host's RUNTIME PROCESS TABLE: a + top-level list in which EVERY record carries a command, an integer OS pid + and a start time. + + Codex keeps one at ~/.codex/process_manager/chat_processes.json -- one + record per command a chat turn started (osPid, startedAtMs, turnId, cwd), + last rewritten by Codex itself. It is a log of what already ran, not + configuration: nothing reads it back to start anything. Walked by shape it + has a `command` in every record, so it was diffed as a delegate config, + and #521 was `npm run build` "changing underneath a static config" because + a node upgrade moved the npm that `which` found. + + The test is on the WHOLE document, never one entry. An attacker who adds + `osPid` to a server block in a real config still registers an exec, + because a real config is a mapping of servers or hooks, not a list of + process records -- and a host that parsed one would be broken by the + rewrite, not armed by it.""" + if not isinstance(obj, list) or not obj: + return False + for rec in obj: + if not isinstance(rec, dict) or not isinstance(rec.get("command"), str): + return False + pid = next((rec[k] for k in _PROCESS_PID_KEYS if k in rec), None) + if isinstance(pid, bool) or not isinstance(pid, int): + return False + if not any(k in rec for k in _PROCESS_START_KEYS): + return False + return True + + def _agent_exec_entries(obj, where=""): """Every exec-capable entry in a parsed agent config, found by SHAPE. @@ -17463,7 +19693,10 @@ def _agent_exec_entries(obj, where=""): holding a string `command`) and the hook shape (a dict holding a string `command` under a hooks/tools key) wherever they appear in the tree, at any nesting depth, under any key name — because the key names change per host - and per release and the shape does not.""" + and per release and the shape does not. A runtime process table has the + same shape and registers nothing; see _is_process_table.""" + if _is_process_table(obj): + return [] out = [] def walk(node, path, depth): @@ -17501,6 +19734,28 @@ def _toml_exec_entries(text): return out +_EXEC_SCRIPT_RE = re.compile(r"\.(?:js|mjs|cjs|py|sh|ts|rb|php|jar|ps1)$") + + +def _expand_exec_word(word): + """One word of a hook's command line, as the shell that runs it names the + path. + + Separator punctuation glued to a word (`"...hook.py";`) is shell syntax, + not part of the path. `$HOME` and `${HOME}` are expanded because the hook + runs under a shell that expands them and shlex does not: the operator's + Codex hooks quote `"$HOME/Ai/Universe/smash/smash.py"`, which came back + as that literal string and resolved to nothing, so #298/#299 were recorded + with no target -- no hash to watch and no file to ask custody of. HOME + only: every other variable is the host's environment, which this process + cannot see.""" + word = word.rstrip(";&|").strip('"').strip("'") + for var in ("${HOME}", "$HOME"): + if word == var or word.startswith((var + "/", var + "\\")): + return os.path.normpath(HOME + word[len(var):]) + return os.path.expanduser(word) + + def _resolve_exec_target(command, args): """(resolved_abs_path_or_None, sha256_or_None). @@ -17536,15 +19791,30 @@ def _looks_like_path(a): if a.startswith(("/", "./", "../", "~")) or (os.sep != "/" and re.match(r"^[A-Za-z]:[\\/]", a)): return True - return bool(re.search(r"\.(?:js|mjs|cjs|py|sh|ts|rb|php|jar|ps1)$", a)) + return bool(_EXEC_SCRIPT_RE.search(a)) - cand = None + scripts, paths = [], [] for a in candidates: if a.startswith("-"): continue + word = _expand_exec_word(a) + if os.path.isabs(word) and _EXEC_SCRIPT_RE.search(word): + scripts.append(word) if _looks_like_path(a): - cand = os.path.expanduser(a) - break + paths.append(os.path.expanduser(a)) + # An absolute script beats the first path-shaped word. In a guarded hook + # line -- `mkdir -p "$HOME/.ai"; if command -v uv ...; then uv run + # "$HOME/.../hook_postwrite.py"; ...` -- no word was path-shaped as + # written, so the resolver fell back to the PROGRAM and hashed /bin/mkdir: + # a macOS update read as the hook's payload being swapped (#453), while + # the script that carries the behaviour was never watched at all. And + # `node ~/.claude/hooks/x.js; fi` resolved to 'x.js;', a file that never + # exists, so every guarded node hook went unhashed. The fallback is the + # old rule on the word AS WRITTEN, so a line with no absolute script + # (`cd && npx tool`, `npx --dir "$HOME/p"`) resolves exactly as it + # did before -- the baseline migration (v4) re-resolves only entries this + # function now answers differently. + cand = (scripts[0] if scripts else None) or (paths[0] if paths else None) target = cand or prog if not os.path.isabs(target): try: @@ -17562,6 +19832,175 @@ def _looks_like_path(a): return target, sha256(target) +# What a hook line may wrap around its target without running anything of its +# own: shell control keywords, no-op and test builtins, and `command -v ` +# (a PATH lookup that executes nothing). +_EXEC_CONTROL_WORDS = frozenset(("if", "then", "else", "elif", "fi", "!")) +_EXEC_NOOP_PROGRAMS = frozenset(("true", "false", ":", "[", "test")) +# Launchers that run the script named after them, and the only words allowed +# between a launcher and that script. Anything else in that slot can load code +# of its own -- `python3 -c`, `node --require x.js`, `uv run --with ` -- +# and is refused rather than enumerated. +_EXEC_LAUNCHERS = frozenset(("bash", "sh", "zsh", "python", "python3", "node", + "uv", "ruby", "perl")) +_EXEC_LAUNCHER_WORDS = frozenset(("run", "--quiet", "-q")) +_EXEC_SEPARATORS = frozenset((";", "&&", "||")) +_EXEC_REDIRECTS = frozenset((">", ">>", "<", ">&", "&>", ">|", "<&")) + + +def _exec_simple_commands(line): + """The simple commands of a hook's shell line, with control words and + /dev/null redirections removed -- or None when the line does anything + this reader will not vouch for: a pipe, a background job, a subshell, + command or process substitution, a redirect into a file, a second line, + or text shlex cannot split.""" + if any(s in line for s in ("`", "$(", "<(", ">(", "\n", "\r")): + return None + try: + lex = shlex.shlex(line, posix=not IS_WIN, punctuation_chars=True) + lex.whitespace_split = True + toks = [t.strip('"').strip("'") if IS_WIN else t for t in lex] + except ValueError: + return None + cmds, cur, i = [], [], 0 + while i < len(toks): + t = toks[i] + if t in _EXEC_SEPARATORS: + cmds.append(cur) + cur = [] + elif t in _EXEC_REDIRECTS: + nxt = toks[i + 1] if i + 1 < len(toks) else "" + if nxt != "/dev/null" and not nxt.isdigit(): + return None + if cur and cur[-1].isdigit(): # the fd in `2>&1` + cur.pop() + i += 1 + elif t and set(t) <= set("|&;<>()"): # |, &, (, ) and the rest + return None + else: + cur.append(t) + i += 1 + cmds.append(cur) + out = [] + for words in cmds: + while words and words[0] in _EXEC_CONTROL_WORDS: + words = words[1:] + if words: + out.append(words) + return out + + +def _exec_runs_only(command, args, target): + """True when an exec entry runs nothing except `target`: every simple + command in it is inert glue or an invocation of the target, by a known + launcher with nothing between launcher and script that could load code. + + This is what lets the TARGET's custody speak for a new entry. It says who + wrote the script; it says nothing about anything else the line runs, so + `bash ; curl ... | sh` must not borrow it. The + operator's own guarded-invocation idiom does qualify, because every word + around the script is inert: `if command -v uv >/dev/null 2>&1; then uv + run --quiet