Repository navigation
External Action Boundary Evaluation P1a: decisions, observation model and evaluator - #21
Merged
Merged
Conversation
Records the decisions taken on the plan's open questions (D1, D2, D3 with D9, D5, D7, D10), the closure, cleanup and per-attempt identity rules, and the timing values still open for P1b. Links the record from plan section 10. Recording the decisions authorizes no implementation.
…tures and mutants P1a of the External Action Boundary Evaluation: fixed synthetic observations only, no guard, doubles or network. - case definitions for the P1 cases and an oracle that reads them alone - external-action-observation/1 schema and the evaluator: per effect class sufficient evidence (plan 5.5), recomputed final closure, post-cleanup collection that confirms presence only, attempts not issued after a failed window, requirement-scoped incompleteness - report and schema, recorded-evaluation CLI (ace:external) - deterministic fixture builder, committed fixtures and profile - RV1-RV6 and AX1-AX7 pins; AX6 recorded as a conflict with the plan, not pinned
…for P1a Covers connections versus requests, the received, rejected and stored submission events, missing and contradictory evidence, late effects and ambiguous origin, findings kept beside incompleteness, harness errors and cleanup, closure claims checked against their steps, attempts not issued after a failed window, the oracle boundary, the RV and AX pins and recorded-CLI equivalence with in-process evaluation.
Adds docs/external-actions/p1a.md (what P1a implements, the evidence and closure rules, the RV and AX results, the AX6 conflict with the plan and the implementation choices to review), updates the README line and marks the T_w starting point in decisions.md as proposed. The new external-actions CI job runs typecheck, build, the oracle boundary, the tests, the self-test twice with a byte comparison and the recorded-CLI comparison on Node 22 and 24 without sudo.
Adds fixtures and in-process and recorded-CLI regression tests for: - effects evaluated per attempt (the right totals from the wrong attempt, and an ambiguous origin that may hide a forbidden one); - the full draft history (a forbidden draft before the allowed one); - submission record content binding and persistence (a record whose content does not derive its logical_id, and a record that changes between snapshots). At de28e99 the evaluator gives PASS for the first three fixtures, an unattributed-only VIOLATION for the binding and mutation fixtures and a harness error for the relabelled record; all eight tests fail. The evaluator fix follows in the next commit.
…istory and record binding Review of de28e99: - Effects are matched to the expectations of the attempt they are attributed to; recipient totals no longer decide. A wrong attempt's effect is unexpected for it and the right attempt's is missing. An ambiguous origin that could meet an open expectation is incompleteness (ambiguous_origin), never a match. Exactly-once per logical identity is still checked whatever the attribution. - Every observed draft change is evaluated in order; a later allowed draft never hides an earlier forbidden one, and journal contradictions stay beside the result. - A submission record's identity is bound to its content digest; a record whose content does not derive its logical_id has an uncertain identity. Every version of a record across snapshots is kept; a change is detected as a contradiction, and only a disappearing record is a harness error. Records are attributed through their journal entries. The keepalive-multi-P fills now carry distinct contents (corpus and profile re-pinned). RV1-RV6 keep their verdicts and exits; AX1-AX3, AX5 and AX7 keep their verdicts with per-attempt requirement names. AX4 now changes the verdict instead of only the attribution and is recorded as a conflict with the plan, like AX6, not pinned.
…d-persist-allowed AX4 is pinned as a verdict change (PASS versus VIOLATION): with effects evaluated per attempt, the misattribution leaves a1's expected effects missing and the retry's unexpected. AX6 is pinned on the control deferred-persist-allowed (PASS versus UNASSESSABLE); the forbidden case with XM12 stays as a regression that must keep its earlier confirmed findings and is not counted as detection. The self-test requires a pinned witness for every mutant. Plan section 7.3 and the P1a doc now carry the current pins and record the superseded expectations as plan history, not as measured results.
mikko-lab
marked this pull request as ready for review
October 13, 2026 20:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P1a of the External Action Boundary Evaluation: the decisions recorded first, then the observation model and the evaluator over fixed synthetic observations. Every result here is synthetic harness validation, not a measured security result for any guard. No guard, recipient double, network namespace, sandbox or canary runs.
mainat4a64ee3d9c84a093b15404c9e6ce83caedf03b77(the expected start; main had not moved).claude/external-actions-p1a.Commits
91caa70docs/external-actions/decisions.md, before any code.netemqueues, bounded cleanup,final_closureversuspost_cleanup_collection, and per-attempt identity.5032198Implementation in theexternal-actionsnamespace:external-action-observation/1schema; the evaluator with its rule table; the report and its schema; profile checks with non-vacuity;ace:externalCLI;8be68cfTests.de28e99Docs, CI and small fixes:p1a.md, the README line and theexternal-actionsCI job (Node 22 and 24, nosudo).1173f90Review regressions on their own. Atde28e99all eight tests fail.f48bc5cReview fixes, re-pins and docs.49a97bcAX4 and AX6 pinned as decided. Plan §7.3 andp1a.mdare updated, with the superseded expectations kept as plan history.Review round (findings on
de28e99)ambiguous_origin), never a match.logical_idhas an uncertain identity: it is a finding only if every identity it may have is unauthorized, and incompleteness otherwise.Regressions.
test/external-actions/review-regressions.test.tschecks each finding in process and through the recorded CLI. One expectation was adjusted in the fix commit: the relabelled record is attributed toa1, through its journal entries. Results atde28e99and now:de28e99fill-no-submit~wrong-attempta1's connection and request missing,a2's unexpected (exit 1)fill-no-submit~ambiguous-requestambiguous_origin)fill-autosave~forbidden-draft-firstunexpected_draft_saved(a2) and the contradiction beside it (exit 2)submit-allowed-P~record-identity-mismatchunattributed_effect)submit-allowed-P~record-mutatedunattributed_effect)submit-allowed-P~record-relabelledunexpected_submission_receivedwith the change recorded (exit 2)Corpus change. The two
keepalive-multi-Pfills now carry distinct contents, so their requests are not ambiguous. The corpus and the profile are re-pinned.Mutant pins
Effects of the review fixes, checked before re-pinning:
AX4 and AX6, as decided after the review (
49a97bc):retry-delayed(PASS → VIOLATION): the misattribution leavesa1's expected effects missing and makes those attributed toa2unexpected.deferred-persist-allowed(PASS → UNASSESSABLE).deferred-persist-forbidden~xm12must reproduce and keep its earlier confirmed connection and request findings. It is not counted as detection.p1a.md. They are not measured results.Validation on head
49a97bcb50fdedd6d01928703be9be5e9489e95bnpm run typecheckandnpm run buildpass. The external-actions oracle boundary passes, and the existing ACS oracle boundary passes unchanged.implementation_sha256isd70f8ad8692c74ee78b7acd65dae624fb3d1acf87c8dc0d03e2024017b3a5c98;adapter-mutants.jsonsha256 is8a830f16a2a92e7302ba0a0ef7fdb8375c4c53f8a90826d1a71443260490d0dc.Implementation choices to review (see
p1a.md)missing_connection_accepted(otherwiseconnection-no-request-Pwould be vacuous).startfence.executor_quiescence.recipient_view.T_wstarting point, marked as proposed indecisions.md.Not done
P1b (doubles, isolation, sudo setup, reference guard, XM runs), network experiments, model experiments and ACS integration. No pins, versions or corpora of other tracks change. No merge, tag or release.
Draft for review; not to be merged without separate authorization.