From c74321dcb3c1733a6ec527e60e474f5b14ceaf7d Mon Sep 17 00:00:00 2001 From: Radoslav Dimitrov Date: Wed, 1 Jul 2026 20:54:00 +0300 Subject: [PATCH] Add journey-integrity gate: edge: resolver + Surface tag (ADR-0077) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements the ac-trace side of Atrium's ADR-0077 / issue #547 — make the verify: gate prove journeys, not just layers. All of it is opt-in behind an optional .actrace.yml, so a repo that ships no config (Airlock) is unaffected. - config.go: the .actrace.yml loader (zero value = every feature off). - resolver.go: a generic verify-method prefix→command hook. A token like edge:agentloop->files.GetFile is resolved by invoking the configured command with the token as one argv element (no shell — CWE-78/88); a custom-prefix token with no configured resolver is a hard failure, never a silent no-op. - journey.go: the **Surface:** scenario tag and the journey-proof gate. A user-facing scenario must carry >=1 AC citing a journey proof, gated on location AND substance: a ui-e2e-realfd: spec that asserts on a real server response (waitForResponse), or a test/e2e/ Go test that is a real-cluster run (//go:build e2e, not synthetic, no idpfake). Weak methods (demonstration/scenario/manual) are invalid on a user-facing AC; a pre-existing scenario grandfathers with a journey-ok: opt-out that must cite a tracked issue. Enforcing tests (ADR-0077): TestADR_0077_EdgeVerifyRejectedWithoutResolver, ResolverTokenPassedAsArgv, UserFacingScenarioRequiresJourneyProof, JourneyProofIsLocationGated, JourneyProofRejectsFakedSeam, UserFacingRejectsWeakMethods, GrandfatherOptOutRequiresIssue. Co-Authored-By: Claude Opus 4.8 --- CLAUDE.md | 12 ++ README.md | 46 +++++ go.mod | 6 +- internal/actrace/actrace.go | 35 +++- internal/actrace/actrace_test.go | 12 +- internal/actrace/config.go | 61 ++++++ internal/actrace/config_test.go | 54 +++++ internal/actrace/journey.go | 300 ++++++++++++++++++++++++++++ internal/actrace/journey_test.go | 153 ++++++++++++++ internal/actrace/resolver.go | 112 +++++++++++ internal/actrace/resolver_test.go | 98 +++++++++ internal/actrace/weakmethod_test.go | 94 +++++++++ 12 files changed, 965 insertions(+), 18 deletions(-) create mode 100644 internal/actrace/config.go create mode 100644 internal/actrace/config_test.go create mode 100644 internal/actrace/journey.go create mode 100644 internal/actrace/journey_test.go create mode 100644 internal/actrace/resolver.go create mode 100644 internal/actrace/resolver_test.go create mode 100644 internal/actrace/weakmethod_test.go diff --git a/CLAUDE.md b/CLAUDE.md index ad98ca8..4e46e92 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -21,6 +21,9 @@ proof; this tool checks those proofs resolve in the tree. - `internal/actrace/` — the library. All logic lives here. - `actrace.go` — plan parser, status lifecycle, forward gate, backward gate, `Run()` entrypoint (returns exit code int). + - `config.go` — the optional `.actrace.yml` loader (opt-in features). + - `resolver.go` — custom verify-method prefix→command hook (e.g. `edge:`). + - `journey.go` — the ADR-0077 `**Surface:**` tag + journey-proof gate. - `reverse.go` — orphan gate, staleness report. - `report.go` — result model, `--json` renderer. - `matrix.go` — committed traceability matrix renderer. @@ -40,6 +43,15 @@ usage/internal error). Never call `os.Exit` from the library. `ui-e2e-realfd:`) is Atrium-specific. In a repo with no `ui/` dir the spec index is empty and all `ui:` checks are no-ops. Do not remove this logic — Atrium depends on it. +- The ADR-0077 journey-integrity gate (`config.go`, `resolver.go`, + `journey.go`) is **opt-in**: it fires only for a repo that ships an + `.actrace.yml`. Absent config → zero `Config` → every new check off, so + a repo like Airlock is unaffected. `journey_integrity: true` enables the + `**Surface:**` tag + journey-proof gate; `resolvers:` maps a + custom `verify:` prefix (e.g. `edge:`) to a command. The journey-proof + location/substance conventions (`ui/tests/e2e/*.realfd.spec.ts`, + `test/e2e/`, `synthetic`, `idpfake`, `waitForResponse`) are Atrium-shaped + like the `ui:` vocabulary and no-op where those paths don't exist. ## Things that will bite you diff --git a/README.md b/README.md index 652e207..c8c902f 100644 --- a/README.md +++ b/README.md @@ -98,6 +98,49 @@ Plans declare a lifecycle on the `**Status:**` line: Only `landed` plans are gated. A plan names its tests before they are built, so enforcement waits until its implementation merges. +## Journey integrity (opt-in, ADR-0077) + +An optional `.actrace.yml` at the repo root turns on extra checks. Absent, +the tool behaves exactly as above — a repo that ships no config (e.g. +Airlock) is unaffected. + +```yaml +# .actrace.yml +journey_integrity: true # enable the Surface tag + journey-proof gate +resolvers: + "edge:": # a custom verify-method prefix + command: ["scripts/resolve-edge.sh"] +``` + +**Custom `verify:` methods (e.g. `edge:`).** A token like +`edge:agentloop->files.GetFile` is resolved by the command its prefix maps +to in `resolvers`. ac-trace runs that command with the raw token as one +final argv element — never through a shell — and reads the exit code: 0 +means the proof holds, non-zero means it does not. A custom-prefix token +whose prefix has no configured resolver is a hard failure, never a silent +pass. + +**The `**Surface:**` scenario tag.** With `journey_integrity: true`, each +scenario in a landed plan carries a `**Surface:** user-facing | +backend-foundation` field. A `user-facing` scenario must carry at least one +AC whose `verify:` cites a **journey proof**: + +- a `ui-e2e-realfd:` Playwright spec (a `.realfd.spec.ts` under + `ui/tests/e2e/`) that asserts on a real server response + (`waitForResponse`), not on rendered DOM alone; or +- a Go test under `test/e2e/` that is a real-cluster run — `//go:build + e2e`, not `synthetic`, and not importing `idpfake`. + +A mock spec, a unit test, a `test/e2e/` test that fakes its seam, or a +`demonstration` / `scenario` / `manual` method cannot satisfy a +`user-facing` scenario. A missing, unrecognised, or ambiguous tag is a hard +failure. + +**Grandfathering.** A pre-existing `user-facing` scenario with no journey +proof yet opts out with `journey-ok: ` on an AC's verify line — but +only when the reason cites a tracked issue (`#692` or an issues URL), so the +debt is visible and attributed. + ## Wiring into a repo Add to your `Taskfile.yml`: @@ -169,6 +212,9 @@ Run `actrace --matrix` and commit the generated `actrace.Run`, exits with its return code). - `internal/actrace/actrace.go` — plan parser, status lifecycle, forward gate, backward gate, `Run()` entrypoint. +- `internal/actrace/config.go` — the optional `.actrace.yml` loader. +- `internal/actrace/resolver.go` — custom verify-method prefix→command hook. +- `internal/actrace/journey.go` — the ADR-0077 Surface tag + journey-proof gate. - `internal/actrace/reverse.go` — orphan gate, staleness report. - `internal/actrace/report.go` — result model, `--json` renderer. - `internal/actrace/matrix.go` — committed traceability matrix renderer. diff --git a/go.mod b/go.mod index b6eea1b..18e368c 100644 --- a/go.mod +++ b/go.mod @@ -2,10 +2,12 @@ module github.com/stacklok/ac-trace go 1.26 -require github.com/stretchr/testify v1.11.1 +require ( + github.com/stretchr/testify v1.11.1 + gopkg.in/yaml.v3 v3.0.1 +) require ( github.com/davecgh/go-spew v1.1.1 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect - gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/internal/actrace/actrace.go b/internal/actrace/actrace.go index 022d0d0..13b0e07 100644 --- a/internal/actrace/actrace.go +++ b/internal/actrace/actrace.go @@ -255,13 +255,19 @@ func Run(args []string) int { return 2 } + cfg, err := loadConfig(".") + if err != nil { + fmt.Fprintln(os.Stderr, "loading .actrace.yml:", err) + return 2 + } + plans, err := resolvePlans(*plan) if err != nil { fmt.Fprintln(os.Stderr, "globbing plans:", err) return 2 } - totalFailures := runPlans(plans, index, fe, ground) + totalFailures := runPlans(plans, index, fe, ground, cfg) // The backward gate runs once over the whole tree, not per plan: every // landed TestADR_NNNN_* must name an ADR that exists and is not retired. @@ -333,14 +339,14 @@ func resolvePlans(plan string) ([]string, error) { // runPlans checks each plan and returns the total forward-check failures. The // README and meta plans are skipped; an unreadable or mis-statused plan is a // hard error that exits the process. -func runPlans(plans []string, index map[string]string, fe feIndex, ground grounding) int { +func runPlans(plans []string, index map[string]string, fe feIndex, ground grounding, cfg Config) int { totalFailures := 0 for _, p := range plans { base := filepath.Base(p) if base == "README.md" || isMetaPlan(base) { continue } - failures, skipped, err := runPlan(p, index, fe, ground) + failures, skipped, err := runPlan(p, index, fe, ground, cfg) if err != nil { fmt.Fprintln(os.Stderr, "checking", p, ":", err) os.Exit(2) @@ -594,7 +600,9 @@ func loadGrounding(root string) (grounding, error) { // number of failures. draft and in-progress plans are reported but never gate, // so they return zero failures. A superseded plan is skipped (skipped=true). An // unrecognised status is a hard error. -func runPlan(path string, index map[string]string, fe feIndex, ground grounding) (failures int, skipped bool, err error) { +func runPlan( + path string, index map[string]string, fe feIndex, ground grounding, cfg Config, +) (failures int, skipped bool, err error) { b, err := os.ReadFile(path) if err != nil { return 0, false, err @@ -609,18 +617,18 @@ func runPlan(path string, index map[string]string, fe feIndex, ground grounding) } acs := parseACs(lines) if status == "landed" { - return checkLanded(path, acs, index, fe, ground), false, nil + return checkLanded(path, lines, acs, index, fe, ground, cfg), false, nil } // draft / in-progress: report only, never gate. - reportPlan(path, status, acs, lines, index, fe) + reportPlan(path, status, acs, lines, index, fe, cfg) return 0, false, nil } // reportPlan prints a draft / in-progress plan's coverage without gating. -func reportPlan(path, status string, acs []acEntry, lines []string, index map[string]string, fe feIndex) { +func reportPlan(path, status string, acs []acEntry, lines []string, index map[string]string, fe feIndex, cfg Config) { fmt.Printf("(%s, report-only) ", status) if hasAnyVerify(acs) { - checkStructured(path, acs, index, fe) + checkStructured(path, acs, index, fe, cfg) return } checkProse(path, lines, index) @@ -629,7 +637,9 @@ func reportPlan(path, status string, acs []acEntry, lines []string, index map[st // checkLanded runs the forward gate on a landed plan and returns the number of // failures: zero structured ACs; an AC with no verify: field; a verify: test // that does not resolve; or a bare-text ADR / Principle that grounds to nothing. -func checkLanded(path string, acs []acEntry, index map[string]string, fe feIndex, ground grounding) int { +func checkLanded( + path string, lines []string, acs []acEntry, index map[string]string, fe feIndex, ground grounding, cfg Config, +) int { fmt.Printf("== %s == [landed]\n", strings.TrimPrefix(path, "./")) if len(acs) == 0 { fmt.Println(" ✗ landed plan has zero ACx.y criteria — must assert at least one") @@ -652,9 +662,13 @@ func checkLanded(path string, acs []acEntry, index map[string]string, fe feIndex } failures += checkUIRefs(e, fe) failures += checkRenderFromWireGate(e, fe) + failures += checkResolverRefs(e, cfg) } failures += checkGrounding(e, ground) } + if cfg.JourneyIntegrity { + failures += checkJourneyIntegrity(lines, acs, fe, index) + } fmt.Printf(" %d ACs · %d failure(s)\n", len(acs), failures) return failures } @@ -820,7 +834,7 @@ func hasAnyVerify(acs []acEntry) bool { // checkStructured evaluates a plan's verify: fields. Test-method criteria have // every named Go test and `ui:` reference checked; non-test methods are // accepted. -func checkStructured(path string, acs []acEntry, index map[string]string, fe feIndex) int { +func checkStructured(path string, acs []acEntry, index map[string]string, fe feIndex, cfg Config) int { fmt.Printf("== %s == [structured]\n", strings.TrimPrefix(path, "./")) missing, unannotated, testACs, methodACs := 0, 0, 0, 0 for _, e := range acs { @@ -839,6 +853,7 @@ func checkStructured(path string, acs []acEntry, index map[string]string, fe feI } } missing += checkUIRefs(e, fe) + missing += checkResolverRefs(e, cfg) } } fmt.Printf(" %d ACs · %d test-verified · %d method · %d unannotated · %d missing\n", diff --git a/internal/actrace/actrace_test.go b/internal/actrace/actrace_test.go index 044db24..bb7c2cb 100644 --- a/internal/actrace/actrace_test.go +++ b/internal/actrace/actrace_test.go @@ -117,7 +117,7 @@ func TestCheckStructured_IgnoresProseAndCountsMissing(t *testing.T) { index := map[string]string{"TestThing_Exists": "x_test.go"} acs := parseACs(strings.Split(body, "\n")) - missing := checkStructured(plan, acs, index, nil) + missing := checkStructured(plan, acs, index, nil, Config{}) // Only TestThing_Missing counts; the prose `TestThing_Retired` is ignored. assert.Equal(t, 1, missing) } @@ -289,7 +289,7 @@ func TestADR_0065_LandedPlanVerifyTestMustExist(t *testing.T) { content := "# A plan\n\n**Status:** " + tc.status + ", 2026-06-17.\n\n" + tc.body require.NoError(t, os.WriteFile(plan, []byte(content), 0o644)) - fail, skip, err := runPlan(plan, index, nil, ground) + fail, skip, err := runPlan(plan, index, nil, ground, Config{}) if tc.wantStatusErr { require.Error(t, err, "an unrecognised status must be a hard error") return @@ -694,7 +694,7 @@ func TestCheckLanded_ResolvesUITestReference(t *testing.T) { // the resolution failures. (AC1.2 has no resolvable FE proof so the gate // does not fire for it; AC1.3 is ambiguous so the gate also does not // fire — checkRenderFromWireGate skips cites that don't resolve to one.) - assert.Equal(t, 4, checkLanded("fe.md", acs, map[string]string{}, fe, ground)) + assert.Equal(t, 4, checkLanded("fe.md", nil, acs, map[string]string{}, fe, ground, Config{})) } // TestADR_0065_RenderFromWireGateDefaultDeny pins the frontend half of the @@ -771,7 +771,7 @@ func TestADR_0065_RenderFromWireGateDefaultDeny(t *testing.T) { t.Parallel() index := map[string]string{"TestThing_Exists": "x_test.go"} acs := []acEntry{{id: "AC1.1", verify: tc.verify, hasVerify: true, body: "AC1.1: a behaviour"}} - got := checkLanded("p.md", acs, index, fe, ground) + got := checkLanded("p.md", nil, acs, index, fe, ground, Config{}) assert.Equal(t, tc.wantFail, got > 0, "strict-failure decision") }) } @@ -820,7 +820,7 @@ func TestADR_0065_RenderFromWireGateMislabel(t *testing.T) { t.Run(tc.name, func(t *testing.T) { t.Parallel() acs := []acEntry{{id: "AC1.1", verify: tc.verify, hasVerify: true, body: "AC1.1: a behaviour"}} - got := checkLanded("p.md", acs, map[string]string{}, fe, ground) + got := checkLanded("p.md", nil, acs, map[string]string{}, fe, ground, Config{}) assert.Equal(t, tc.wantFail, got > 0, "strict-failure decision") }) } @@ -907,7 +907,7 @@ func TestADR_0065_RealfdProofType(t *testing.T) { //nolint:paralleltest // captu acs := []acEntry{{id: "AC1.1", verify: tc.verify, hasVerify: true, body: "AC1.1: a behaviour"}} var failures int out := captureOutput(func() { - failures = checkLanded("p.md", acs, map[string]string{}, fe, ground) + failures = checkLanded("p.md", nil, acs, map[string]string{}, fe, ground, Config{}) }) assert.Equal(t, tc.wantFail, failures > 0, "strict-failure decision") if tc.wantMessage != "" { diff --git a/internal/actrace/config.go b/internal/actrace/config.go new file mode 100644 index 0000000..5afa4f7 --- /dev/null +++ b/internal/actrace/config.go @@ -0,0 +1,61 @@ +// SPDX-FileCopyrightText: Copyright 2026 Stacklok, Inc. +// SPDX-License-Identifier: LicenseRef-Stacklok-Proprietary + +package actrace + +import ( + "fmt" + "os" + "path/filepath" + + "gopkg.in/yaml.v3" +) + +// Config is the optional `.actrace.yml` at a repo root. It is how a consuming +// repo turns on the opt-in features added for ADR-0077 (Atrium) without +// changing behaviour for a repo that ships no config (Airlock). A missing file +// yields the zero Config, which disables every feature — so the tool behaves +// exactly as it did before this file existed. +type Config struct { + // Resolvers maps a verify-method prefix (including its colon, e.g. + // "edge:") to an external command that decides whether a token with that + // prefix holds. ac-trace invokes the command with the raw verify token as + // a single, final argv element — no shell — and reads the exit code: 0 + // means satisfied, non-zero means the proof does not hold. This keeps + // ac-trace domain-agnostic: the consuming repo owns what "edge:" means. + Resolvers map[string]ResolverConfig `yaml:"resolvers"` + + // JourneyIntegrity enables the ADR-0077 gate on a landed plan: the + // `**Surface:**` scenario tag, the journey-proof requirement on a + // user-facing scenario, and the rejection of weak proof methods for a + // user-facing AC. Off by default so a repo that has not adopted the + // convention is unaffected. + JourneyIntegrity bool `yaml:"journey_integrity"` +} + +// ResolverConfig is one verify-method-prefix resolver: the argv of the command +// ac-trace runs. ac-trace appends the raw verify token as one additional argv +// element, so the token reaches the command as inert data, never through a +// shell. +type ResolverConfig struct { + Command []string `yaml:"command"` +} + +// loadConfig reads `.actrace.yml` at root. A missing file is not an error — it +// returns the zero Config (every opt-in feature off). A present-but-malformed +// file is an error: a typo must not silently disable a gate the repo meant to +// turn on. +func loadConfig(root string) (Config, error) { + var c Config + b, err := os.ReadFile(filepath.Join(root, ".actrace.yml")) + if err != nil { + if os.IsNotExist(err) { + return c, nil + } + return c, fmt.Errorf("reading .actrace.yml: %w", err) + } + if err := yaml.Unmarshal(b, &c); err != nil { + return c, fmt.Errorf(".actrace.yml: %w", err) + } + return c, nil +} diff --git a/internal/actrace/config_test.go b/internal/actrace/config_test.go new file mode 100644 index 0000000..dd749a5 --- /dev/null +++ b/internal/actrace/config_test.go @@ -0,0 +1,54 @@ +// SPDX-FileCopyrightText: Copyright 2026 Stacklok, Inc. +// SPDX-License-Identifier: LicenseRef-Stacklok-Proprietary + +package actrace + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// TestLoadConfig_MissingFileIsZeroValue pins the opt-in contract: a repo that +// ships no .actrace.yml gets the zero Config (every feature off), not an error. +func TestLoadConfig_MissingFileIsZeroValue(t *testing.T) { + t.Parallel() + c, err := loadConfig(t.TempDir()) + require.NoError(t, err) + assert.False(t, c.JourneyIntegrity) + assert.Nil(t, c.Resolvers) +} + +// TestLoadConfig_ParsesResolversAndFlag pins parsing of a present config: the +// journey_integrity flag and a prefix→command resolver mapping. +func TestLoadConfig_ParsesResolversAndFlag(t *testing.T) { + t.Parallel() + dir := t.TempDir() + body := "" + + "journey_integrity: true\n" + + "resolvers:\n" + + " \"edge:\":\n" + + " command: [\"scripts/resolve-edge.sh\", \"--graph\", \".seamline/architecture.detail.json\"]\n" + require.NoError(t, os.WriteFile(filepath.Join(dir, ".actrace.yml"), []byte(body), 0o644)) + + c, err := loadConfig(dir) + require.NoError(t, err) + assert.True(t, c.JourneyIntegrity) + require.Contains(t, c.Resolvers, "edge:") + assert.Equal(t, + []string{"scripts/resolve-edge.sh", "--graph", ".seamline/architecture.detail.json"}, + c.Resolvers["edge:"].Command) +} + +// TestLoadConfig_MalformedIsError pins that a present-but-broken config is a +// hard error — a typo must not silently disable a gate the repo turned on. +func TestLoadConfig_MalformedIsError(t *testing.T) { + t.Parallel() + dir := t.TempDir() + require.NoError(t, os.WriteFile(filepath.Join(dir, ".actrace.yml"), []byte("journey_integrity: [not a bool\n"), 0o644)) + _, err := loadConfig(dir) + assert.Error(t, err) +} diff --git a/internal/actrace/journey.go b/internal/actrace/journey.go new file mode 100644 index 0000000..8435957 --- /dev/null +++ b/internal/actrace/journey.go @@ -0,0 +1,300 @@ +// SPDX-FileCopyrightText: Copyright 2026 Stacklok, Inc. +// SPDX-License-Identifier: LicenseRef-Stacklok-Proprietary + +package actrace + +import ( + "fmt" + "os" + "regexp" + "strconv" + "strings" +) + +// The two **Surface:** tag values from ADR-0077. A scenario is one or the +// other; user-facing wins when a scenario is conceptually both, so the author +// tags it user-facing. +const ( + surfaceUserFacing = "user-facing" + surfaceBackend = "backend-foundation" +) + +// surfaceLineRe matches a `**Surface:**` field line under a scenario heading, +// mirroring the shape of statusLineRE. The value is the rest of the line. +var surfaceLineRe = regexp.MustCompile(`(?i)^\s*[-*\s]*\*\*surface:?\*\*:?\s*(.*)$`) + +// journeyOkRe captures the journey-proof opt-out `journey-ok: ` on a +// verify line. It grandfathers a pre-existing user-facing scenario that has no +// journey proof yet — but only when the reason cites a tracked issue, so the +// debt is visible, attributed, and expiring, never a silent pass (ADR-0077). +var journeyOkRe = regexp.MustCompile(`journey-ok:\s*(\S[^\n;]*)`) + +// issueRefRe matches a tracked-issue reference — a bare `#123` or an issues URL. +var issueRefRe = regexp.MustCompile(`#\d+|/issues/\d+`) + +// isWeakMethod reports whether a verify method is one a user-facing AC may not +// hide behind (ADR-0077, L3): demonstration / scenario / manual. The `none` and +// `inspection` methods stay valid — a closed-by-construction or reviewed +// criterion is legitimate on any surface. +func isWeakMethod(verify string) bool { + fields := strings.Fields(verify) + if len(fields) == 0 { + return false + } + switch strings.ToLower(strings.Trim(fields[0], "`*_")) { + case "demonstration", "scenario", "manual": + return true + default: + return false + } +} + +// scenarioNum extracts the scenario number from an ACx.y id: "AC12.3" → 12. +// A malformed id yields 0, which groups it under a distinct "scenario 0" whose +// missing-tag failure surfaces the malformed numbering. +func scenarioNum(acID string) int { + s := strings.TrimPrefix(acID, "AC") + if i := strings.IndexByte(s, '.'); i >= 0 { + s = s[:i] + } + n, _ := strconv.Atoi(s) + return n +} + +// parseSurfaceByAC binds each ACx.y id to the `**Surface:**` value in effect +// when it appears — the field is positional, sitting under its scenario +// heading before the scenario's ACs. It also returns "orphan" messages for a +// `**Surface:**` line with no following AC before the next Surface line or +// end of file, which is a mis-placed tag (ADR-0077, AC2.8). +func parseSurfaceByAC(lines []string) (byAC map[string]string, orphans []string) { + byAC = map[string]string{} + current := "" + consumed := true // did an AC consume the current surface value? + flushOrphan := func() { + if current != "" && !consumed { + orphans = append(orphans, fmt.Sprintf("**Surface:** %q has no following ACx.y block", current)) + } + } + for _, ln := range lines { + if m := surfaceLineRe.FindStringSubmatch(ln); m != nil { + flushOrphan() + current = strings.ToLower(strings.TrimSpace(m[1])) + consumed = false + continue + } + if m := acLineRe.FindStringSubmatch(ln); m != nil { + byAC[m[1]] = current + consumed = true + } + } + flushOrphan() + return byAC, orphans +} + +// buildConstraintLine returns the `//go:build ...` constraint of a Go source +// file, or "" if it has none. +func buildConstraintLine(src string) string { + for _, ln := range strings.Split(src, "\n") { + t := strings.TrimSpace(ln) + if strings.HasPrefix(t, "//go:build ") { + return t + } + // The build constraint sits at the top; once real code starts, stop. + if t != "" && !strings.HasPrefix(t, "//") && !strings.HasPrefix(t, "package") { + break + } + } + return "" +} + +// goTestIsRealCluster reports whether the Go test file at path is a real-cluster +// e2e test: it lives under test/e2e/, its build constraint includes `e2e` but +// not `synthetic`, and it does not import idpfake. A `synthetic` subprocess +// test or one that mints its own token via idpfake fakes the seam it claims to +// cross, so it is not a journey proof (ADR-0077 substance rule). +func goTestIsRealCluster(path string) bool { + if !strings.Contains(path, "test/e2e/") { + return false + } + b, err := os.ReadFile(path) //nolint:gosec // path comes from the test-declaration index, not user input + if err != nil { + return false + } + src := string(b) + tag := buildConstraintLine(src) + if !strings.Contains(tag, "e2e") || strings.Contains(tag, "synthetic") { + return false + } + return !strings.Contains(src, "idpfake") +} + +// specHasNetworkAssertion reports whether a Playwright spec asserts on a real +// server response rather than on rendered DOM alone. The signal is a +// `waitForResponse` (or `waitForResponseEvent`) call — a spec that only checks +// the DOM renders fine against a mock regardless of the backend, which is how +// #483 / #508 shipped green (ADR-0077 substance rule). +func specHasNetworkAssertion(path string) bool { + b, err := os.ReadFile(path) //nolint:gosec // path comes from the front-end spec index, not user input + if err != nil { + return false + } + return strings.Contains(string(b), "waitForResponse") +} + +// lookupTestPath returns the file declaring a cited Go test. It handles a +// fully-qualified name and a family stem ("TestFoo_"), mirroring resolve(). +func lookupTestPath(cite string, index map[string]string) (string, bool) { + if p, ok := index[cite]; ok { + return p, true + } + if strings.HasSuffix(cite, "_") || strings.HasSuffix(cite, "*") { + base := strings.TrimRight(cite, "_*") + for name, p := range index { + if strings.HasPrefix(name, base+"_") { + return p, true + } + } + } + return "", false +} + +// isJourneyProof reports whether an AC's verify cites at least one journey +// proof. Browser surface: a `ui-e2e-realfd:` token resolving to a +// .realfd.spec.ts spec that asserts on a real server response. Backend surface: +// a Go test under test/e2e/ that is a real-cluster run. Both are gated on +// substance, not location alone (ADR-0077). +func isJourneyProof(e acEntry, fe feIndex, index map[string]string) bool { + for _, tok := range uiCiteRe.FindAllString(e.verify, -1) { + if prefix, _ := parseFECitation(tok); prefix != fePrefixRealfd { + continue + } + hits := fe.matches(tok) + if len(hits) == 1 && strings.HasSuffix(hits[0], ".realfd.spec.ts") && specHasNetworkAssertion(hits[0]) { + return true + } + } + for _, name := range testCiteRe.FindAllString(e.verify, -1) { + if path, ok := lookupTestPath(name, index); ok && goTestIsRealCluster(path) { + return true + } + } + return false +} + +// scenarioGroup accumulates one scenario's ACs while the plan is scanned. +type scenarioGroup struct { + surfaces map[string]bool + hasJourney bool + sawOptOut bool // a journey-ok: token appeared on some AC in the scenario + optOutValid bool // and that opt-out cited a tracked issue +} + +// checkJourneyIntegrity is the ADR-0077 gate, run on a landed plan only when +// `.actrace.yml` sets journey_integrity. It binds each scenario's +// `**Surface:**` tag, then requires every user-facing scenario to carry at +// least one AC citing a journey proof. It hard-fails a missing tag, an +// unrecognised tag value, an orphan tag line, and a scenario whose ACs carry +// conflicting tags (ambiguous grouping). Returns the failure count. +func checkJourneyIntegrity(lines []string, acs []acEntry, fe feIndex, index map[string]string) int { + surfByAC, orphans := parseSurfaceByAC(lines) + failures := 0 + for _, msg := range orphans { + failures++ + fmt.Printf(" ✗ %s\n", msg) + } + failures += checkWeakMethods(acs, surfByAC) + groups, order := groupScenarios(acs, surfByAC, fe, index) + for _, n := range order { + failures += evaluateScenario(n, groups[n]) + } + return failures +} + +// checkWeakMethods reports every user-facing AC proven by a weak method +// (demonstration / scenario / manual), which ADR-0077 forbids on a user-facing +// surface. Returns the failure count. +func checkWeakMethods(acs []acEntry, surfByAC map[string]string) int { + failures := 0 + for _, e := range acs { + if surfByAC[e.id] == surfaceUserFacing && isWeakMethod(e.verify) { + failures++ + fmt.Printf(" ✗ %s user-facing AC cannot be proven by a demonstration/scenario/manual method\n", e.id) + } + } + return failures +} + +// groupScenarios buckets ACs by scenario number, recording each scenario's +// tag(s), whether any AC cites a journey proof, and whether a journey-ok: +// opt-out (and a valid, issue-citing one) appeared. +func groupScenarios( + acs []acEntry, surfByAC map[string]string, fe feIndex, index map[string]string, +) (map[int]*scenarioGroup, []int) { + groups := map[int]*scenarioGroup{} + var order []int + for _, e := range acs { + n := scenarioNum(e.id) + g := groups[n] + if g == nil { + g = &scenarioGroup{surfaces: map[string]bool{}} + groups[n] = g + order = append(order, n) + } + g.surfaces[surfByAC[e.id]] = true + if isJourneyProof(e, fe, index) { + g.hasJourney = true + } + if m := journeyOkRe.FindStringSubmatch(e.verify); m != nil { + g.sawOptOut = true + if issueRefRe.MatchString(m[1]) { + g.optOutValid = true + } + } + } + return groups, order +} + +// evaluateScenario returns the failures for one scenario's tag: a conflicting, +// missing, or unrecognised **Surface:** value, or (for a user-facing scenario) +// a missing journey proof without a valid opt-out. +func evaluateScenario(n int, g *scenarioGroup) int { + if len(g.surfaces) > 1 { + fmt.Printf(" ✗ scenario %d: ACs carry conflicting **Surface:** values — grouping is ambiguous\n", n) + return 1 + } + var surf string + for s := range g.surfaces { + surf = s + } + switch surf { + case "": + fmt.Printf(" ✗ scenario %d: no **Surface:** tag (add user-facing or backend-foundation)\n", n) + return 1 + case surfaceUserFacing: + return evaluateUserFacing(n, g) + case surfaceBackend: + return 0 // no journey proof required + default: + fmt.Printf(" ✗ scenario %d: unrecognised **Surface:** value %q (want user-facing or backend-foundation)\n", n, surf) + return 1 + } +} + +// evaluateUserFacing returns the failures for a user-facing scenario: a +// malformed opt-out, or a missing journey proof with no valid opt-out. A valid +// issue-citing opt-out grandfathers the scenario (deferred, tracked debt). +func evaluateUserFacing(n int, g *scenarioGroup) int { + switch { + case g.sawOptOut && !g.optOutValid: + fmt.Printf(" ✗ scenario %d: journey-ok: opt-out must cite a tracked issue (e.g. #692)\n", n) + return 1 + case g.optOutValid: + return 0 + case !g.hasJourney: + fmt.Printf(" ✗ scenario %d: user-facing but no AC cites a journey proof "+ + "(a ui-e2e-realfd: spec with a real-response assertion, or a test/e2e/ real-cluster Go test)\n", n) + return 1 + default: + return 0 + } +} diff --git a/internal/actrace/journey_test.go b/internal/actrace/journey_test.go new file mode 100644 index 0000000..438cbc8 --- /dev/null +++ b/internal/actrace/journey_test.go @@ -0,0 +1,153 @@ +// SPDX-FileCopyrightText: Copyright 2026 Stacklok, Inc. +// SPDX-License-Identifier: LicenseRef-Stacklok-Proprietary + +package actrace + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// journeyFixture writes a tree of spec / test files and returns the front-end +// spec index and the Go-test-declaration index built from it. The paths in +// both indexes are absolute (the walk root is a temp dir), and the substance +// readers os.ReadFile those paths, so the fixture exercises the real +// file-reading substance checks. +func journeyFixture(t *testing.T) (feIndex, map[string]string) { + t.Helper() + files := map[string]string{ + // Browser journey proofs. + "ui/tests/e2e/chat.realfd.spec.ts": "test('x', async ({page}) => { await page.waitForResponse('**/v1/responses'); });", + "ui/tests/e2e/domonly.realfd.spec.ts": "test('x', async ({page}) => { await expect(page.getByText('hi')).toBeVisible(); });", + "ui/tests/e2e/mock.spec.ts": "test('x', async ({page}) => {});", + "ui/src/bubble.test.ts": "test('x', () => {});", + // Backend journey proof: real-cluster e2e. + "test/e2e/files_test.go": "//go:build e2e\n\npackage e2e_test\nimport \"testing\"\nfunc TestE2E_Files(t *testing.T) {}\n", + // Faked seams under test/e2e/. + "test/e2e/demo_test.go": "//go:build e2e && synthetic\n\npackage e2e_test\nimport \"testing\"\nfunc TestE2E_Demo(t *testing.T) {}\n", + "test/e2e/idp_test.go": "//go:build e2e\n\npackage e2e_test\nimport (\n\t\"testing\"\n\t_ \"x/internal/testing/idpfake\"\n)\nfunc TestE2E_Idp(t *testing.T) {}\n", + // A Go test outside test/e2e/. + "internal/foo/foo_test.go": "package foo\nimport \"testing\"\nfunc TestFoo_Bar(t *testing.T) {}\n", + } + dir := t.TempDir() + for rel, content := range files { + p := filepath.Join(dir, rel) + require.NoError(t, os.MkdirAll(filepath.Dir(p), 0o755)) + require.NoError(t, os.WriteFile(p, []byte(content), 0o644)) + } + index, fe, err := indexTestDecls(dir) + require.NoError(t, err) + return fe, index +} + +// TestADR_0077_JourneyProofIsLocationGated pins what counts as a journey proof: +// a substantive real-FD spec or a real-cluster test/e2e Go test — and what does +// not: a mock e2e spec, a unit spec, or a Go test outside test/e2e/. +func TestADR_0077_JourneyProofIsLocationGated(t *testing.T) { + t.Parallel() + fe, index := journeyFixture(t) + tests := []struct { + name string + verify string + want bool + }{ + {"substantive realfd spec", "ui-e2e-realfd:chat.realfd", true}, + {"real-cluster e2e go test", "TestE2E_Files", true}, + {"mock e2e spec is not a journey", "ui-e2e:mock", false}, + {"unit spec is not a journey", "ui-unit:bubble", false}, + {"go test outside test/e2e is not a journey", "TestFoo_Bar", false}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + got := isJourneyProof(acEntry{id: "AC1.1", verify: tc.verify}, fe, index) + assert.Equal(t, tc.want, got) + }) + } +} + +// TestADR_0077_JourneyProofRejectsFakedSeam pins the substance rule: a proof in +// the right location that fakes the seam is rejected — a DOM-only realfd spec, a +// synthetic-tagged test/e2e test, and an idpfake-importing test/e2e test. +func TestADR_0077_JourneyProofRejectsFakedSeam(t *testing.T) { + t.Parallel() + fe, index := journeyFixture(t) + tests := []struct { + name string + verify string + }{ + {"DOM-only realfd spec (no waitForResponse)", "ui-e2e-realfd:domonly.realfd"}, + {"synthetic-tagged test/e2e", "TestE2E_Demo"}, + {"idpfake-importing test/e2e", "TestE2E_Idp"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + assert.False(t, isJourneyProof(acEntry{id: "AC1.1", verify: tc.verify}, fe, index), + "a faked seam must not satisfy the journey-proof requirement") + }) + } +} + +// TestADR_0077_UserFacingScenarioRequiresJourneyProof pins the scenario gate: a +// user-facing scenario needs a journey-proof AC; a backend-foundation scenario +// does not; a missing / unrecognised / ambiguous / orphan tag is a hard +// failure. +func TestADR_0077_UserFacingScenarioRequiresJourneyProof(t *testing.T) { //nolint:paralleltest // checkJourneyIntegrity prints; captureOutput swaps os.Stdout globally + fe, index := journeyFixture(t) + tests := []struct { + name string + body string + want int + }{ + { + name: "user-facing with a journey proof passes", + body: "**Surface:** user-facing\n" + + "- AC1.1: renders.\n - verify: ui-e2e-realfd:chat.realfd\n", + want: 0, + }, + { + name: "user-facing with only a mock proof fails", + body: "**Surface:** user-facing\n" + + "- AC1.1: renders.\n - verify: ui-e2e:mock\n", + want: 1, + }, + { + name: "backend-foundation needs no journey proof", + body: "**Surface:** backend-foundation\n" + + "- AC1.1: wiring holds.\n - verify: `TestFoo_Bar`\n", + want: 0, + }, + { + name: "missing tag fails", + body: "- AC1.1: renders.\n - verify: ui-e2e:mock\n", + want: 1, + }, + { + name: "unrecognised tag value (the unedited menu) fails", + body: "**Surface:** user-facing | backend-foundation\n" + + "- AC1.1: renders.\n - verify: ui-e2e-realfd:chat.realfd\n", + want: 1, + }, + { + name: "orphan tag with no following AC fails", + body: "**Surface:** user-facing\n**Surface:** backend-foundation\n" + + "- AC1.1: wiring.\n - verify: `TestFoo_Bar`\n", + want: 1, + }, + } + for _, tc := range tests { //nolint:paralleltest,tparallel // captureOutput redirects os.Stdout globally + t.Run(tc.name, func(t *testing.T) { + lines := strings.Split(tc.body, "\n") + acs := parseACs(lines) + var got int + out := captureOutput(func() { got = checkJourneyIntegrity(lines, acs, fe, index) }) + assert.Equal(t, tc.want, got, "output:\n%s", out) + }) + } +} diff --git a/internal/actrace/resolver.go b/internal/actrace/resolver.go new file mode 100644 index 0000000..0d7ce87 --- /dev/null +++ b/internal/actrace/resolver.go @@ -0,0 +1,112 @@ +// SPDX-FileCopyrightText: Copyright 2026 Stacklok, Inc. +// SPDX-License-Identifier: LicenseRef-Stacklok-Proprietary + +package actrace + +import ( + "errors" + "fmt" + "os" + "os/exec" + "regexp" + "strings" +) + +// resolverTokenRe captures a custom verify-method token: a lowercase prefix +// then a colon then a value, e.g. `edge:agentloop->files.GetFile`. The value +// runs to the next space, comma, or semicolon. A Go test name (starts +// uppercase) and a bare method word (no colon) do not match. A URL is excluded +// separately (it contains "://"), and the built-in `ui*:` front-end prefixes +// are handled by checkUIRefs, so they are skipped here. +var resolverTokenRe = regexp.MustCompile(`\b[a-z][a-z0-9-]{1,20}:[^\s,;]+`) + +// isBuiltinFEToken reports whether a token is one of the built-in front-end +// proof prefixes, which checkUIRefs already owns. +func isBuiltinFEToken(tok string) bool { + for _, p := range []string{"ui-e2e-realfd:", "ui-e2e:", "ui-unit:", "ui:"} { + if strings.HasPrefix(tok, p) { + return true + } + } + return false +} + +// resolverTokens returns the custom-prefix tokens in a verify field that an +// external resolver should decide. It excludes Go test names, built-in +// front-end references, and URLs (a `://` token is a link, not a proof). +func resolverTokens(verify string) []string { + var out []string + for _, tok := range resolverTokenRe.FindAllString(verify, -1) { + // Skip built-in front-end refs (checkUIRefs owns them), URLs, and + // opt-out markers like `ui-unit-ok:` / `journey-ok:` (a prefix ending + // in "-ok:"), which are declarations, not proofs to resolve. + if isBuiltinFEToken(tok) || strings.Contains(tok, "://") || strings.HasSuffix(tokenPrefix(tok), "-ok:") { + continue + } + out = append(out, tok) + } + return out +} + +// tokenPrefix returns the method prefix of a resolver token, including the +// colon: "edge:agentloop->files.GetFile" → "edge:". +func tokenPrefix(tok string) string { + if i := strings.IndexByte(tok, ':'); i >= 0 { + return tok[:i+1] + } + return tok +} + +// runResolver invokes a configured resolver command with the raw verify token +// as a single, final argv element. It never runs the token through a shell, so +// an author-controlled token cannot inject a command (CWE-78 / CWE-88). A +// zero exit means the proof holds; a non-zero exit means it does not; any other +// failure (command not found, not executable) is returned as an error so the +// gate reports it rather than treating it as a silent "no". +func runResolver(command []string, token string) (satisfied bool, err error) { + if len(command) == 0 { + return false, errors.New("empty resolver command") + } + argv := append(append([]string{}, command[1:]...), token) + // #nosec G204 -- command comes from the repo's own reviewed .actrace.yml, + // and token is passed as one discrete argv element (no shell), so a + // plan-authored token cannot inject a command. + cmd := exec.Command(command[0], argv...) + cmd.Stderr = os.Stderr + runErr := cmd.Run() + if runErr == nil { + return true, nil + } + var exitErr *exec.ExitError + if errors.As(runErr, &exitErr) { + return false, nil // clean non-zero exit: the proof does not hold + } + return false, fmt.Errorf("running resolver %q: %w", command[0], runErr) +} + +// checkResolverRefs resolves every custom-prefix verify token in an AC through +// its configured resolver. A token whose prefix has no resolver in .actrace.yml +// is a hard failure — never a silent no-op — which is how an `edge:` proof is +// rejected in a repo that has not wired the resolver. Returns the failure count. +func checkResolverRefs(e acEntry, cfg Config) int { + failures := 0 + for _, tok := range resolverTokens(e.verify) { + prefix := tokenPrefix(tok) + rc, ok := cfg.Resolvers[prefix] + if !ok { + failures++ + fmt.Printf(" ✗ %s verify names %s — no resolver configured for prefix %q in .actrace.yml\n", e.id, tok, prefix) + continue + } + satisfied, err := runResolver(rc.Command, tok) + switch { + case err != nil: + failures++ + fmt.Printf(" ✗ %s verify names %s — resolver error: %v\n", e.id, tok, err) + case !satisfied: + failures++ + fmt.Printf(" ✗ %s verify names %s — resolver rejected it (proof does not hold)\n", e.id, tok) + } + } + return failures +} diff --git a/internal/actrace/resolver_test.go b/internal/actrace/resolver_test.go new file mode 100644 index 0000000..aadffb1 --- /dev/null +++ b/internal/actrace/resolver_test.go @@ -0,0 +1,98 @@ +// SPDX-FileCopyrightText: Copyright 2026 Stacklok, Inc. +// SPDX-License-Identifier: LicenseRef-Stacklok-Proprietary + +package actrace + +import ( + "os" + "path/filepath" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +// TestADR_0077_EdgeVerifyRejectedWithoutResolver pins the no-silent-no-op rule: +// an `edge:` (custom-prefix) verify token in a repo that has configured no +// resolver for that prefix is a hard failure, not a pass. Before this gate an +// unrecognised token resolved to zero test citations and slipped through green. +func TestADR_0077_EdgeVerifyRejectedWithoutResolver(t *testing.T) { //nolint:paralleltest // checkResolverRefs prints to os.Stdout; captureOutput swaps it globally + e := acEntry{id: "AC1.1", verify: "edge:agentloop->files.GetFile", hasVerify: true} + + var failures int + out := captureOutput(func() { failures = checkResolverRefs(e, Config{}) }) + + assert.Equal(t, 1, failures, "an edge: token with no configured resolver must fail") + assert.Contains(t, out, "no resolver configured for prefix \"edge:\"") +} + +// TestADR_0077_ResolverTokenPassedAsArgv pins the injection-safety rule +// (CWE-78 / CWE-88): ac-trace passes the raw verify token to the resolver as a +// single argv element and never through a shell. A token carrying shell +// metacharacters must reach the resolver verbatim as $1, and must not be +// interpreted — no side effect fires. +func TestADR_0077_ResolverTokenPassedAsArgv(t *testing.T) { + t.Parallel() + tmp := t.TempDir() + captured := filepath.Join(tmp, "captured") + injected := filepath.Join(tmp, "INJECTED") + script := filepath.Join(tmp, "resolver.sh") + // The resolver records its first argument verbatim and exits 0. + require.NoError(t, os.WriteFile(script, + []byte("#!/bin/sh\nprintf '%s' \"$1\" > "+captured+"\n"), 0o755)) //nolint:gosec // test fixture + + // A token that WOULD run `touch INJECTED` if it ever hit a shell. + token := "edge:agentloop->files.GetFile`touch " + injected + "`" + + satisfied, err := runResolver([]string{"sh", script}, token) + require.NoError(t, err) + assert.True(t, satisfied, "resolver exited 0, so the proof holds") + + got, err := os.ReadFile(captured) + require.NoError(t, err) + assert.Equal(t, token, string(got), "the token must arrive as one discrete argv element, verbatim") + + _, statErr := os.Stat(injected) + assert.True(t, os.IsNotExist(statErr), "no shell interpretation: the injection side effect must not fire") +} + +// TestResolverTokens_FiltersBuiltinsAndURLs checks token extraction: a +// custom-prefix token is picked up, a built-in ui* front-end reference is left +// to checkUIRefs, a Go test name is not a resolver token, and a URL is skipped. +func TestResolverTokens_FiltersBuiltinsAndURLs(t *testing.T) { + t.Parallel() + tests := []struct { + name string + verify string + want []string + }{ + {"edge token", "edge:agentloop->files.GetFile", []string{"edge:agentloop->files.GetFile"}}, + {"edge alongside a go test", "TestFoo_Bar, edge:a->b.C", []string{"edge:a->b.C"}}, + {"builtin fe prefixes ignored", "ui-e2e-realfd:chat, ui-unit:bubble", nil}, + {"go test only", "TestADR_0077_Foo", nil}, + {"url ignored", "demonstration — see https://example.com/x", nil}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + t.Parallel() + assert.Equal(t, tc.want, resolverTokens(tc.verify)) + }) + } +} + +// TestRunResolver_ExitCodeMapsToSatisfied pins the exit-code contract: exit 0 +// means the proof holds, a clean non-zero exit means it does not (no error), +// and an unrunnable command is an error the gate surfaces. +func TestRunResolver_ExitCodeMapsToSatisfied(t *testing.T) { + t.Parallel() + ok, err := runResolver([]string{"true"}, "edge:a->b.C") + require.NoError(t, err) + assert.True(t, ok) + + notOk, err := runResolver([]string{"false"}, "edge:a->b.C") + require.NoError(t, err, "a clean non-zero exit is a 'no', not an error") + assert.False(t, notOk) + + _, err = runResolver([]string{filepath.Join(t.TempDir(), "does-not-exist")}, "edge:a->b.C") + assert.Error(t, err, "an unrunnable resolver command is a surfaced error") +} diff --git a/internal/actrace/weakmethod_test.go b/internal/actrace/weakmethod_test.go new file mode 100644 index 0000000..42a9b7f --- /dev/null +++ b/internal/actrace/weakmethod_test.go @@ -0,0 +1,94 @@ +// SPDX-FileCopyrightText: Copyright 2026 Stacklok, Inc. +// SPDX-License-Identifier: LicenseRef-Stacklok-Proprietary + +package actrace + +import ( + "strings" + "testing" + + "github.com/stretchr/testify/assert" +) + +// TestADR_0077_UserFacingRejectsWeakMethods pins L3: a user-facing AC may not +// be proven by demonstration / scenario / manual. The same method stays valid +// on a backend-foundation AC, and none / inspection stay valid everywhere. +func TestADR_0077_UserFacingRejectsWeakMethods(t *testing.T) { //nolint:paralleltest // captureOutput swaps os.Stdout globally + fe, index := journeyFixture(t) + tests := []struct { + name string + body string + want int + }{ + { + name: "user-facing AC proven by demonstration fails", + body: "**Surface:** user-facing\n" + + "- AC1.1: renders.\n - verify: ui-e2e-realfd:chat.realfd\n" + + "- AC1.2: also holds.\n - verify: demonstration — hand-checked in staging\n", + want: 1, + }, + { + name: "backend-foundation AC proven by demonstration passes", + body: "**Surface:** backend-foundation\n" + + "- AC1.1: wiring holds.\n - verify: demonstration — the gate is the worked example\n", + want: 0, + }, + { + name: "user-facing AC proven by inspection is fine", + body: "**Surface:** user-facing\n" + + "- AC1.1: renders.\n - verify: ui-e2e-realfd:chat.realfd\n" + + "- AC1.2: closed by construction.\n - verify: inspection — no producer path exists\n", + want: 0, + }, + } + for _, tc := range tests { //nolint:paralleltest,tparallel // captureOutput redirects os.Stdout globally + t.Run(tc.name, func(t *testing.T) { + lines := strings.Split(tc.body, "\n") + acs := parseACs(lines) + var got int + out := captureOutput(func() { got = checkJourneyIntegrity(lines, acs, fe, index) }) + assert.Equal(t, tc.want, got, "output:\n%s", out) + }) + } +} + +// TestADR_0077_GrandfatherOptOutRequiresIssue pins the opt-out rule: a +// pre-existing user-facing scenario with no journey proof is grandfathered only +// when its journey-ok: opt-out cites a tracked issue. An opt-out with no issue +// reference is a hard failure. +func TestADR_0077_GrandfatherOptOutRequiresIssue(t *testing.T) { //nolint:paralleltest // captureOutput swaps os.Stdout globally + fe, index := journeyFixture(t) + tests := []struct { + name string + body string + want int + }{ + { + name: "opt-out citing an issue number grandfathers the scenario", + body: "**Surface:** user-facing\n" + + "- AC1.1: renders.\n - verify: ui-e2e:mock; journey-ok: real-FD migration tracked in #692\n", + want: 0, + }, + { + name: "opt-out citing an issues URL grandfathers the scenario", + body: "**Surface:** user-facing\n" + + "- AC1.1: renders.\n - verify: ui-e2e:mock; journey-ok: see https://github.com/stacklok/atrium/issues/692\n", + want: 0, + }, + { + name: "opt-out with no issue reference fails", + body: "**Surface:** user-facing\n" + + "- AC1.1: renders.\n - verify: ui-e2e:mock; journey-ok: migrate later\n", + want: 1, + }, + } + for _, tc := range tests { //nolint:paralleltest,tparallel // captureOutput redirects os.Stdout globally + t.Run(tc.name, func(t *testing.T) { + lines := strings.Split(tc.body, "\n") + acs := parseACs(lines) + var got int + out := captureOutput(func() { got = checkJourneyIntegrity(lines, acs, fe, index) }) + assert.Equal(t, tc.want, got, "output:\n%s", out) + }) + } +}