diff --git a/README.ko.md b/README.ko.md index 6278c8e..bfb1727 100644 --- a/README.ko.md +++ b/README.ko.md @@ -82,7 +82,7 @@ $EDITOR .sdlc/config.md # build, test, lint, run, e2e, PLAN.md 한 장: 이해, AS-IS → TO-BE, 부작용, 변경, 시나리오 review.md 리뷰 라운드, 각 라운드는 VERDICT로 끝남 evidence.md 검사, 시나리오별 결과 한 행, 납품 기록 - logs/ 모든 PASS·FAIL 뒤의 원본 출력 + logs/ 모든 PASS·FAIL 뒤의 원본 출력, scripts/는 시나리오별 재실행 스크립트 ``` 끝난 기능은 `.sdlc/archive/`로 옮겨집니다. 검증된 사실은 `knowledge` 폴더로 diff --git a/README.md b/README.md index a82d46b..b90dcac 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ always says what comes next. PLAN.md one page: understanding, AS-IS → TO-BE, side effects, changes, scenarios review.md reviewer rounds, each ending in VERDICT evidence.md checks, one result row per scenario, delivery - logs/ raw output behind every PASS and FAIL + logs/ raw output behind every PASS and FAIL; scripts/ re-runs each scenario ``` Closed features move to `.sdlc/archive/`. Verified facts move to the diff --git a/VERSION b/VERSION index 276cbf9..197c4d5 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.3.0 +2.4.0 diff --git a/log/changelog-v2.4.0.md b/log/changelog-v2.4.0.md new file mode 100644 index 0000000..cd54642 --- /dev/null +++ b/log/changelog-v2.4.0.md @@ -0,0 +1,41 @@ +# v2.4.0 — find consumers before QA does, cheaper QA rounds + +Lessons from one feature (a front-end route restructure) that took three +review rounds and three QA rounds. Each QA round re-drove 18 to 20 browser +scenarios by hand (45 and 38 minutes); two of its three code defects were +knowable before QA. No script change: the rules moved. + +## Changes + +- **Consumers of changed identifiers.** The change turned list routes into + redirects. A privacy-audit logger still compared the old route names, so + its log stopped silently; the plan missed it and the first review called + the old references harmless. The logic scout now lists every consumer of + an identifier the change renames, retires or redirects — searching the + literal value too, and counting consumers that only compare it (loggers, + audit and analytics events, permission checks, watchers, flags, links from + other apps). PLAN.md's side effects cover each; the reviewer blocks a plan + that skips them and a diff that leaves one comparing the old value. +- **Observable review findings become scenarios.** A finding the first + review rated non-blocking (a redirect missing a required parameter) was + only driven in QA round 2, where it failed and cost a loop. A non-blocking + finding a screen, API, job or log could show now ends in `(observable)`, + and QA runs each as a scenario in the first round. +- **QA preflight.** Three environment faults were each found mid-run: a dev + proxy pointing at a local API that was not running, a dev server killed by + its host's 30-minute background limit, and a browser driver that cannot + start in a sandboxed peer agent. QA now checks once, before any scenario, + that one real call per route reaches the intended environment, that the + driver starts, and that long-running processes outlive the run, and records + what it learned. +- **Re-runnable scenarios, cheaper rounds.** QA saves the script behind each + scenario in `logs/scripts/`. A later round re-runs every script against + the new commit and drives by hand only FAIL and BLOCKED rows, new rows and + rows the new diff touches, writing `logs/r-.txt` beside the old + logs. Every row still gets a run on the new commit. At delivery, scripts + worth keeping go to the project's `e2e` suite or into Delivery as a + follow-up. +- **A code FAIL is a bug fix.** Back from Evidence, each code FAIL gets its + regression test first, failing for the reported reason, as Implement + already asks of bug fixes. In that feature one fix went to review without + one and the review failed on exactly that. diff --git a/phases/2-plan.md b/phases/2-plan.md index fa56cf0..33a6cff 100644 --- a/phases/2-plan.md +++ b/phases/2-plan.md @@ -15,7 +15,7 @@ writing one file in `refs/`: |---|---|---| | source | `refs/source.md` | the request's text, the policies and canonical docs it touches, earlier attempts and reverts and why | | data | `refs/data.md` | data models and fields, who writes and reads each, the states existing records are in | -| logic | `refs/logic.md` | entry points, call chains, every caller in every tier and repository the change crosses, shared code | +| logic | `refs/logic.md` | entry points, call chains, every caller in every tier and repository the change crosses, shared code, every consumer of an identifier the change renames, retires or redirects | | flow | `refs/flow.md` | how each persona reaches the behavior today, observed by running it, with the result | A bug adds `refs/repro.md`: the failure reproduced — as a failing test where @@ -35,7 +35,8 @@ evidence. Link, do not copy. TO-BE is `unchanged`, with a scenario. - **Side effects** — every other reader or writer of the touched data, every caller that sends today's shapes (missing, empty, null, old values), - records made before the change, shared code — each with how it stays + records made before the change, shared code, every consumer of a changed + identifier (even one that only compares it) — each with how it stays correct and the scenario that checks it. Then the risk flags line. - **Changes** — files and symbols, in an order that keeps checks passing; the smallest runnable path first. diff --git a/phases/3-build.md b/phases/3-build.md index aa1f186..d5924f3 100644 --- a/phases/3-build.md +++ b/phases/3-build.md @@ -60,4 +60,6 @@ the loop: show the human the trail; only `sdlc.sh approve plan ## Back from Evidence `sdlc.sh next` names the scenarios that failed for code reasons; evidence.md -and `logs/` hold what was seen. Fix them, commit, and review again. +and `logs/` hold what was seen. Each is a bug fix: write its regression test +first and watch it fail for the reported reason, then fix, commit, and review +again. diff --git a/phases/4-evidence.md b/phases/4-evidence.md index 0acea00..a862f59 100644 --- a/phases/4-evidence.md +++ b/phases/4-evidence.md @@ -9,7 +9,10 @@ Dispatch QA (`roles/qa.md`) with intent.md, PLAN.md, `refs/`, config.md and the reviewed commit. QA runs the checks and every scenario, saves raw output in `logs/`, adds scenarios for anything new it finds, and fills `evidence.md` (`templates/evidence.md`): one row per scenario — PASS, FAIL with a cause, -or BLOCKED with what is missing. Scenarios are never dropped: `sdlc.sh next` +or BLOCKED with what is missing. QA preflights the environment once, runs each +`(observable)` review finding as a scenario, and saves the script behind +every scenario, so a later round re-runs them all and drives by hand only +what failed or what the new diff touches. Scenarios are never dropped: `sdlc.sh next` lists every ID the table needs, earlier QA-added ones included. A cause is `plan` when the plan was wrong or missed something (a caller, a data state, a rule, a scenario), and `code` when the plan was right and the @@ -50,6 +53,8 @@ Delivery's `Policy:` line. Then: - move verified facts from the refs (and `refs/learnings.md`, if any) into `knowledge`: one place per fact, updating a page rather than adding a duplicate; +- add the saved scripts (`logs/scripts/`) worth keeping to the project's + `e2e` suite, or name them as a follow-up in Delivery; - `sdlc.sh close shipped ""` — refused if the delivered tree differs from the verified one. diff --git a/roles/qa.md b/roles/qa.md index d18de0e..0238d0c 100644 --- a/roles/qa.md +++ b/roles/qa.md @@ -3,8 +3,15 @@ Prove whether the change works for each persona through the real interfaces. Report what you observed; fix nothing. -Inputs: intent.md, PLAN.md, `refs/`, config.md, and the commit(s) to verify. +Inputs: intent.md, PLAN.md, `refs/`, review.md, config.md, and the +commit(s) to verify. +0. Preflight the environment once, before any scenario: the app answers; + one real call through each route the scenarios use reaches the intended + environment (proxy, host, account); the driver starts in this sandbox; + anything long-running is started detached with a limit longer than the + run. Fix what you may, or mark the scenarios it blocks BLOCKED with the + cause, and put what you learned in `refs/learnings.md`. 1. Confirm the checked-out commit is the one you were given. Run `build`, `test`, `lint`, then `e2e` from config.md. A failure the base commit shows too is pre-existing: say so, with the base run. @@ -13,7 +20,9 @@ Inputs: intent.md, PLAN.md, `refs/`, config.md, and the commit(s) to verify. 3. For each PLAN.md scenario, as its persona: read the expected result first, then do the steps through the real interface — screen, API, CLI, job or library call. Read state back after every write. Save the raw output - (command, response, screenshot path, query result) to `logs/.txt`. + (command, response, screenshot path, query result) to `logs/.txt`, + and the script or command sequence that drove it to `logs/scripts/`, so + a later round re-runs it with one command. 4. A scenario that cannot run here is BLOCKED, with what is missing. Never PASS by assumption; never a unit test in place of the real interface. 5. Bug fix: run the regression test at the commit before the fix, in a @@ -21,12 +30,18 @@ Inputs: intent.md, PLAN.md, `refs/`, config.md, and the commit(s) to verify. for the reported reason. Then run it after, and re-drive the surface where the bug was reported. 6. Something the plan missed — another caller, a record state, a side - effect: add a row with the next free ID and run it. Besides your new + effect, an `(observable)` finding in review.md: add a row with the next + free ID and run it. Besides your new rows, the table holds exactly the IDs `sdlc.sh next ` lists: a missing one is refused, and so is a row for a scenario the human removed. Write `\|` for a literal pipe inside a cell. 7. Each FAIL gets a cause, with evidence: `plan` if the plan was wrong or missed something, `code` if the plan was right and the code is not. +8. A later round (after a return to Build or Plan) re-runs every saved + script against the new commit; drive by hand only the FAIL and BLOCKED + rows, new rows, and rows whose surface `git diff ..` touches. Write the new output beside the old + (`logs/r-.txt`); earlier logs stay. Fill `evidence.md` from `templates/evidence.md` and stop. If the feature folder is not writable from your sandbox (a symlink outside the allowed diff --git a/roles/reviewer.md b/roles/reviewer.md index e62b6d8..fc0b2d3 100644 --- a/roles/reviewer.md +++ b/roles/reviewer.md @@ -9,7 +9,9 @@ Inputs: intent.md, PLAN.md, `refs/`. Blocking: - a caller, data reader or writer, record state or tier the refs show but PLAN.md ignores; a side effect without a scenario; - AS-IS stated without observation; a risk flag set to no that the changes - trip (migration, data deletion, public API, security, infra). + trip (migration, data deletion, public API, security, infra); +- an identifier the plan renames, retires or redirects whose consumers + `refs/` do not list, or a listed consumer with no scenario. ## Diff mode (phase 3) @@ -17,7 +19,8 @@ Inputs: intent.md, PLAN.md, `refs/`, the commit or uncommitted diff (untracked files included). Read every changed file. Blocking: - it does not do what PLAN.md says, or does more; -- a side effect PLAN.md names is left unhandled; input that callers send +- a side effect PLAN.md names is left unhandled; a consumer that still + compares a renamed, retired or redirected value; input that callers send today is newly refused without intent.md authorizing it; - a policy changed with neither an update where it is written nor a recorded follow-up; @@ -46,5 +49,7 @@ Append to `review.md`: - VERDICT: PASS | FAIL -FAIL when anything blocks. Every finding carries its evidence. +FAIL when anything blocks. Every finding carries its evidence. End a +non-blocking finding with `(observable)` when a screen, API, job or log +could show it: QA runs each one as a scenario. Must not: write anything but `review.md`, or run mutating commands. diff --git a/roles/scout.md b/roles/scout.md index 32bd13a..0cdc5b9 100644 --- a/roles/scout.md +++ b/roles/scout.md @@ -11,6 +11,14 @@ and config.md. tier and repository, every reader and writer of the data, the states existing records are in. Query real data read-only where a tool and permission exist; redact personal data. +- Changed identifiers: for each name or value the change renames, retires, + redirects or reshapes (a route or path, an event, message or log key, an + enum value, a column, a config key, an exported symbol), search the literal + value as well as the symbol, in every repository, and list each consumer — + including those that only compare it: loggers, audit and analytics events, + permission checks, watchers, feature flags, links from other apps. One row + per consumer: file:line, what it does with the value, what happens to it + after the change. - Observe rather than infer: run the command, open the screen, call the API. For deployed behavior read the deployed revision (`git show :`), not the working tree. diff --git a/templates/config.md b/templates/config.md index 6062a69..6e60e53 100644 --- a/templates/config.md +++ b/templates/config.md @@ -5,7 +5,7 @@ kit: build: test: lint: -# start the app for end-to-end scenarios, and the project's own e2e command +# start the app for end-to-end scenarios (QA runs it detached, outliving the run), and the project's own e2e command run: e2e: # how QA drives the interface: a browser driver, an HTTP client, the shell; empty = whatever the agent has