diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 9c30c65..22e565a 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "allium", - "version": "3.12.0", + "version": "3.13.0", "description": "Velocity through clarity.", "author": { "name": "JUXT", diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index d191079..e8a6c3f 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "allium", - "version": "3.12.0", + "version": "3.13.0", "description": "Velocity through clarity.", "author": { "name": "JUXT", diff --git a/.github/agents/distill.agent.md b/.github/agents/distill.agent.md index 949df97..5793bb0 100644 --- a/.github/agents/distill.agent.md +++ b/.github/agents/distill.agent.md @@ -5,7 +5,7 @@ description: "Extract an Allium specification from an existing codebase. Use whe Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Scope the distillation from the goal you were given, record unconfirmed judgement calls as `open question` declarations in the distilled spec, and list the parked questions in your final output. -Reading the source code is your job precisely so it stays out of the caller's context: return the distilled spec's path, a short summary of what it covers, and the parked questions — not the code you read. +Reading the source code is your job precisely so it stays out of the caller's context. Return your result as a single JSON object conforming to the distill-result schema (see the skill's "Typed result" section) and nothing else — the spec path, a one-line summary of what it covers, and the parked questions as fields. Not the code you read, and no prose around the object. # Distillation guide @@ -843,6 +843,19 @@ If any remain, ask: "Would a stakeholder include this in a requirements doc?" The extracted spec is a starting point. If distillation reveals gaps that need structured discovery (unclear requirements, complex entity relationships, unstated business rules), use the `elicit` skill to fill them. For targeted changes as requirements evolve, use the `tend` skill. For checking ongoing alignment between the spec and implementation, use the `weed` skill. +## Typed result (loop hand-off) + +When running as the `distill` subagent inside the Allium loop, return your result as a single JSON object conforming to [distill-result.schema.json](../../skills/allium/references/schemas/distill-result.schema.json), and nothing else: the `spec_path`, the parked `open_questions`, and a one-line `summary` of what the spec covers. Emit every field, using `[]` for empty lists. The source you read stays out of the caller's context; only these fields come back. Running interactively, present your findings in prose as usual — the typed record is for the machine hand-off, not the conversation. + +```json +{ + "phase": "distill", + "spec_path": "giftcard.allium", + "open_questions": ["Is forcing an over-redeemed balance to zero intended or accidental?"], + "summary": "GiftCard redemption and status lifecycle" +} +``` + ## References - [Language reference](../../skills/allium/references/language-reference.md), full Allium syntax diff --git a/.github/agents/propagate.agent.md b/.github/agents/propagate.agent.md index 110f8da..2c9e4e9 100644 --- a/.github/agents/propagate.agent.md +++ b/.github/agents/propagate.agent.md @@ -5,7 +5,7 @@ description: "Generate tests from Allium specifications. Use when the user wants Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Report anything that needs a human decision in your final output and continue with the work that does not depend on it. You have full shell access because obligation reconciliation requires running the project's test command; use it for the allium CLI and test runs, not for modifying implementation code — implementation belongs to the loop's implement phase, not to you. -Return the generated test file paths, the reconciliation summary line (`N obligations, M covered, K uncovered`), and any uncovered obligations with their classification — not the file contents. +Return your result as a single JSON object conforming to the propagate-result schema (see the skill's "Typed result" section) and nothing else — the reconciliation counts, the uncovered obligations with their classification, and the generated tests with their hashes, all as fields. Not the file contents, and no prose around the object. # Propagation @@ -246,6 +246,22 @@ When running inside the Allium loop, record the reconciliation baseline so the l The hash is the ground truth the `witness` skill re-derives: a generated test whose hash changes with no intervening propagate run is a hand-edited test — the anti-cheat violation the loop must never reach convergence with. Recording the baseline is what makes that check possible; skip it and the witness can confirm the tests pass but not that they were not weakened. This is cheap bookkeeping, not a report — do not narrate it. +### Typed result (loop hand-off) + +When running as the `propagate` subagent inside the Allium loop, return your result as a single JSON object conforming to [propagate-result.schema.json](../../skills/allium/references/schemas/propagate-result.schema.json), and nothing else. The loop routes on the structured fields: `obligations` and `uncovered_obligations` decide whether coverage is complete (the loop must not converge while `uncovered_obligations` is non-empty), and `generated_tests` carries the path-and-hash baseline the witness re-derives — the same hashes recorded above, now first-class in the hand-off. Keep the `summary` field to the `N obligations, M covered, K uncovered` line. Emit every field, using `[]` for empty lists — do not omit them. Running interactively, the summary line remains the whole user-facing output as before — the typed record is for the machine hand-off, not the conversation. + +```json +{ + "phase": "propagate", + "obligations": { "total": 12, "covered": 12, "uncovered": 0 }, + "uncovered_obligations": [], + "generated_tests": [{ "path": "order.test.js", "hash": "sha256:9f2c…" }], + "test_paths": ["order.test.js"], + "open_questions": [], + "summary": "12 obligations, 12 covered, 0 uncovered" +} +``` + ## Interaction with other tools - **distill** produces specs from code. Those specs feed propagate. diff --git a/.github/agents/tend.agent.md b/.github/agents/tend.agent.md index b4746b9..ec81d33 100644 --- a/.github/agents/tend.agent.md +++ b/.github/agents/tend.agent.md @@ -3,7 +3,9 @@ name: tend description: "Tend the Allium garden. Use when the user wants to write, edit, update, add to, improve, clarify, refine, restructure, fix or migrate Allium specs. Covers adding entities, rules, triggers, surfaces and contracts, fixing syntax or validation errors, renaming or refactoring within specs, migrating specs to a new language version, and translating requirements into well-formed specifications. Pushes back on vague requirements." --- -Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Record anything that needs a human decision as an `open question` declaration in the spec, continue with the work that does not depend on it, and list the parked questions in your final output. +Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Record anything that needs a human decision as an `open question` declaration in the spec and continue with the work that does not depend on it. + +Return your result as a single JSON object conforming to the tend-result schema (see the skill's "Typed result" section) and nothing else — the spec path, the changes made, and the parked questions as fields. No prose around the object. # Tend @@ -106,3 +108,17 @@ After edits that change rules, surfaces or transition graphs, run `allium analys ## Output When proposing spec changes, explain the behavioural intent first, then show the changes. If you have questions or concerns about the request, raise them before writing anything. + +### Typed result (loop hand-off) + +When running as the `tend` subagent inside the Allium loop, return your result as a single JSON object conforming to [tend-result.schema.json](../../skills/allium/references/schemas/tend-result.schema.json), and nothing else: the `spec_path`, the `changes` you made (each a short string, not an object), the parked `open_questions`, and a one-line `summary`. Emit every field, using `[]` for empty lists. Running interactively, present the intent-then-changes prose above as usual — the typed record is for the machine hand-off, not the conversation. + +```json +{ + "phase": "tend", + "spec_path": "shop.allium", + "changes": ["Added expiry field to GiftCard", "Added GiftCardExpires temporal rule"], + "open_questions": ["Expiry period undecided"], + "summary": "Added gift card expiry behaviour" +} +``` diff --git a/.github/agents/weed.agent.md b/.github/agents/weed.agent.md index 63a19e0..8baf54f 100644 --- a/.github/agents/weed.agent.md +++ b/.github/agents/weed.agent.md @@ -3,7 +3,9 @@ name: weed description: "Weed the Allium garden. Find where Allium specifications and implementation code have diverged, and help resolve the divergences. Use when the user wants to check spec-code alignment, compare specs against implementation, audit for spec drift or violations, sync specs with code or code with specs, or verify whether the implementation matches what the spec says." --- -Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Report anything that needs a human decision as an open finding in your output (and, when updating the spec, as an `open question` declaration), then continue with the work that does not depend on it. +Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Report anything that needs a human decision as an open finding (and, when updating the spec, as an `open question` declaration), then continue with the work that does not depend on it. + +Return your result as a single JSON object conforming to the weed-result schema (see the skill's "Typed result" section) and nothing else — the loop routes on its fields, not on prose. # Weed @@ -113,3 +115,21 @@ Classification: [proposed classification with reasoning] ``` Group related divergences together. Lead with the most consequential findings. + +### Typed result (loop hand-off) + +When running as the `weed` subagent inside the Allium loop, return your result as a single JSON object conforming to [weed-result.schema.json](../../skills/allium/references/schemas/weed-result.schema.json), and nothing else. The loop routes on the structured fields (`verdict`, each divergence's `classification`, `open_questions`) rather than parsing prose, so the routing and the convergence check stay deterministic. Keep the `summary` field to the one human-readable line; put the detail in the structured fields. Emit every field, using `[]` for empty lists. Running interactively, present the prose format above as before — the typed record is for the machine hand-off, not the conversation. + +```json +{ + "phase": "weed", + "mode": "check", + "verdict": "dirty", + "divergences": [ + { "subject": "Order.cancel", "classification": "code-bug", "spec": "cancel allowed from paid (shop.allium:42)", "code": "guarded to pending only (order.py:88)" } + ], + "open_questions": [], + "artefacts": [], + "summary": "1 divergence: Order.cancel (code-bug)" +} +``` diff --git a/.github/agents/witness.agent.md b/.github/agents/witness.agent.md index 41eb25f..1d8a1b2 100644 --- a/.github/agents/witness.agent.md +++ b/.github/agents/witness.agent.md @@ -5,7 +5,7 @@ description: "Independently witness that an Allium loop's convergence claim is t Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Write the witness record, then report the verdict and every violation with its routing in your final output and continue — the caller acts on them. You have full shell access because independent verification requires re-running the project's test command, hashing the generated tests, and running the allium CLI; use it to re-derive the deterministic checks and read ground truth, never to modify the spec, the tests, or the code. Write only the witness record — everything else you read, hash or re-run. -Return the verdict, the one-line witness summary, every violation with its routing, and the record's path — not the file contents or the code you read. +Return your result as a single JSON object conforming to the witness-result schema (see the skill's "Output format" section) and nothing else — the verdict, each check with the ground truth it read, every violation with its routing, and the record path as fields. Not the file contents or the code you read, and no prose around the object. # Witness @@ -75,13 +75,33 @@ Do not embed file contents or code — the record holds verdicts and the evidenc ## Output format -Close with a single summary line the loop can fold into its report: +When running as the `witness` subagent inside the Allium loop, return your result as a single JSON object conforming to [witness-result.schema.json](../../skills/allium/references/schemas/witness-result.schema.json), and nothing else: the `verdict`, each `check` with the `ground_truth` it read, every `violation` with its `routing`, the `record_path`, and a one-line `summary`. Emit every field, using `[]` for an empty `violations` list on a PASS. The loop gates convergence on `verdict` directly — no prose to parse. The object mirrors the durable record you wrote to `.allium-loop/.witness.json`. + +```json +{ + "phase": "witness", + "verdict": "FAIL", + "checks": [ + { "name": "tests-pass", "result": "pass", "ground_truth": "runner exit 0, 12/12" }, + { "name": "no-test-weakened", "result": "fail", "ground_truth": "sha256 mismatch on order.test.js" } + ], + "violations": [ + { "violation": "order.test.js edited after propagate", "routing": "revert + propagate" } + ], + "record_path": ".allium-loop/gift-cards.witness.json", + "summary": "witness: FAIL · tampering on order.test.js" +} +``` + +As the loop subagent, return **only** that JSON object — no prose before or after it, even though you also wrote the durable record to disk. The returned object is your result; the file is its durable copy. + +Running interactively (not as the loop subagent), skip the JSON and close with a single human-readable summary line instead: ``` witness: PASS · checks 6/6 · tests 12/12 (runner) · tampering none · openQ 0 blocking · record .allium-loop/.witness.json ``` -On failure, lead with the verdict and the violations, each with its routing, then the record path. Keep the body to the verdict and its evidence — the record holds the detail. +On an interactive failure, lead with the verdict and the violations, each with its routing, then the record path. Keep the body to the verdict and its evidence — the record holds the detail. ## Interaction with other tools diff --git a/agents/distill.md b/agents/distill.md index 40a55ce..17294a1 100644 --- a/agents/distill.md +++ b/agents/distill.md @@ -19,4 +19,4 @@ You are the non-interactive entry point for the `distill` skill, whose content i Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Scope the distillation from the goal you were given, record unconfirmed judgement calls as `open question` declarations in the distilled spec, and list the parked questions in your final output. -Reading the source code is your job precisely so it stays out of the caller's context: return the distilled spec's path, a short summary of what it covers, and the parked questions — not the code you read. +Reading the source code is your job precisely so it stays out of the caller's context. Return your result as a single JSON object conforming to the distill-result schema (see the skill's "Typed result" section) and nothing else — the spec path, a one-line summary of what it covers, and the parked questions as fields. Not the code you read, and no prose around the object. diff --git a/agents/propagate.md b/agents/propagate.md index c55f410..2ae68f8 100644 --- a/agents/propagate.md +++ b/agents/propagate.md @@ -19,4 +19,4 @@ You are the non-interactive entry point for the `propagate` skill, whose content Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Report anything that needs a human decision in your final output and continue with the work that does not depend on it. You have full Bash access because obligation reconciliation requires running the project's test command; use it for the allium CLI and test runs, not for modifying implementation code — implementation belongs to the loop's implement phase, not to you. -Return the generated test file paths, the reconciliation summary line (`N obligations, M covered, K uncovered`), and any uncovered obligations with their classification — not the file contents. +Return your result as a single JSON object conforming to the propagate-result schema (see the skill's "Typed result" section) and nothing else — the reconciliation counts, the uncovered obligations with their classification, and the generated tests with their hashes, all as fields. Not the file contents, and no prose around the object. diff --git a/agents/tend.md b/agents/tend.md index 914840f..402ec94 100644 --- a/agents/tend.md +++ b/agents/tend.md @@ -17,4 +17,6 @@ skills: You are the non-interactive entry point for the `tend` skill, whose content is preloaded above. If it is not present, read `${CLAUDE_PLUGIN_ROOT}/skills/tend/SKILL.md` and follow it. Relative file references in the skill resolve from that directory. -Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Record anything that needs a human decision as an `open question` declaration in the spec, continue with the work that does not depend on it, and list the parked questions in your final output. +Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Record anything that needs a human decision as an `open question` declaration in the spec and continue with the work that does not depend on it. + +Return your result as a single JSON object conforming to the tend-result schema (see the skill's "Typed result" section) and nothing else — the spec path, the changes made, and the parked questions as fields. No prose around the object. diff --git a/agents/weed.md b/agents/weed.md index aa50fb5..68c0d1d 100644 --- a/agents/weed.md +++ b/agents/weed.md @@ -17,4 +17,6 @@ skills: You are the non-interactive entry point for the `weed` skill, whose content is preloaded above. If it is not present, read `${CLAUDE_PLUGIN_ROOT}/skills/weed/SKILL.md` and follow it. Relative file references in the skill resolve from that directory. -Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Report anything that needs a human decision as an open finding in your output (and, when updating the spec, as an `open question` declaration), then continue with the work that does not depend on it. +Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Report anything that needs a human decision as an open finding (and, when updating the spec, as an `open question` declaration), then continue with the work that does not depend on it. + +Return your result as a single JSON object conforming to the weed-result schema (see the skill's "Typed result" section) and nothing else — the loop routes on its fields, not on prose. diff --git a/agents/witness.md b/agents/witness.md index 21de037..924f8e8 100644 --- a/agents/witness.md +++ b/agents/witness.md @@ -18,4 +18,4 @@ You are the non-interactive entry point for the `witness` skill, whose content i Operate in the skill's non-interactive mode: no user is reachable, so never wait for an answer. Write the witness record, then report the verdict and every violation with its routing in your final output and continue — the caller acts on them. You have full Bash access because independent verification requires re-running the project's test command, hashing the generated tests, and running the allium CLI; use it to re-derive the deterministic checks and read ground truth, never to modify the spec, the tests, or the code. Write only the witness record — everything else you read, hash or re-run. -Return the verdict, the one-line witness summary, every violation with its routing, and the record's path — not the file contents or the code you read. +Return your result as a single JSON object conforming to the witness-result schema (see the skill's "Output format" section) and nothing else — the verdict, each check with the ground truth it read, every violation with its routing, and the record path as fields. Not the file contents or the code you read, and no prose around the object. diff --git a/scripts/test-skills.mjs b/scripts/test-skills.mjs index 34aed69..96a33de 100644 --- a/scripts/test-skills.mjs +++ b/scripts/test-skills.mjs @@ -10,7 +10,7 @@ * node scripts/test-skills.mjs structure # run one group * node scripts/test-skills.mjs portability links # run multiple groups * - * Groups: structure, codex, consistency, portability, links, routing, generation, loopdocs, hooks, modes, discovery, parking, witnessing, crosstalk + * Groups: structure, codex, consistency, portability, links, routing, generation, loopdocs, hooks, modes, handoffs, discovery, parking, witnessing, crosstalk * * All groups except discovery, parking, witnessing and crosstalk are offline (free, fast); * those four require --live and make Claude API calls. @@ -183,6 +183,80 @@ function isObject(value) { return value && typeof value === "object" && !Array.isArray(value); } +// Minimal dependency-free validator for the JSON-Schema subset our hand-off +// schemas use: const, enum, type (object/array/string/integer/number/boolean), +// required, properties, items, additionalProperties:false. Returns an array of +// error strings (empty means valid). Deterministic — this is the check that +// turns "does the phase's output conform" into a test rather than an eval. +function validateAgainstSchema(schema, value, pathStr = "") { + const errors = []; + const here = pathStr || "(root)"; + if ("const" in schema) { + if (value !== schema.const) + errors.push(`${here}: expected ${JSON.stringify(schema.const)}, got ${JSON.stringify(value)}`); + return errors; + } + if (schema.enum) { + if (!schema.enum.includes(value)) + errors.push(`${here}: ${JSON.stringify(value)} not in ${JSON.stringify(schema.enum)}`); + return errors; + } + switch (schema.type) { + case "object": + if (!isObject(value)) { errors.push(`${here}: expected object`); break; } + for (const req of schema.required || []) + if (!(req in value)) errors.push(`${here}: missing required '${req}'`); + if (schema.additionalProperties === false) + for (const k of Object.keys(value)) + if (!(schema.properties && k in schema.properties)) + errors.push(`${here}: unexpected property '${k}'`); + for (const [k, sub] of Object.entries(schema.properties || {})) + if (k in value) errors.push(...validateAgainstSchema(sub, value[k], `${here}.${k}`)); + break; + case "array": + if (!Array.isArray(value)) { errors.push(`${here}: expected array`); break; } + if (schema.items) + value.forEach((el, i) => errors.push(...validateAgainstSchema(schema.items, el, `${here}[${i}]`))); + break; + case "string": + if (typeof value !== "string") errors.push(`${here}: expected string`); + break; + case "integer": + if (!Number.isInteger(value)) errors.push(`${here}: expected integer`); + break; + case "number": + if (typeof value !== "number") errors.push(`${here}: expected number`); + break; + case "boolean": + if (typeof value !== "boolean") errors.push(`${here}: expected boolean`); + break; + } + return errors; +} + +// The property typed hand-offs buy: convergence is a pure function of the +// phases' typed fields, not a prose read. Mirrors driving-the-loop §3 for the +// two piloted phases (test counts come from the runner, not yet a schema'd +// phase). Deterministic by construction. +function isConverged({ weed, propagate, testsFailed, blockingQuestions }) { + return ( + testsFailed === 0 && + weed.verdict === "clean" && + propagate.uncovered_obligations.length === 0 && + blockingQuestions === 0 + ); +} + +// Pull the JSON record out of a relayed agent message: prefer the marked +// region, then take the outermost { ... }. Returns null if none parses. +function extractJsonRecord(text) { + const marked = text.match(/<<>>/); + const body = (marked ? marked[1] : text).replace(/```json\n?/g, "").replace(/```\n?/g, ""); + const m = body.match(/\{[\s\S]*\}/); + if (!m) return null; + try { return JSON.parse(m[0]); } catch { return null; } +} + // --------------------------------------------------------------------------- // Structure — frontmatter validity for all artifact types // --------------------------------------------------------------------------- @@ -651,6 +725,146 @@ if (shouldRun("modes")) { } } +// --------------------------------------------------------------------------- +// Handoffs — typed phase result records (pilot: weed, propagate). The schemas +// are the deterministic contract between a phase and the loop. These offline +// tests prove three things: valid records validate, malformed ones are caught, +// and convergence reduces to a pure function of the typed fields. No model runs +// here — conformance is a check, not an eval. +// --------------------------------------------------------------------------- + +if (shouldRun("handoffs")) { + console.log("\n── handoffs: typed phase result records ──\n"); + + const schemaDir = path.join(ROOT, "skills", "allium", "references", "schemas"); + const schemas = {}; + for (const name of ["distill-result", "weed-result", "tend-result", "propagate-result", "witness-result", "ledger"]) { + const fp = path.join(schemaDir, `${name}.schema.json`); + if (!existsSync(fp)) { fail(`schemas/${name}`, "file not found"); continue; } + schemas[name] = readJson(fp); + if (schemas[name]) pass(`schemas/${name}.schema.json is valid JSON`); + } + + console.log(""); + + // Valid fixtures — one clean, one dirty/uncovered — must validate. + const weedClean = { + phase: "weed", mode: "check", verdict: "clean", + divergences: [], open_questions: [], summary: "spec and code agree", + }; + const weedDirty = { + phase: "weed", mode: "check", verdict: "dirty", + divergences: [ + { subject: "Order.cancel", classification: "code-bug", spec: "cancel allowed from paid (spec:42)", code: "guarded to pending only (order.py:88)" }, + ], + open_questions: ["Should cancellation from shipped be allowed?"], + summary: "1 divergence: Order.cancel (code-bug)", + }; + const propagateCovered = { + phase: "propagate", + obligations: { total: 12, covered: 12, uncovered: 0 }, + uncovered_obligations: [], + generated_tests: [{ path: "order.test.js", hash: "sha256:abc123" }], + test_paths: ["order.test.js"], open_questions: [], + summary: "12 obligations, 12 covered, 0 uncovered", + }; + const propagateGap = { + phase: "propagate", + obligations: { total: 12, covered: 11, uncovered: 1 }, + uncovered_obligations: [ + { obligation: "temporal: InvitationExpires deadline", classification: "infrastructure-gap", reason: "no injectable clock" }, + ], + generated_tests: [{ path: "order.test.js", hash: "sha256:abc123" }], + test_paths: ["order.test.js"], open_questions: [], + summary: "12 obligations, 11 covered, 1 uncovered", + }; + + const distillRec = { + phase: "distill", spec_path: "giftcard.allium", + open_questions: ["Is forcing an over-redeemed balance to zero intended?"], + summary: "GiftCard redemption and status lifecycle", + }; + const tendRec = { + phase: "tend", spec_path: "shop.allium", + changes: ["Added expiry field to GiftCard", "Added GiftCardExpires rule"], + open_questions: ["Expiry period undecided"], + summary: "Added gift card expiry behaviour", + }; + const witnessPass = { + phase: "witness", verdict: "PASS", + checks: [{ name: "tests-pass", result: "pass", ground_truth: "runner exit 0, 12/12" }], + violations: [], record_path: ".allium-loop/giftcard.witness.json", + summary: "witness: PASS · checks 6/6", + }; + const witnessFail = { + phase: "witness", verdict: "FAIL", + checks: [{ name: "no-test-weakened", result: "fail", ground_truth: "sha256 mismatch on order.test.js" }], + violations: [{ violation: "order.test.js edited after propagate", routing: "revert + propagate" }], + record_path: ".allium-loop/giftcard.witness.json", + summary: "witness: FAIL · tampering on order.test.js", + }; + const ledgerRec = { + goal: "gift-cards", mode: "spec-first", tick: 3, + completed_sub_goals: ["redemption"], open_questions: [], + generated_test_hashes: { "order.test.js": "sha256:abc123" }, + reconciliation: "12 obligations, 12 covered, 0 uncovered", + }; + + const validFixtures = [ + ["weed-result", "clean", weedClean], ["weed-result", "dirty", weedDirty], + ["propagate-result", "covered", propagateCovered], ["propagate-result", "gap", propagateGap], + ["distill-result", "spec", distillRec], + ["tend-result", "changes", tendRec], + ["witness-result", "pass", witnessPass], ["witness-result", "fail", witnessFail], + ["ledger", "state", ledgerRec], + ]; + for (const [schemaName, label, rec] of validFixtures) { + if (!schemas[schemaName]) continue; + const errs = validateAgainstSchema(schemas[schemaName], rec); + errs.length === 0 ? pass(`${schemaName} valid fixture (${label})`) : fail(`${schemaName} valid fixture (${label})`, errs.join("; ")); + } + + console.log(""); + + // Malformed fixtures — each violates the schema in one way and MUST be caught. + const badCases = [ + ["weed-result", "bad enum verdict", { ...weedClean, verdict: "green" }], + ["weed-result", "missing required field", (() => { const r = { ...weedClean }; delete r.summary; return r; })()], + ["weed-result", "wrong type for divergences", { ...weedClean, divergences: "none" }], + ["weed-result", "unexpected property", { ...weedClean, extra: true }], + ["weed-result", "bad classification in item", { ...weedDirty, divergences: [{ ...weedDirty.divergences[0], classification: "typo" }] }], + ["propagate-result", "obligations not integer", { ...propagateCovered, obligations: { total: "12", covered: 12, uncovered: 0 } }], + ["propagate-result", "uncovered item missing reason", { ...propagateGap, uncovered_obligations: [{ obligation: "x", classification: "infrastructure-gap" }] }], + ["propagate-result", "generated_tests missing hash", { ...propagateCovered, generated_tests: [{ path: "x.js" }] }], + ["distill-result", "missing spec_path", (() => { const r = { ...distillRec }; delete r.spec_path; return r; })()], + ["tend-result", "changes wrong type", { ...tendRec, changes: "added expiry" }], + ["witness-result", "bad verdict enum", { ...witnessPass, verdict: "OK" }], + ["witness-result", "check bad result enum", { ...witnessPass, checks: [{ name: "x", result: "green", ground_truth: "y" }] }], + ["ledger", "bad mode enum", { ...ledgerRec, mode: "hybrid" }], + ["ledger", "tick not integer", { ...ledgerRec, tick: "3" }], + ]; + for (const [schemaName, label, rec] of badCases) { + if (!schemas[schemaName]) continue; + const errs = validateAgainstSchema(schemas[schemaName], rec); + errs.length > 0 ? pass(`${schemaName} rejects: ${label}`) : fail(`${schemaName} rejects: ${label}`, "malformed record validated"); + } + + console.log(""); + + // Convergence is a deterministic function of the typed fields. One converged + // case, and one per dimension that must block it. + const convCases = [ + ["all clean → converged", { weed: weedClean, propagate: propagateCovered, testsFailed: 0, blockingQuestions: 0 }, true], + ["tests failing → not converged", { weed: weedClean, propagate: propagateCovered, testsFailed: 2, blockingQuestions: 0 }, false], + ["weed dirty → not converged", { weed: weedDirty, propagate: propagateCovered, testsFailed: 0, blockingQuestions: 0 }, false], + ["uncovered obligation → not converged", { weed: weedClean, propagate: propagateGap, testsFailed: 0, blockingQuestions: 0 }, false], + ["blocking question → not converged", { weed: weedClean, propagate: propagateCovered, testsFailed: 0, blockingQuestions: 1 }, false], + ]; + for (const [label, state, expected] of convCases) { + isConverged(state) === expected ? pass(`convergence: ${label}`) : fail(`convergence: ${label}`, `expected ${expected}`); + } +} + // --------------------------------------------------------------------------- // Discovery — live Claude Code skill and agent loading // --------------------------------------------------------------------------- @@ -917,8 +1131,8 @@ if (shouldRun("witnessing")) { dir, "Use the Agent tool to spawn the 'allium:witness' subagent with exactly this task: " + '"Witness the convergence of the giftcard loop in this directory. The ledger is ' + - ".allium-loop/giftcard.json and records the generated test hashes. Re-derive the checks " + - 'and write the witness record." ' + + ".allium-loop/giftcard.json and records the generated test hashes. Re-derive the checks, " + + 'write the witness record, and return the witness-result JSON object as your result." ' + "Then output the subagent's final message verbatim between <<>> markers." ); @@ -932,6 +1146,17 @@ if (shouldRun("witnessing")) { } else { fail("witness: record", "no .allium-loop/giftcard.witness.json written"); } + // Same spawn, second assertion: the returned record conforms to the + // witness-result schema and its verdict is FAIL (the typed hand-off). + const witnessSchema = readJson(path.join(ROOT, "skills", "allium", "references", "schemas", "witness-result.schema.json")); + const wrec = extractJsonRecord(out); + if (witnessSchema && wrec) { + const errs = validateAgainstSchema(witnessSchema, wrec); + if (errs.length === 0 && wrec.verdict === "FAIL") pass("witness: record conforms to witness-result schema (verdict=FAIL)"); + else fail("witness: schema conformance", errs.slice(0, 3).join("; ") || `verdict=${wrec.verdict}`); + } else { + fail("witness: schema conformance", "no JSON record in relayed output"); + } } catch (e) { fail("witness tamper probe", e.message?.slice(0, 200)); } finally { @@ -940,6 +1165,143 @@ if (shouldRun("witnessing")) { } } +// --------------------------------------------------------------------------- +// Handoffs (live) — the eval: do the real weed and propagate agents emit +// JSON that conforms to their schemas? Stochastic input (an LLM writes the +// record), deterministic assertion (the record validates or it does not). +// This is the one part of the typed-handoff contract a fixture can't settle. +// --------------------------------------------------------------------------- + +if (shouldRun("handoffs")) { + console.log("\n── handoffs (live): agents emit schema-conforming records ──\n"); + + if (!LIVE) { + skip("handoff conformance probes", "pass --live to enable (uses API tokens)"); + } else { + const schemaDir = path.join(ROOT, "skills", "allium", "references", "schemas"); + const weedSchema = readJson(path.join(schemaDir, "weed-result.schema.json")); + const propagateSchema = readJson(path.join(schemaDir, "propagate-result.schema.json")); + const distillSchema = readJson(path.join(schemaDir, "distill-result.schema.json")); + const tendSchema = readJson(path.join(schemaDir, "tend-result.schema.json")); + + // weed: a spec/code pair that genuinely diverges, so a real record has + // structure to fill (verdict dirty, at least one classified divergence). + { + const dir = mkdtempSync(path.join(tmpdir(), "allium-handoff-weed-")); + try { + writeFileSync(path.join(dir, "shop.allium"), GIFTCARD_SPEC); + writeFileSync(path.join(dir, "giftcard.py"), GIFTCARD_PY); + const out = runAgentProbe( + dir, + "Use the Agent tool to spawn the 'allium:weed' subagent with exactly this task: " + + '"In check mode, compare shop.allium against giftcard.py and report the divergences." ' + + "Then output the subagent's final message verbatim between <<>> markers." + ); + const rec = extractJsonRecord(out); + if (!rec) { + fail("weed: emitted a JSON record", "no JSON object in relayed output"); + } else { + const errs = validateAgainstSchema(weedSchema, rec); + errs.length === 0 + ? pass(`weed: record conforms (verdict=${rec.verdict})`) + : fail("weed: schema conformance", errs.slice(0, 3).join("; ")); + } + } catch (e) { + fail("weed handoff probe", e.message?.slice(0, 200)); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + } + + // propagate: self-contained runnable project so reconciliation runs and the + // record's obligation counts and generated_tests are real. + { + const dir = mkdtempSync(path.join(tmpdir(), "allium-handoff-propagate-")); + try { + writeFileSync(path.join(dir, "shop.allium"), GIFTCARD_SPEC); + writeFileSync( + path.join(dir, "package.json"), + JSON.stringify({ name: "shop", version: "1.0.0", type: "module", scripts: { test: "node --test" } }, null, 2) + "\n" + ); + writeFileSync( + path.join(dir, "giftcard.js"), + "export class GiftCard {\n" + + " constructor(code, balance) { this.code = code; this.balance = balance; this.status = 'active'; }\n" + + " redeem(amount) { this.balance = Math.max(0, this.balance - amount); if (this.balance === 0) this.status = 'redeemed'; }\n" + + "}\n" + ); + const out = runAgentProbe( + dir, + "Use the Agent tool to spawn the 'allium:propagate' subagent with exactly this task: " + + '"Propagate tests from shop.allium against this project (giftcard.js, Node built-in test runner via npm test)." ' + + "Then output the subagent's final message verbatim between <<>> markers." + ); + const rec = extractJsonRecord(out); + if (!rec) { + fail("propagate: emitted a JSON record", "no JSON object in relayed output"); + } else { + const errs = validateAgainstSchema(propagateSchema, rec); + errs.length === 0 + ? pass(`propagate: record conforms (${rec.summary || "obligations reported"})`) + : fail("propagate: schema conformance", errs.slice(0, 3).join("; ")); + } + } catch (e) { + fail("propagate handoff probe", e.message?.slice(0, 200)); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + } + + // distill: a small source file to capture as a spec. + { + const dir = mkdtempSync(path.join(tmpdir(), "allium-handoff-distill-")); + try { + writeFileSync(path.join(dir, "giftcard.py"), GIFTCARD_PY); + const out = runAgentProbe( + dir, + "Use the Agent tool to spawn the 'allium:distill' subagent with exactly this task: " + + '"Distil an Allium spec for giftcard.py into giftcard.allium." ' + + "Then output the subagent's final message verbatim between <<>> markers." + ); + const rec = extractJsonRecord(out); + if (!rec) fail("distill: emitted a JSON record", "no JSON object in relayed output"); + else { + const errs = validateAgainstSchema(distillSchema, rec); + errs.length === 0 ? pass(`distill: record conforms (${rec.spec_path || "spec written"})`) : fail("distill: schema conformance", errs.slice(0, 3).join("; ")); + } + } catch (e) { + fail("distill handoff probe", e.message?.slice(0, 200)); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + } + + // tend: edit an existing spec, undecided points parked. + { + const dir = mkdtempSync(path.join(tmpdir(), "allium-handoff-tend-")); + try { + writeFileSync(path.join(dir, "shop.allium"), GIFTCARD_SPEC); + const out = runAgentProbe( + dir, + "Use the Agent tool to spawn the 'allium:tend' subagent with exactly this task: " + + '"Add gift card expiry behaviour to shop.allium. The expiry period is undecided." ' + + "Then output the subagent's final message verbatim between <<>> markers." + ); + const rec = extractJsonRecord(out); + if (!rec) fail("tend: emitted a JSON record", "no JSON object in relayed output"); + else { + const errs = validateAgainstSchema(tendSchema, rec); + errs.length === 0 ? pass(`tend: record conforms (${rec.spec_path || "spec edited"})`) : fail("tend: schema conformance", errs.slice(0, 3).join("; ")); + } + } catch (e) { + fail("tend handoff probe", e.message?.slice(0, 200)); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + } + } +} + // --------------------------------------------------------------------------- // Crosstalk — skills from the plugin don't bleed into unrelated projects, // and local agents/ don't leak outside the repo diff --git a/skills/allium/references/driving-the-loop.md b/skills/allium/references/driving-the-loop.md index ce345b1..2094327 100644 --- a/skills/allium/references/driving-the-loop.md +++ b/skills/allium/references/driving-the-loop.md @@ -73,7 +73,7 @@ After the slices converge, run a **whole-spec integration pass** — cross-entit ## 7. Delegate each phase to an isolated sub-agent (default) -By default, run each phase as an isolated sub-agent — `distill`, `weed`, `tend`, `propagate` and `witness` all ship as agents — and keep this orchestrator **thin**: it holds only the loop state (goal, ledger, current verdicts) and **reads no source files itself**. Hand each phase only what it needs to find its own inputs on disk — the spec's path and the ledger, never code you have read into your own context. Each phase reads the spec and the code it needs in *its own* fresh context and returns a short result (artefact path + summary + parked questions); that result is all you carry forward. The shared interface between phases is the on-disk artefacts (spec, tests, code) plus the ledger — never in-memory state. +By default, run each phase as an isolated sub-agent — `distill`, `weed`, `tend`, `propagate` and `witness` all ship as agents — and keep this orchestrator **thin**: it holds only the loop state (goal, ledger, current verdicts) and **reads no source files itself**. Hand each phase only what it needs to find its own inputs on disk — the spec's path and the ledger, never code you have read into your own context. Each phase reads the spec and the code it needs in *its own* fresh context and returns a **typed result record** — a JSON object conforming to that phase's schema (§12), not prose. That record is all you carry forward. The shared interface between phases is the on-disk artefacts (spec, tests, code) plus the ledger — never in-memory state. This is what keeps a long or large run within budget: the orchestrator's context stays flat (loop state only) while each phase's reading is bounded to that phase and then discarded. An inline run, by contrast, accumulates every phase's reads into one context that grows tick over tick until it is slow, expensive, or overflows the window. @@ -83,7 +83,7 @@ This is what keeps a long or large run within budget: the orchestrator's context Keep loop state in `.allium-loop/.json`: goal, mode, tick count, active inner loop, last verdicts, completed sub-goals, and parked (non-blocking) open questions. This makes the loop resumable — a fresh run reads it and continues where it left off. -The ledger also carries the evidence the witness (§11) re-derives convergence from, so record it as the phases produce it: `generated_test_hashes` (a content hash per generated test file, written by `propagate`) and the `reconciliation` line, the recorded `weed` verdict, and — for spec-first — the red-before-green observations per new test. The witness reads these; it does not take them on trust, but it needs them to exist. The witness writes its own artefact alongside the ledger, `.allium-loop/.witness.json` — the durable convergence record, keep it out of git the same way. +The ledger is itself typed — it conforms to [ledger.schema.json](./schemas/ledger.schema.json), so a resuming run reads structured state rather than re-parsing prose. It also carries the evidence the witness (§11) re-derives convergence from, so record it as the phases produce it: `generated_test_hashes` (a content hash per generated test file, written by `propagate`) and the `reconciliation` line, the recorded `weed` verdict, and — for spec-first — the red-before-green observations per new test. The witness reads these; it does not take them on trust, but it needs them to exist. The witness writes its own artefact alongside the ledger, `.allium-loop/.witness.json` — the durable convergence record, keep it out of git the same way. Git-ignore it: resolve the repo root (`git rev-parse --show-toplevel`; skip if not a git repo), then ensure `.allium-loop/` is ignored there — create `.gitignore` if absent, append if missing, no-op if already ignored (`git check-ignore` first). Best-effort: if it can't be written, continue and say so. Mention it once; don't prompt. @@ -106,3 +106,24 @@ The witness is independent and **deterministic** — it re-runs the cheap, deter - **INCONCLUSIVE** (no evidence to witness — e.g. verification could not run, §9) → do not declare convergence; degrade loudly. Run the witness at the convergence gate, not every tick — the phases already verify each tick; the witness confirms the *final* claim (and each slice's claim, §6). On a small inline scope where you ran the phases in your own context, you may witness inline too; the checks are the same, only the isolation differs. + +## 12. Typed hand-offs and routing + +Each phase returns a **typed result record**, not prose — a JSON object conforming to that phase's schema in [`schemas/`](./schemas/). Prose has to be *interpreted*; a typed record is *parsed*, so the loop routes and decides convergence as a deterministic function of fields rather than a read of a summary. The phases stay probabilistic inside; the control flow around them does not. + +The schemas: [`distill`](./schemas/distill-result.schema.json), [`weed`](./schemas/weed-result.schema.json), [`tend`](./schemas/tend-result.schema.json), [`propagate`](./schemas/propagate-result.schema.json), [`witness`](./schemas/witness-result.schema.json), and the [`ledger`](./schemas/ledger.schema.json). Each record carries a one-line `summary` for the human report, with the decision-bearing detail in structured fields. (Interactively, the skills still speak prose — the typed record is the machine hand-off, not the conversation.) + +**Routing** is a lookup on those fields, not a judgement: + +| Field | Route | +|---|---| +| `weed.verdict = clean` | no divergence; proceed | +| `weed.divergences[].classification = spec-bug` | `tend` the spec, then `propagate` | +| `weed.divergences[].classification = code-bug` | fix the code | +| `weed.divergences[].classification = aspirational` \| `intentional-gap` | leave both; note it | +| `propagate.uncovered_obligations` non-empty | not converged; back to `propagate` (or escalate if `infrastructure-gap`) | +| `tend.open_questions` / `distill.open_questions` non-empty | classify each (§5): blocking → escalate, else park | +| `witness.verdict = FAIL` | route each `violation` by its `routing` field (§11) | +| any record fails schema validation | reject the hand-off; the phase must re-emit — a malformed record is never treated as a result | + +**Convergence** is the boolean over the typed fields: `tests.failed = 0` ∧ `weed.verdict = clean` ∧ no blocking open questions ∧ `propagate.uncovered_obligations` empty ∧ (code-first) a fresh `distill` finds nothing new ∧ `witness.verdict = PASS`. When every conjunct reads from a field, "are we done" stops being a vibe and becomes an evaluation. diff --git a/skills/allium/references/schemas/distill-result.schema.json b/skills/allium/references/schemas/distill-result.schema.json new file mode 100644 index 0000000..662483f --- /dev/null +++ b/skills/allium/references/schemas/distill-result.schema.json @@ -0,0 +1,19 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "allium/handoffs/distill-result", + "title": "Distill phase result", + "description": "The typed record the distill phase returns to the loop orchestrator in place of a prose summary. The source code distill read stays out of the caller's context; only the path, a summary, and the parked judgement calls come back.", + "type": "object", + "required": ["spec_path", "summary"], + "additionalProperties": false, + "properties": { + "phase": { "const": "distill" }, + "spec_path": { "type": "string", "description": "the distilled .allium spec that was written" }, + "open_questions": { + "type": "array", + "description": "judgement calls parked as `open question` declarations in the spec (intended vs accidental behaviour, actor identity, scope)", + "items": { "type": "string" } + }, + "summary": { "type": "string", "description": "what the spec covers, one line" } + } +} diff --git a/skills/allium/references/schemas/ledger.schema.json b/skills/allium/references/schemas/ledger.schema.json new file mode 100644 index 0000000..96e2a78 --- /dev/null +++ b/skills/allium/references/schemas/ledger.schema.json @@ -0,0 +1,35 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "allium/handoffs/ledger", + "title": "Loop ledger", + "description": "The loop's durable state at .allium-loop/.json. Typed so a resuming run reads structured state rather than re-parsing prose, and so the witness has a schema'd baseline to re-derive from. Open to extra fields — loop state evolves — but the core is fixed.", + "type": "object", + "required": ["goal", "mode", "tick"], + "additionalProperties": true, + "properties": { + "goal": { "type": "string" }, + "mode": { "enum": ["spec-first", "code-first"] }, + "tick": { "type": "integer" }, + "active_inner_loop": { "type": "string" }, + "completed_sub_goals": { "type": "array", "items": { "type": "string" } }, + "open_questions": { + "type": "array", + "description": "parked, non-blocking questions", + "items": { "type": "string" } + }, + "generated_test_hashes": { + "type": "object", + "description": "path -> content hash, written by propagate; the witness tamper baseline" + }, + "reconciliation": { "type": "string", "description": "propagate's `N obligations, M covered, K uncovered` line" }, + "last_results": { + "type": "object", + "description": "the most recent typed record per phase (distill, weed, tend, propagate, witness)" + }, + "witness_records": { + "type": "array", + "description": "witness verdicts across ticks", + "items": { "type": "string" } + } + } +} diff --git a/skills/allium/references/schemas/propagate-result.schema.json b/skills/allium/references/schemas/propagate-result.schema.json new file mode 100644 index 0000000..aa31dbe --- /dev/null +++ b/skills/allium/references/schemas/propagate-result.schema.json @@ -0,0 +1,59 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "allium/handoffs/propagate-result", + "title": "Propagate phase result", + "description": "The typed record the propagate phase returns to the loop orchestrator in place of a prose summary. `obligations` and `uncovered_obligations` drive convergence; `generated_tests` is the tamper baseline the witness re-derives; `summary` carries the one human-readable line.", + "type": "object", + "required": ["obligations", "uncovered_obligations", "generated_tests", "summary"], + "additionalProperties": false, + "properties": { + "phase": { "const": "propagate" }, + "obligations": { + "type": "object", + "required": ["total", "covered", "uncovered"], + "additionalProperties": false, + "properties": { + "total": { "type": "integer" }, + "covered": { "type": "integer" }, + "uncovered": { "type": "integer" } + } + }, + "uncovered_obligations": { + "type": "array", + "description": "one entry per uncovered obligation; must be empty for the loop to treat the spec as converged. Each carries a reason so nothing is silently dropped.", + "items": { + "type": "object", + "required": ["obligation", "classification", "reason"], + "additionalProperties": false, + "properties": { + "obligation": { "type": "string" }, + "classification": { "enum": ["infrastructure-gap", "unmappable-construct"] }, + "reason": { "type": "string" } + } + } + }, + "generated_tests": { + "type": "array", + "description": "the tamper baseline the witness re-derives; one entry per generated test file", + "items": { + "type": "object", + "required": ["path", "hash"], + "additionalProperties": false, + "properties": { + "path": { "type": "string" }, + "hash": { "type": "string", "description": "content hash, e.g. sha256:..." } + } + } + }, + "test_paths": { + "type": "array", + "description": "test files generated or updated this run", + "items": { "type": "string" } + }, + "open_questions": { + "type": "array", + "items": { "type": "string" } + }, + "summary": { "type": "string", "description": "the `N obligations, M covered, K uncovered` line" } + } +} diff --git a/skills/allium/references/schemas/tend-result.schema.json b/skills/allium/references/schemas/tend-result.schema.json new file mode 100644 index 0000000..66f1d26 --- /dev/null +++ b/skills/allium/references/schemas/tend-result.schema.json @@ -0,0 +1,24 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "allium/handoffs/tend-result", + "title": "Tend phase result", + "description": "The typed record the tend phase returns to the loop orchestrator in place of a prose summary. Lists what changed in the spec and any judgement calls parked for a human.", + "type": "object", + "required": ["spec_path", "summary"], + "additionalProperties": false, + "properties": { + "phase": { "const": "tend" }, + "spec_path": { "type": "string", "description": "the .allium spec that was edited" }, + "changes": { + "type": "array", + "description": "the edits made, one entry each (entity/rule/surface added or changed)", + "items": { "type": "string" } + }, + "open_questions": { + "type": "array", + "description": "decisions parked as `open question` declarations in the spec", + "items": { "type": "string" } + }, + "summary": { "type": "string", "description": "one line" } + } +} diff --git a/skills/allium/references/schemas/weed-result.schema.json b/skills/allium/references/schemas/weed-result.schema.json new file mode 100644 index 0000000..4912568 --- /dev/null +++ b/skills/allium/references/schemas/weed-result.schema.json @@ -0,0 +1,46 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "allium/handoffs/weed-result", + "title": "Weed phase result", + "description": "The typed record the weed phase returns to the loop orchestrator in place of a prose summary. The structured fields drive deterministic routing and convergence; `summary` carries the one human-readable line so nothing is lost.", + "type": "object", + "required": ["verdict", "divergences", "summary"], + "additionalProperties": false, + "properties": { + "phase": { "const": "weed" }, + "mode": { + "enum": ["check", "update-spec", "update-code"], + "description": "the mode weed ran in" + }, + "verdict": { + "enum": ["clean", "dirty"], + "description": "clean = spec and code agree; dirty = at least one divergence remains" + }, + "divergences": { + "type": "array", + "description": "every unresolved divergence; empty when verdict is clean", + "items": { + "type": "object", + "required": ["subject", "classification", "spec", "code"], + "additionalProperties": false, + "properties": { + "subject": { "type": "string", "description": "the entity, rule or trigger the divergence is about" }, + "classification": { "enum": ["spec-bug", "code-bug", "aspirational", "intentional-gap"] }, + "spec": { "type": "string", "description": "what the spec says, with file:line where known" }, + "code": { "type": "string", "description": "what the code does, with file:line where known" } + } + } + }, + "open_questions": { + "type": "array", + "description": "questions parked for a human; each also recorded as an `open question` in the spec when weed writes one", + "items": { "type": "string" } + }, + "artefacts": { + "type": "array", + "description": "spec files edited, in update-spec / update-code modes", + "items": { "type": "string" } + }, + "summary": { "type": "string", "description": "the single human-readable line" } + } +} diff --git a/skills/allium/references/schemas/witness-result.schema.json b/skills/allium/references/schemas/witness-result.schema.json new file mode 100644 index 0000000..b815f3d --- /dev/null +++ b/skills/allium/references/schemas/witness-result.schema.json @@ -0,0 +1,44 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "allium/handoffs/witness-result", + "title": "Witness phase result", + "description": "The typed record the witness phase returns to the loop orchestrator. The verdict gates convergence; each check names the ground truth it read; each violation names the phase that fixes it. Mirrors the durable record written to .allium-loop/.witness.json.", + "type": "object", + "required": ["verdict", "checks", "violations", "record_path", "summary"], + "additionalProperties": false, + "properties": { + "phase": { "const": "witness" }, + "verdict": { + "enum": ["PASS", "FAIL", "INCONCLUSIVE"], + "description": "PASS only when every check with evidence passed; INCONCLUSIVE when there was no evidence to witness" + }, + "checks": { + "type": "array", + "items": { + "type": "object", + "required": ["name", "result", "ground_truth"], + "additionalProperties": false, + "properties": { + "name": { "type": "string", "description": "e.g. tests-pass, no-test-weakened, coverage, weed-verdict, open-questions, convergence, red-before-green" }, + "result": { "enum": ["pass", "fail", "inconclusive"] }, + "ground_truth": { "type": "string", "description": "the evidence read: runner exit status, hash comparison, reconciliation line, weed verdict, open-questions diff" } + } + } + }, + "violations": { + "type": "array", + "description": "empty when verdict is PASS; each names its routing", + "items": { + "type": "object", + "required": ["violation", "routing"], + "additionalProperties": false, + "properties": { + "violation": { "type": "string" }, + "routing": { "type": "string", "description": "the phase that resolves it (e.g. revert + propagate, implement, escalate)" } + } + } + }, + "record_path": { "type": "string", "description": ".allium-loop/.witness.json" }, + "summary": { "type": "string", "description": "the one-line witness summary" } + } +} diff --git a/skills/distill/SKILL.md b/skills/distill/SKILL.md index 3bcf8c1..48f5c4c 100644 --- a/skills/distill/SKILL.md +++ b/skills/distill/SKILL.md @@ -839,6 +839,19 @@ If any remain, ask: "Would a stakeholder include this in a requirements doc?" The extracted spec is a starting point. If distillation reveals gaps that need structured discovery (unclear requirements, complex entity relationships, unstated business rules), use the `elicit` skill to fill them. For targeted changes as requirements evolve, use the `tend` skill. For checking ongoing alignment between the spec and implementation, use the `weed` skill. +## Typed result (loop hand-off) + +When running as the `distill` subagent inside the Allium loop, return your result as a single JSON object conforming to [distill-result.schema.json](../allium/references/schemas/distill-result.schema.json), and nothing else: the `spec_path`, the parked `open_questions`, and a one-line `summary` of what the spec covers. Emit every field, using `[]` for empty lists. The source you read stays out of the caller's context; only these fields come back. Running interactively, present your findings in prose as usual — the typed record is for the machine hand-off, not the conversation. + +```json +{ + "phase": "distill", + "spec_path": "giftcard.allium", + "open_questions": ["Is forcing an over-redeemed balance to zero intended or accidental?"], + "summary": "GiftCard redemption and status lifecycle" +} +``` + ## References - [Language reference](../allium/references/language-reference.md), full Allium syntax diff --git a/skills/propagate/SKILL.md b/skills/propagate/SKILL.md index 840569b..25a1eb8 100644 --- a/skills/propagate/SKILL.md +++ b/skills/propagate/SKILL.md @@ -242,6 +242,22 @@ When running inside the Allium loop, record the reconciliation baseline so the l The hash is the ground truth the `witness` skill re-derives: a generated test whose hash changes with no intervening propagate run is a hand-edited test — the anti-cheat violation the loop must never reach convergence with. Recording the baseline is what makes that check possible; skip it and the witness can confirm the tests pass but not that they were not weakened. This is cheap bookkeeping, not a report — do not narrate it. +### Typed result (loop hand-off) + +When running as the `propagate` subagent inside the Allium loop, return your result as a single JSON object conforming to [propagate-result.schema.json](../allium/references/schemas/propagate-result.schema.json), and nothing else. The loop routes on the structured fields: `obligations` and `uncovered_obligations` decide whether coverage is complete (the loop must not converge while `uncovered_obligations` is non-empty), and `generated_tests` carries the path-and-hash baseline the witness re-derives — the same hashes recorded above, now first-class in the hand-off. Keep the `summary` field to the `N obligations, M covered, K uncovered` line. Emit every field, using `[]` for empty lists — do not omit them. Running interactively, the summary line remains the whole user-facing output as before — the typed record is for the machine hand-off, not the conversation. + +```json +{ + "phase": "propagate", + "obligations": { "total": 12, "covered": 12, "uncovered": 0 }, + "uncovered_obligations": [], + "generated_tests": [{ "path": "order.test.js", "hash": "sha256:9f2c…" }], + "test_paths": ["order.test.js"], + "open_questions": [], + "summary": "12 obligations, 12 covered, 0 uncovered" +} +``` + ## Interaction with other tools - **distill** produces specs from code. Those specs feed propagate. diff --git a/skills/tend/SKILL.md b/skills/tend/SKILL.md index 29908b9..8fe86b3 100644 --- a/skills/tend/SKILL.md +++ b/skills/tend/SKILL.md @@ -104,3 +104,17 @@ After edits that change rules, surfaces or transition graphs, run `allium analys ## Output When proposing spec changes, explain the behavioural intent first, then show the changes. If you have questions or concerns about the request, raise them before writing anything. + +### Typed result (loop hand-off) + +When running as the `tend` subagent inside the Allium loop, return your result as a single JSON object conforming to [tend-result.schema.json](../allium/references/schemas/tend-result.schema.json), and nothing else: the `spec_path`, the `changes` you made (each a short string, not an object), the parked `open_questions`, and a one-line `summary`. Emit every field, using `[]` for empty lists. Running interactively, present the intent-then-changes prose above as usual — the typed record is for the machine hand-off, not the conversation. + +```json +{ + "phase": "tend", + "spec_path": "shop.allium", + "changes": ["Added expiry field to GiftCard", "Added GiftCardExpires temporal rule"], + "open_questions": ["Expiry period undecided"], + "summary": "Added gift card expiry behaviour" +} +``` diff --git a/skills/weed/SKILL.md b/skills/weed/SKILL.md index fae1bc6..45a04ce 100644 --- a/skills/weed/SKILL.md +++ b/skills/weed/SKILL.md @@ -111,3 +111,21 @@ Classification: [proposed classification with reasoning] ``` Group related divergences together. Lead with the most consequential findings. + +### Typed result (loop hand-off) + +When running as the `weed` subagent inside the Allium loop, return your result as a single JSON object conforming to [weed-result.schema.json](../allium/references/schemas/weed-result.schema.json), and nothing else. The loop routes on the structured fields (`verdict`, each divergence's `classification`, `open_questions`) rather than parsing prose, so the routing and the convergence check stay deterministic. Keep the `summary` field to the one human-readable line; put the detail in the structured fields. Emit every field, using `[]` for empty lists. Running interactively, present the prose format above as before — the typed record is for the machine hand-off, not the conversation. + +```json +{ + "phase": "weed", + "mode": "check", + "verdict": "dirty", + "divergences": [ + { "subject": "Order.cancel", "classification": "code-bug", "spec": "cancel allowed from paid (shop.allium:42)", "code": "guarded to pending only (order.py:88)" } + ], + "open_questions": [], + "artefacts": [], + "summary": "1 divergence: Order.cancel (code-bug)" +} +``` diff --git a/skills/witness/SKILL.md b/skills/witness/SKILL.md index ae1ad50..d43a047 100644 --- a/skills/witness/SKILL.md +++ b/skills/witness/SKILL.md @@ -71,13 +71,33 @@ Do not embed file contents or code — the record holds verdicts and the evidenc ## Output format -Close with a single summary line the loop can fold into its report: +When running as the `witness` subagent inside the Allium loop, return your result as a single JSON object conforming to [witness-result.schema.json](../allium/references/schemas/witness-result.schema.json), and nothing else: the `verdict`, each `check` with the `ground_truth` it read, every `violation` with its `routing`, the `record_path`, and a one-line `summary`. Emit every field, using `[]` for an empty `violations` list on a PASS. The loop gates convergence on `verdict` directly — no prose to parse. The object mirrors the durable record you wrote to `.allium-loop/.witness.json`. + +```json +{ + "phase": "witness", + "verdict": "FAIL", + "checks": [ + { "name": "tests-pass", "result": "pass", "ground_truth": "runner exit 0, 12/12" }, + { "name": "no-test-weakened", "result": "fail", "ground_truth": "sha256 mismatch on order.test.js" } + ], + "violations": [ + { "violation": "order.test.js edited after propagate", "routing": "revert + propagate" } + ], + "record_path": ".allium-loop/gift-cards.witness.json", + "summary": "witness: FAIL · tampering on order.test.js" +} +``` + +As the loop subagent, return **only** that JSON object — no prose before or after it, even though you also wrote the durable record to disk. The returned object is your result; the file is its durable copy. + +Running interactively (not as the loop subagent), skip the JSON and close with a single human-readable summary line instead: ``` witness: PASS · checks 6/6 · tests 12/12 (runner) · tampering none · openQ 0 blocking · record .allium-loop/.witness.json ``` -On failure, lead with the verdict and the violations, each with its routing, then the record path. Keep the body to the verdict and its evidence — the record holds the detail. +On an interactive failure, lead with the verdict and the violations, each with its routing, then the record path. Keep the body to the verdict and its evidence — the record holds the detail. ## Interaction with other tools