Add a witness phase to the loop (3.12) - #68
Merged
yavorpanayotov merged 3 commits intoAug 19, 2026
Conversation
The loop's phases run inside isolated sub-agents (3.10) and now fan out (3.11), which moved the convergence signal out of the orchestrator's sight: it converges on prose self-reports from actors it cannot see into -- tests passed, no test weakened, no blocking question quietly parked. This adds the phase that confirms those claims instead of trusting them. allium:witness is an independent, deterministic verifier spawned at the convergence gate. It re-derives the claim from ground truth the run cannot fabricate -- the test runner's own exit status (not the reported count), the sha256 of each generated test vs. the hash propagate recorded in the ledger (a changed hash with no intervening propagate is a weakened test), the reconciliation line, the weed verdict, and the spec's open-questions section -- and gates convergence on its verdict. It writes a durable witness record to .allium-loop/<slug>.witness.json. It is cheap by construction: it re-runs only the deterministic tools (test command, allium check/analyse, hashing, grep) and reads the machine output the phases already emitted. It never re-runs the model-heavy phases (propagate, distill, weed reasoning), so it costs one light pass per converged run, not a second loop. The anti-cheat rule "never weaken a generated test" stops being prose the actor is trusted to honour and becomes a check the loop enforces. Built as a thin dual-entry capability like distill/propagate (skill is the single source; agent preloads it; VS Code agent materialised at build time): - skills/witness/SKILL.md, agents/witness.md: the new capability. - propagate records generated_test_hashes + reconciliation in the ledger so the witness has a baseline to check. - plugin.json registers the skill and agent; generate-multi-editor emits the VS Code agent (and regenerates the stale distill/propagate agents). - test-skills.mjs: witness joins every structural/modes/portability group, plus a live 'witnessing' probe that weakens a hashed test and asserts the witness returns FAIL naming the tamper -- deterministic, no model judging a model.
…eport Wire allium:witness into the loop as the convergence gate. driving-the-loop.md gains a fifth convergence condition (an independent witness PASS), a new §11 describing the deterministic checks and the PASS/FAIL/INCONCLUSIVE routing, and the anti-cheat rule (§4) is now enforced by the witness rather than trusted. Large goals (§6) witness each slice before it counts as converged and the whole spec after integration; the ledger (§8) records the evidence the witness reads. recommended-loops.md adds the witness to the exit condition, the guardrails and the operator table; the top skill's routing table and loop summary follow.
The Allium loop now witnesses its own convergence: an independent, deterministic allium:witness pass re-derives the claim from ground truth (the test runner's exit status, the generated tests' hashes, the reconciliation and weed verdicts) and gates convergence on its verdict. Bounded loops (3.10/3.11) that also prove they got to green honestly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The loop keeps getting stronger. 3.10 moved each phase into its own subagent. 3.11 fanned distill out across services. Each phase runs isolated and hands back a short result. This is the next step. The loop should be able to stand behind that result and prove it. Witness gives it that. It confirms the run's own answer against ground truth, and it leaves a record the loop can show for it.
What it does
This adds a new phase called witness. It runs at the convergence gate in its own subagent. It rebuilds the result from ground truth the run cannot fake. It reads the test runner's own exit status. It hashes each generated test and compares it to the hash propagate recorded. It reads the reconciliation line and the weed verdict. It checks the open questions. If any of that was faked, the witness fails and the loop keeps working. The loop converges only when the witness passes.
It stays cheap. It re-runs the fast deterministic tools and reads output the phases already produced. It never re-runs propagate, distill, or weed. So it costs one light pass per run.
Implementation
.allium-loop/<slug>.witness.json.