Phases hand back typed results, not prose - #69
Merged
Conversation
Each loop phase returned prose the orchestrator had to re-read and interpret. A prose read is a model step, so the routing and the convergence check inherited the model's non-determinism. This turns the contract between a phase and the loop into typed data. Add a JSON Schema per phase result (distill, weed, tend, propagate, witness) and one for the ledger, under skills/allium/references/schemas/. driving-the-loop gains section 12: a routing table that is a lookup on the typed fields, and convergence stated as a boolean over them. The phases stay probabilistic inside; the control flow around them becomes deterministic. Section 7 now carries a typed result record between phases, and section 8's ledger is itself typed. Required fields are only the decision-bearing ones (verdict, divergences, obligations, uncovered_obligations, generated_tests). Redundant and informational fields are optional, so a phase can never fail the contract on a field the loop does not route on.
Each skill gains a "Typed result" section: when running as the loop subagent, return a single JSON object conforming to the phase's schema and nothing else, with a worked example to mirror. Interactively the skills still speak prose; the typed record is the machine hand-off, not the conversation. The agent shells return the record, and the VS Code agents are regenerated to match. The witness returns its verdict as the JSON hand-off as the loop subagent, and keeps the human summary line only for interactive use. propagate's generated test hashes are now a first-class field of its record, the same baseline the witness re-derives.
…ormance A dependency-free validator for the JSON-Schema subset the records use, plus a handoffs group. Offline and deterministic: valid fixtures validate, malformed records are caught (bad enum, missing field, wrong type, unexpected property), and convergence is proven as a pure function of the typed fields. This is the part of the contract a test settles rather than an eval. Live (--live): spawn each phase agent and assert its real output conforms to the schema. Stochastic input, deterministic assertion. The witness conformance rides the existing tamper probe's spawn. Running this eval is what surfaced that propagate, tend and witness omitted required fields or returned prose; the fix was relaxing the schemas to the decision-bearing fields and giving each skill a worked example. All five agents now return conforming records.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The loop runs each phase in its own subagent. Each phase hands back a result. Until now that result was prose, so the loop had to read it and work out what it meant. This makes the result typed. The loop reads a field instead of reading a paragraph. The routing and the done check get simpler and easier to trust, with less room to misread a result. It also lays the groundwork for moving these checks into the CLI later, where code can run them instead of the model.
What it does
Every phase now returns a small JSON record instead of prose. weed returns a verdict and its divergences. propagate returns the coverage counts. The loop reads those fields to decide what to do next. A clean verdict moves on, a code bug goes back to the code, and whether the loop is done becomes a plain check over the fields.
Implementation
Tests
Potential next steps
Right now the loop reads the records itself, and the validator that checks them lives in the tests, not the live loop. A possible next step is to run that check inside the loop too, first as a small script the loop calls on each hand-off, and later as part of the CLI, alongside the routing and the done check. That would be the point where these checks stop being the model's judgement and become code.