test: the plugin suites on Haiku 5.5

evals/dynamic/README.md

@@ -98,7 +98,8 @@ the spec-analyzer agent open the plugin's spec model. `session-docs` has Claude
 session context. The control, `without-rule`, runs without the rule, where the same read is denied.
 Agents that ask for `opus` run on Haiku (`ANTHROPIC_DEFAULT_OPUS_MODEL`). Three sessions cost about
 $0.60; run it after any change to how the plugin's skills, agents, or hooks name the reference docs.
-Report: [`reports/2026-10-05-plugin-docs.md`](reports/2026-10-05-plugin-docs.md).
+Report: [`reports/2026-10-05-plugin-docs.md`](reports/2026-10-05-plugin-docs.md). Both suites on
+Haiku 5.5: [`reports/2026-10-09-haiku-5-5.md`](reports/2026-10-09-haiku-5-5.md).
 
 The **adopt** suite builds its own project (`fixtures/adopt/project.sh`: a new repository with no
 commits) and appends each run's end state (`inspect.sh`) to the transcript. `{{FRAMEWORK}}` in a

evals/dynamic/reports/2026-10-09-haiku-5-5.md

@@ -0,0 +1,57 @@
+# Session evals — the plugin suites on Haiku 5.5 — 2026-10-09
+
+The `plugin-hooks` and `plugin-docs` suites first ran on Haiku 4.5 (the 2026-10-04 and 2026-10-05
+reports). This run repeats them on Haiku 5.5, passed as `--models claude-haiku-5-5`, with real
+headless Claude Code sessions (v2.1.286) and the `adf` plugin loaded from a checkout of
+`fix/prompt-audit`. Agents that ask for `opus` now run on `claude-haiku-5-5` too: the runner's
+stand-in was still Haiku 4.5. Every transcript names `claude-haiku-5-5`, the agents' included.
+
+| Run | Sessions | Checks |
+|---|---|---|
+| `plugin-hooks` | 7 | 8 passed, 0 failed |
+| `plugin-docs` | 3 | 7 passed, 0 failed |
+| `agent-spec-model` without the reference-docs note | 1 | 2 passed, 0 failed |
+
+## Results
+
+Haiku 5.5 matched Haiku 4.5 on every check. In `plugin-docs`, `agent-spec-model`'s agent read the
+plugin's `SPEC-MODEL.md` and the project's files without asking. That session then stopped at its
+$0.75 cap before the final reply (see the CLI note below); its checks had already passed.
+
+## Without the note
+
+Decision 0019 added one line to the plugin's skills and agents — the reference docs they name are the
+plugin's copies, outside the project — after `spec-analyzer`, on Haiku 4.5, read `docs/SPEC-MODEL.md`
+in the project although its instructions gave the plugin's full path. To see whether Haiku 5.5 still
+needs it, `agent-spec-model` ran once from a copy of the checkout with the line removed from
+`plugins/adf/agents/spec-analyzer.md`, and a $2 cap.
+
+The agent read the plugin's `SPEC-MODEL.md` at the full path and never looked for one in the project,
+and the session ended normally.
+
+| Note | Model | The agent read the plugin's spec model |
+|---|---|---|
+| None | Haiku 4.5 (0019) | No — `docs/SPEC-MODEL.md` in the project |
+| Present | Haiku 4.5 (0019) | Yes |
+| Present | Haiku 5.5 | Yes |
+| None | Haiku 5.5 | Yes |
+
+The note stays. One run doesn't show the failure can't happen, the line states a fact about where the
+docs are, and in projects the agent runs on Opus. What changes is its evidence: the failure that
+motivated it was Haiku 4.5's, and Haiku 5.5 didn't repeat it.
+
+## The CLI doesn't know Haiku 5.5 yet
+
+Claude Code v2.1.286 logs `unrecognized_model` for `claude-haiku-5-5` and the API serves it, but the
+CLI prices it at a rate other than Haiku 5.5's: these runs report about $4.26 API-equivalent in all,
+more than the Haiku 4.5 runs, for a model priced lower. The session budget cap is checked against the
+same estimate, which is why `agent-spec-model` stopped at $0.75. Update Claude Code before reading the
+costs or tightening the caps. The suites' default `haiku` alias resolves per CLI version; pass
+`--models claude-haiku-5-5` to be sure which Haiku runs.
+
+## What Haiku doesn't stand in for
+
+These suites test plumbing — that hooks fire and their messages arrive, and that a read rule and a
+path reach the plugin's files — where the cheapest model is enough. They don't test judgment, and
+they don't test how Opus 5.5 or Sonnet 5.5 return text written just before a tool call, which can come
+back as a progress-update thinking block. The `triage` and `debug` suites stay on `sonnet` and `opus`.

evals/dynamic/run-session-evals.sh

@@ -257,7 +257,7 @@ PY
     [ -n "$DOCS_RULE" ] || dirs+=("$FWC")
   fi
   if [ -n "$DOCS_RULE" ]; then # reads in the project need no rule; the plugin's folder, the committed one
-    reads=() run_env=(ANTHROPIC_DEFAULT_OPUS_MODEL=claude-haiku-4-5-20251001)
+    reads=() run_env=(ANTHROPIC_DEFAULT_OPUS_MODEL=claude-haiku-5-5)
     grep -q '<!-- run: no-read-rule -->' "$input" || reads=("Read(/$FWC/plugins/adf/**)")
   fi
   [ -d "$SOURCE" ] && [ "$SOURCE" != "$FWC" -o ${#dirs[@]} -eq 0 ] && dirs+=("$SOURCE")