You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
US-60 — A repeatable score for ai/query.py, run in CI #393
As a maintainer, I want the question-answering layer scored the same way every time, so a change that makes it answer wrongly fails CI instead of reaching a user.
The score runs in CI, which has neither Batfish nor Ollama
"Answered and wrong" is counted separately from "refused", as docs/evaluation.md Part 2 does
Shape
Not a pipeline feature: AI layer and tooling.
Why this is in the milestone
The question layer has been measured once, by hand: 20 questions on 10 August (docs/evaluation.md, Part 2). It has never been pinned in CI.
Depends on #385: its method is reused, so it merges first.
Epic: Specialist pick (explainable and trustworthy AI for network security) | Size: estimated at sprint planning | Assigned:@patelankeet2
Specialist pick agreed by all four on 13 October 2026 (slate row 3). Acceptance criteria are drafted from the slate's wording; the owner refines them at sprint planning on 19 October.
Activity