[LinuxDevil]

What

• LOU-D7: defineEval() gains a trajectory form: { name, agent | () => agent, tags, cases, judge?, test(t, c) }. t offers send, reply, completed, calledTool(name, { args, times }) (partial deep arg match), notCalledTool, toolOrder (subsequence), maxSteps, maxTokens, maxCostUsd (skipped cleanly: main reports no cost), check (gate), soft (reported only), judge. New checks: includes, matches, equals, atLeast, atMost. Every case yields a structured EvalResult. The classic score/threshold form is unchanged and also reports a result.
• LOU-D8: loushy eval [globs...] [--tag] [--junit] [--json] [--strict] [--judge] [--config]. Wired through src/cli/eval.ts, bin/loushy.js, tsup entry, fallow entry glob.
• docs/evals.md (authoring, assertions table, datasets, mockModel default, judge evals, CLI, CI + JUnit upload), linked from README; README evals section and api-overview updated; llms files regenerated.

Why

Audit: "one score per eval, no datasets". Parity with eve's defineEval({ test(t) }) plus CI ergonomics (JUnit/JSON, tags, strict).

How verified

• npx tsc --noEmit: clean
• npm run lint: 0 errors (433 warnings, none new in touched files)
• npm run build + create-loushy-agent build: OK
• npm run test:coverage: Test Files 120 passed (120), Tests 1710 passed | 4 skipped
• npm run fallow: exit 0
• npm run test:types: no errors
• npm run docs:verify-snippets: all 60 snippets type-check, 6 run cleanly
• npm run docs:llms:check: OK
• Merged origin/main (995f649) into the branch; diff-filter=D vs origin/main is empty.

Notes for reviewer

• Results transport: each case appends a JSON line to $LOUSHY_EVAL_RESULTS, instead of a custom vitest reporter. It works across vitest versions and pools, needs no module resolved from the user's project, and short appends are safe across workers. Tag filter travels the other way via LOUSHY_EVAL_TAGS.
• vitest config: the user's project has no loushy config, so loushy eval generates a plain-object config in a temp dir (globs, *.judge.eval.* excluded unless --judge, which sets LOUSHY_ALLOW_LLM_JUDGE). --config uses the user's own.
• t.judge() requires judge: { provider, model } on the eval (throws a clear error otherwise); it sets a new allowOutsideJudgeRunner option on llmJudge() because the provider was configured explicitly.
• Exit codes: 1 gate failure / soft with --strict / vitest failing without a recorded failure; 2 cannot run (bad args, vitest missing, install hint printed).
• Fixture evals live in src/cli/__fixtures__/eval/, excluded from the default run in vitest.config.ts.
• Two flaky tests under load seen once each, passing on rerun: NodeWorkspace.test.ts (env leak) and http.test.ts (timeout).

🤖 Generated with Claude Code