Skip to content

[EPIC-V][LOU-V4] Structured output: createAgent({ output: zodSchema }) yields a typed, validated result.object - #90

Merged
LinuxDevil merged 2 commits into
mainfrom
lou-v4-structured-output
Oct 1, 2026
Merged

LinuxDevil merged 2 commits into
mainfrom
lou-v4-structured-output

Conversation

@LinuxDevil

Copy link
Copy Markdown
Owner

What

  • createAgent({ output: zodSchema }) and ExecuteOptions.output: the agent's final reply must be a JSON object that matches the schema. send(), run.result and agent.approvals.resolve() resolve with it as result.object. For createAgent() agents it is typed z.output<typeof schema>: SimpleAgent<TObject>, AgentRun<TObject> and ExecutionResult<TObject> gain a type parameter that defaults to unknown. result.text keeps the raw JSON.
  • How it works (the same for every provider):
    • The system prompt gets an ## Output format block with the schema rendered as JSON Schema by the ai SDK's zodSchema(). ai is a required peer dependency that the executor already imports at runtime, so no dependency is added.
    • Every model call carries the new GenerateOptions.responseFormat: { type: 'json', schema? } hint. aiSdkProvider maps it to experimental_output, a small Output that turns on JSON mode. It sends the schema only to models that support structured outputs, as Output.object() does, and leaves the prompt and reply text unchanged. Providers without a JSON mode ignore the hint.
  • After the loop produces its final text, the text is parsed as JSON (a code fence around it is accepted) and validated. If it is invalid, the run takes exactly one repair step: a user message [output-invalid] … N issues (path: message) goes to the model. The repair step counts as a step and respects maxSteps. If the reply is still invalid, the run resolves with the new finishReason: 'output-invalid', no object, and outputError: { message, issues }. Tool calls still work before the final answer.
  • run.done gains an optional object field. docs/streaming.md allows new optional fields without changing v, so the schema version stays 1.
  • toolArgsValidation.ts now exports parseWithIssues() and formatIssues(), so tool arguments and output use the same zod issue formatting.
  • Docs:
    • New docs/structured-output.md, linked from README's feature list and its docs table.
    • docs/api-overview.md: new "Structured output" section, and 'output-invalid' added to the finish reasons.
    • docs/streaming.md: the run.done row mentions object.
    • CHANGELOG entry under Added.
    • llms files regenerated.

Why

Parity gap with eve's output schemas (Epic V). Callers who want machine-readable results currently have to parse and validate free text themselves.

Acceptance criteria

  • createAgent({ output }) and ExecuteOptions.output make the final reply a JSON object validated by the schema. send() resolves with result.object typed as z.output<typeof schema> (type test), and result.text keeps the raw JSON.
  • agent.stream(): run.result and run.done carry object. The field is additive and allowed by the docs/streaming.md versioning rules.
  • The executor adds an output instruction to the system prompt and passes a responseFormat hint, which aiSdkProvider maps to the ai v4 JSON mode (experimental_output).
  • The reply is parsed (fences accepted) and validated. One repair step follows (counted, respects maxSteps). If that fails too, the run ends with 'output-invalid', object: undefined and outputError. Tool calls still work before the final answer.
  • Tests with mockModel:
    • valid JSON on the first try
    • invalid, then repaired
    • invalid twice, giving 'output-invalid'
    • fenced JSON
    • no repair step left in the maxSteps budget
    • tool calls before the answer
    • the stream() path, valid and invalid
    • zod defaults and transforms applied
    • agents without output unchanged
    • type test for result.object
    • provider mapping test
  • Docs (api-overview, new structured-output.md linked from README), llms files regenerated, CHANGELOG.

How verified

  • npx tsc --noEmit: no errors
  • npm run lint: 0 errors, 422 warnings. These are all on lines that existed before; none of the touched files has a new warning.
  • npm run test:types: 6 files, 26 tests passed, no type errors
  • npx vitest run: 147 files passed; 2109 tests passed, 3 skipped. Run after npm run build, which the dist-dependent tests need.
  • npm run build && npm run build --workspace=packages/create-loushy-agent: success
  • npm run docs:verify-snippets -- --skip-build: all 105 snippets type-check against src/; 6 also run cleanly
  • npm run docs:llms:check: exit 0
  • npm run test:coverage && npm run fallow: both exit 0. Fallow reports no dead code. Its one duplication note (OpenAIProvider/AnthropicProvider, 15 lines) was already on main and is in files this PR does not touch.

Notes:

  • Sub-agent runs do not inherit output: childOptions() in delegation.ts picks fields explicitly, and that file is out of scope.
  • session() wiring was not edited. Session turns get the same runtime validation because they share executeOptions, but session.send() stays untyped (object: unknown).

🤖 Generated with Claude Code

https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR


Generated by Claude Code

claude added 2 commits October 1, 2026 14:35
…) yields a typed, validated result.object

- ExecuteOptions.output / createAgent({ output }): the system prompt gets an
  "## Output format" block (schema as JSON Schema via the ai SDK's zodSchema),
  and every model call carries GenerateOptions.responseFormat
  ({ type: 'json', schema }), which the ai-SDK providers map to a JSON-mode
  experimental_output that leaves prompt and text untouched.
- The final reply is parsed (code fences tolerated) and validated; an invalid
  reply gets one repair step (a user message listing the issues, counted
  against maxSteps); still invalid -> finishReason 'output-invalid',
  no object, outputError { message, issues }.
- result.object is typed z.output<schema> for createAgent() agents
  (SimpleAgent / AgentRun / ExecutionResult gain a defaulted type param);
  run.done gains an optional, additive `object` field.
- toolArgsValidation exports parseWithIssues()/formatIssues() so tool args
  and output share the zod issue formatting.
- Docs: docs/structured-output.md, api-overview, streaming, README, CHANGELOG,
  regenerated llms files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR
@LinuxDevil
LinuxDevil merged commit 02983fe into main Oct 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants