[LinuxDevil]

What

• Message.content is now string | ContentPart[]. The part types are { type: 'text', text }, { type: 'image', image: string | Uint8Array, mimeType? } and { type: 'file', data: string | Uint8Array, mimeType, filename? }. New exported types: ContentPart, TextContentPart, ImageContentPart, FileContentPart. New helper textOf(message) (exported from the root and ./providers) returns the string content, or the text parts joined.
• AiSdkProvider.convertMessages turns the parts of a user message into ai v4 TextPart / ImagePart / FilePart for all four built-in providers. System, assistant and tool messages are sent as their text.
• Images are passed through to every provider. For Anthropic and Ollama (supportsImageUrls = false) the ai SDK downloads an image URL before sending it.
• Files become a [file <name> (<mimeType>) not sent] text note, and each provider prints one console.warn. The reason is the peer versions this package pins (@ai-sdk/openai / @ai-sdk/anthropic 0.0.42, ollama-ai-provider 1.2):
  • OpenAI's converter maps a file part to undefined.
  • Anthropic and Ollama silently drop file parts.
  • A subclass can set protected readonly acceptsFileParts = true to send real FileParts instead.
• Code that read content as a string now uses textOf():
  • estimateTokens: text parts count as text, and each image or file part counts as a flat 1,000 tokens.
  • Compaction: pruning a tool result, and building the summary text.
  • recordReplay fingerprint: stores the text plus a new optional attachments list, holding the URL or a 32-bit FNV-1a digest of the image/file bytes. The cassette schema accepts that field.
  • mockModel's error text, and MockLLMProvider's tool matching and token count.
  • The React reducer: ui.send also accepts parts.
• FileSessionStore saves Uint8Array/Buffer values as { "$bytes": "<base64>" } and loads them back as Uint8Array. MemorySessionStore already kept them through structuredClone.
• Docs:
  • New docs/providers.md "Multimodal input" section, with a type-checked Message[] example.
  • docs/api-overview.md row and note.
  • docs/testing.md normalize example narrows content, so it type-checks again.
  • CHANGELOG: an Added entry, plus a Changed entry for the type widening (marked BREAKING, types only, with a migration note).
  • llms files regenerated.

Why

LOU-V11 / Epic V: the "Multimodal input" row of the feature matrix is ❌, while eve and open-harness accept image and file parts. Before this change, Message.content could only be a string.

Acceptance criteria

• Message.content: string | ContentPart[] with the specified part shapes. textOf(message) is used for estimateTokens, compaction, fingerprint, mockModel, the React reducer, and the session store's base64 round trip of Uint8Array. String content works unchanged everywhere.
• convertMessages maps parts to the ai v4 user-content parts for all four providers; assistant/tool messages stay text. Providers that cannot take a part type (file parts with the pinned peers) degrade it to text with a warning; this is documented in docs/providers.md.
• Contract test src/providers/multimodal.contract.test.ts uses ai/test's MockLanguageModelV1 and checks, for each provider, that the prompt the model receives contains the image part (as bytes, from a data URL, and as a URL). It also covers file degradation, the acceptsFileParts FilePart mapping, textOf, and the AgentExecutor.execute({ input: Message[] }) path. There are new estimateTokens, compaction and recordReplay fingerprint tests for part content, plus a FileSessionStore round-trip test and a reducer test.
• agent.send(parts) / session.send(parts) / t.send are left for LOU-V12. src/createAgent.ts and src/session/AgentSession.ts are untouched.
• Docs: the providers.md "Multimodal input" section (type-checked snippet), api-overview.md, regenerated llms files, and CHANGELOG.

Outside the listed scope (the build forced it): widening the type broke tsc in src/execution/genAiSpans.ts (3 reads of content as a string, for span content attributes) and src/execution/transcript.ts (one startsWith). Each read was replaced with textOf(); there are no other changes in those files. Two other out-of-scope edits: src/testing/cassette.ts gained the optional attachments field in its schema (without it zod strips the field on load), and docs/testing.md's normalize snippet was updated.

How verified

• npx tsc --noEmit: clean.
• npm run lint: 0 errors, 404 warnings. The changed files show only warnings that were already there; no new ones.
• npm run test:types: 6 files, 26 tests passed, no type errors.
• npx vitest run (after the build): 153 of 154 files passed; 2248 tests passed, 1 failed, 3 skipped.
  • The failure is cloudflare.test.ts > wrangler dev serves the worker on the local workerd runtime. It is not caused by this PR: workerd fails with No such module "crypto", imported by src/execution/toolRunContext.ts (import { randomUUID } from 'node:crypto', which is already on main from LOU-U15).
• npm run build && npm run build --workspace=packages/create-loushy-agent: passed.
• npm run docs:verify-snippets -- --skip-build: all 116 snippets type-check against src/; 6 also run cleanly.
• npm run docs:llms:check: passed (exit 0).
• npm run test:coverage && npm run fallow:
  • Coverage only fails on the wrangler test above, so it was re-run with --coverage.reportOnFailure to write the coverage file.
  • fallow exits 0 with "No issues found". It also lists one OpenAI/AnthropicProvider duplicate clone group that this PR does not touch.

🤖 Generated with Claude Code

https://claude.ai/code/session_016bY29ibQgv3MDdjAbUT4YR

---

Generated by Claude Code