Support images and PDFs from tools and sub-agents - #92
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tools previously left images and PDFs as stored output or rejected them as non-text files, so models could not inspect or reference them. This change delivers admitted media through native provider content and lets models link it in replies with stable
media:handles.Changes
read_fileon both VFS and execution-environment filesystems. Supported images and PDFs are stored in CAS; unsupported or excess media produces an omission note or tool error.media:links in the transcript, using browser object URLs for clickable previews with per-view cleanup and loader-scoped fetch caching.Validation
cargo test -p engine -p temporal-workflow -p tools -p llm-runtime -p api -p api-projection -p temporal-server --lib --quiet— 1,172 passed, 1 ignored.npm run test --workspace @lightspeed/platform-web -- src/components/session/media.test.tsx src/lib/sessions/transcript.test.ts src/pages/SessionsPage.test.tsx— 85 passed.npm run typecheck --workspace @lightspeed/platform-web— passed.git diff --check origin/main...HEAD— passed.Live/credentialed suites were not run during PR preparation.
Deployment and accepted limits
The await/join resume activity contract and scheduling change without a Temporal versioning patch. Existing session workflows whose histories already exercised the affected resume paths must be closed before deployment.
The web resolves handles against the loaded transcript; a reference whose originating media is outside that loaded history may remain unresolved. Sub-agent hand-off resolves links against the child's active context, so media compacted out before its final answer is not handed up. A session-wide media lookup is deferred.