Repository navigation
docs: add run arguments and Message input to eval suite Case - #770
Closed
harshsinha03 wants to merge 1 commit into
Closed
harshsinha03 wants to merge 1 commit into
harshsinha03 wants to merge 1 commit into
Conversation
sannya-singal
approved these changes
Oct 19, 2026
Comment on lines
+153
to
+161
| name="offers_slots_for_the_pictured_venue", | ||
| agent=agent, | ||
| input=Message( | ||
| role="user", | ||
| content="I'd like to book a call about a tour of this landmark. Which one is it, and what times do you have?", | ||
| images=[Image(url="https://upload.wikimedia.org/wikipedia/commons/0/0c/GoldenGateBridge-001.jpg")], | ||
| ), | ||
| criteria="Names the Golden Gate Bridge and offers Tuesday 10:00 and Wednesday 15:00.", | ||
| expected_tool_calls=("get_open_slots",), |
Contributor
There was a problem hiding this comment.
This snippet is missing user_id and session_state from the cookbook case.
|
|
||
| `session_id`, `user_id`, `session_state`, `dependencies`, `metadata`, and `knowledge_filters` are passed to `agent.arun()` or `team.arun()` for the case's turn. A case that sets none of them runs in a generated session, with the agent or team's own `user_id`, if any, and no state, dependencies, metadata, or filters. | ||
|
|
||
| Each run gets its own copy of `session_state` and `dependencies`, so a run that writes into them leaves the `Case` unchanged. |
Contributor
There was a problem hiding this comment.
dependencies values that can't be deep copied are passed as the same object, so they are shared across runs. Maybe add: "Values that can't be deep-copied are passed through as is."
Member
|
wrong repository , please raise prs here : https://github.com/agno-agi/agno-docs/pulls |
Contributor
Author
mb, raising it there, thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Documents the run arguments and
Messageinput added to the eval suiteCasein agno-agi/agno#10829.evals/suite/overview.mdx:inputnow takesstrorMessage; new rows forsession_id,user_id,session_state,dependencies,metadata, andknowledge_filters; new sections Run arguments, Multi-turn cases, and Media input.reference/evals/suite.mdx: the same rows on theCasetable;CaseResult.session_idisCase.session_idwhen set, else generated.examples/evals/suite/suite-session.mdx(new): thesuite_session.pycookbook, registered indocs.jsonafter Suite Team Scoring.Type of Change
Related Issues/PRs (if applicable)
Checklist
scripts/make_openapi.py --checkpasses (if applicable)mint broken-links -t falseandmint validate -t falsepass