Skip to content

fix(core): retry invalid structured model responses - #1433

Open
BlueCatPro wants to merge 1 commit into
google:mainfrom
BlueCatPro:fix/set-model-response-validation-feedback
Open

fix(core): retry invalid structured model responses#1433
BlueCatPro wants to merge 1 commit into
google:mainfrom
BlueCatPro:fix/set-model-response-validation-feedback

Conversation

@BlueCatPro

@BlueCatPro BlueCatPro commented Aug 15, 2026

Copy link
Copy Markdown

Link to Issue or Description of Change

1. Link to an existing issue:

Problem:

When set_model_response arguments fail output-schema validation, the validation exception prevents the model from receiving actionable feedback and retrying with a corrected structured response.

Solution:

  • Return concise schema-validation feedback from set_model_response so the model can retry.
  • Record successfully validated responses in EventActions.
  • Promote only validated responses to the final structured model output.
  • Remove the appended full schema from validation feedback to keep it concise.
  • Add unit and Runner-level regression tests covering validation failure, feedback propagation, retry, and successful final output.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • The unit tests relevant to this change pass locally.

Tests executed:

  • EventActionsTest
  • OutputSchemaTest
  • SetModelResponseToolTest

Result:

  • BUILD SUCCESS
  • 27 tests
  • 0 failures
  • 0 errors

Manual End-to-End (E2E) Tests:

Not run against a live model. A Runner-level regression test using the scripted TestLlm verifies that validation feedback from the invalid call is included in the retry request and that only the corrected, validated response becomes the final structured output.

Checklist

  • I have read the CONTRIBUTING.md document.
  • My pull request contains a single commit.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective.
  • New and existing tests relevant to this change pass locally.
  • I have manually tested the change end-to-end with a live model.
  • No dependent downstream changes are required.

Additional context

This is my first open-source contribution and my first contribution to adk-java. I may have missed some repository conventions, so I would appreciate any guidance and am happy to make follow-up changes. Thank you for your time and review!

@google-cla

google-cla Bot commented Aug 15, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@hemasekhar-p hemasekhar-p self-assigned this Aug 17, 2026
@hemasekhar-p

Copy link
Copy Markdown
Contributor

Hi @BlueCatPro, thank you for your contribution! We appreciate you taking the time to submit this pull request. Currently this PR is under review by our team, we will keep you posted if any additional information is required. thank you.

@sherryfox sherryfox left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @BlueCatPro, thanks for this — and for the issue write-up that preceded it.

I left one parity question about what gets stored so please take a look 🙏

+ "\nRecall the set_model_response function correctly, fix the errors, and"
+ " call it again with all required fields using the correct types.");
}
toolContext.actions().setSetModelResponse(args);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Python records the validated and dumped object here, not the input:

validated_response = self.output_schema.model_validate(args)
result = validated_response.model_dump(exclude_none=True)
...
tool_context.actions.set_model_response = result

SchemaUtils.validateMapOnSchema returns void, so there's nothing to store here but args. Confirmed on this branch: {"a":"x","b":null} passes validation unchanged and lands in setSetModelResponse as-is, where Python would emit {"a":"x"}.

Is the difference intended?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this, and sorry I missed this distinction. The difference was not intended.
I’ve updated SetModelResponseTool to create a copy after successful validation, omit null-valued fields to match the concrete model_dump(exclude_none=True) behavior you identified, and use the same result both as the tool return value and the value stored in EventActions.

@BlueCatPro
BlueCatPro force-pushed the fix/set-model-response-validation-feedback branch from e4880d1 to 6a562f2 Compare August 30, 2026 10:18
@BlueCatPro
BlueCatPro force-pushed the fix/set-model-response-validation-feedback branch from 6a562f2 to fa32b78 Compare August 30, 2026 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SetModelResponseTool: schema validation failure terminates the tool flow instead of returning feedback

3 participants