Repository navigation
Fix 0-d new_zeros/new_ones in remove_const_ops; run test_quant_LLM without the KV cache - #296
Merged
shivadbhavsar merged 2 commits intoOct 28, 2026
Conversation
_remove_new_const_ops built the constant with torch.zeros(*size) / torch.ones(*size). For a 0-d result (size == []) that is a call with no size at all and raises "missing 1 required positional arguments: size", so any graph with a 0-d new_zeros/new_ones fails to compile. GPT2 under transformers 5.x produces one. Pass the size list as is. test_remove_const_ops gains 0-d new_zeros and new_ones cases: both fail with the TypeError before this change and pass after (9 passed).
With transformers 5.x, GPT2Model returns a DynamicCache in its output when use_cache is on (the default). torch.export cannot flatten that type, so the test failed in export_for_quantization before reaching MIGraphX. The test only compares last_hidden_state, so load the model with use_cache=False. Both parametrizations fail on master with the DynamicCache error and pass with this change and the remove_const_ops fix (gfx1100, torch 2.13, transformers 5.16.1). tests/dynamo: 378 passed, 10 skipped.
5 tasks done
shivadbhavsar
approved these changes
Oct 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
tests/dynamo/quantization/test_quantized_models_dynamo.py::test_quant_LLMfails on master with transformers 5.x (measured on 5.16.1; CI pins 4.41.2, which I have not run), and fixing that exposes a compile crash in the backend that any graph with a 0-dnew_zeros/new_oneshits, whatever the transformers version. No existing issue covers either.Technical Details
remove_const_ops(_remove_new_const_ops) built the constant withtorch.zeros(*size)/torch.ones(*size). For a 0-d result (size == []) that is a call without a size:TypeError: ones() missing 1 required positional arguments: "size". It now passes the size list as is.test_quant_LLM: under transformers 5.x, GPT2Model returns aDynamicCachein its output whenuse_cacheis on (its default), whichtorch.exportcannot flatten, so the test failed inexport_for_quantizationbefore reaching MIGraphX. The test only compareslast_hidden_state, so it now loads the model withuse_cache=False.Test Plan
tests/dynamo/passes/test_remove_const_ops.py: new 0-dnew_zerosandnew_onescases.test_quant_LLMon master vs this branch.tests/dynamo.Setup: RX 7900 XTX (gfx1100), ROCm 7.2, torch 2.13, transformers 5.16.1, MIGraphX develop (a3b5db6).
Test Result
test_quant_LLM: 2 failed on master (DynamicCache), 2 passed here.tests/dynamo: 378 passed, 10 skipped.Submission Checklist