fix[next-dace]: block state fusion creating a WAR hazard on global data - #2817
Open
edopao wants to merge 3 commits into
Open
fix[next-dace]: block state fusion creating a WAR hazard on global data#2817edopao wants to merge 3 commits into
edopao wants to merge 3 commits into
Conversation
GT4PyStateFusion merged two states whenever the consumer component of the second state exchanged a messenger with the producer component of the first state, assuming the transient dataflow imposes all necessary orderings. However, the messenger only orders the producer's writes before the consumer's accesses; it does not order the producer's reads of global data before the consumer's writes to the same data. If the first state reads a global field both upstream and downstream of the messenger it produces - a pattern generated by in-place update stencils such as update_theta_and_exner in icon4py - the merged state contains an unordered write-after-read pair on the global and the computation result becomes undefined. Add a node-level check to GT4PyStateFusion that rejects the fusion unless every read of the conflicting global in the first state is guaranteed to be dataflow-ordered before the consumer's write through an exchanged messenger, and a regression test.
edopao
force-pushed
the
fix/state-fusion-war-merge
branch
from
August 24, 2026 09:57
b68728e to
dd5890b
Compare
edopao
force-pushed
the
fix/state-fusion-war-merge
branch
from
August 24, 2026 09:59
dd5890b to
e76652f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
GT4PyStateFusionmerged two states whenever the consumer component of the second state exchanged a messenger with the producer component of the first state, assuming the transient dataflow imposes all necessary orderings. However, the messenger only orders the producer's writes before the consumer's accesses; it does not order the producer's reads of global data before the consumer's writes to the same data.If the first state reads a global field both upstream and downstream of the messenger it produces, the merged state contains an unordered write-after-read pair on that global, and the computation result becomes undefined.
This pattern is generated by in-place update stencils, e.g.
update_theta_and_exnerin icon4py:which the lowering turns into a compute state followed by a write-back state. The compute state reads
theta_vtwice — upstream to produce the new value (the messenger) and downstream, together with the messenger, in theexnerupdate. WhenGT4PyStateFusionmerged the write-back state into the compute state, the writetheta_v = theta_v_newwas dataflow-ordered after the first read oftheta_vvia the messenger, but not after the second read, since the reader of the messenger and the write-back are siblings downstream of it.Fix
Add a node-level check to
GT4PyStateFusion: when the consumer component writes global data that the producer component reads, the fusion is rejected unless every read of the conflicting global in the first state is guaranteed to be dataflow-ordered before the consumer's write through an exchanged messenger — i.e. there is a messenger that (a) is dataflow-upstream of the write in the second state and (b) every read of the global in the first state is dataflow-upstream of that messenger's producing AccessNode.This keeps allowing the safe patterns covered by the existing tests (
test_global_in_both_read_and_write,test_global_merge_2,test_non_concurrent_data_dependency,test_double_producer), where the write is chained through the messenger.Tests
test_global_read_up_and_downstream_of_messenger: without the fix the fusion is applied and executes with wrong results (verified numerically:u = (g+1)/(g+1) ≡ 1instead of(g+1)/g); with the fix the fusion is rejected.tests/next_tests/unit_tests/program_processor_tests/runners_tests/dace_tests/pass.dace_cpubackend, 177 tests), which was the original motivation.