Skip to content

fix[next-dace]: do not relocate distributed buffer write back into loop bodies - #2818

Open
edopao wants to merge 2 commits into
GridTools:mainfrom
edopao:fix/dace-buffer-relocator-loop-guard
Open

fix[next-dace]: do not relocate distributed buffer write back into loop bodies#2818
edopao wants to merge 2 commits into
GridTools:mainfrom
edopao:fix/dace-buffer-relocator-loop-guard

Conversation

@edopao

@edopao edopao commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

DistributedBufferRelocator can move the final write back of a temporary into a state nested inside a LoopRegion. The write back then executes in every loop iteration on data that the loop has only written partially, racing with the per-iteration partial writes and producing nondeterministic garbage in the unaffected parts of the output.

This is what broke the scan tests on the run_dace_stree_* backends in #2690 (e.g. CI run 32715667466): for a scan, the state that (partially) defines the scan temporary is the scan loop body, and the full-array write back after the loop was relocated into it.

Enabler: since DaCe 2.0, SDFG.states() recurses into control-flow scopes (return list(self.all_states())), so find_upstream_states() now also discovers loop-body states as candidate definition locations — on top-level-only semantics this relocation could never have happened.

Fix

Never relocate into a state that is (transitively) nested inside a LoopRegion: _find_candidates() now skips such definition states via the new _is_inside_loop() helper. Relocation into conditional branches — the use case this pass was designed for — remains legal, as does any relocation into states that are not executed repeatedly.

Test

New unit test test_distributed_buffer_definition_in_loop builds the scan-shaped SDFG (per-iteration write of t[k] inside a loop body, full write back t -> b in the state after the loop) and asserts that the write back stays after the loop and that no write to the output is added inside the loop. The test fails without the fix (the pass relocates into the loop body) and passes with it.

Validation

  • New reproducer test: fails without the fix, passes with it.
  • Full transformation_tests/ unit suite: 326 passed, 3 xfailed (pre-existing).
  • On feat[next-dace]: lowering to SchedTree #2690's branch, an equivalent guard made all 32 CI-failing scan tests pass locally (incl. 5/5 stress runs of test_scan.py::test_solve_triag and test_icon_like_scan.py across all four run_dace_stree_* backends).

@edopao edopao changed the title fix[next-dace]: Do not relocate distributed buffer write back into loop bodies fix[next-dace]: do not relocate distributed buffer write back into loop bodies Aug 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant