Skip to content

Attribute stable-scatter parallel utilization - #64

Draft
mbbill wants to merge 3 commits into
mainfrom
compute/scatter-utilization
Draft

mbbill wants to merge 3 commits into
mainfrom
compute/scatter-utilization

Conversation

@mbbill

@mbbill mbbill commented Sep 14, 2026

Copy link
Copy Markdown
Owner

The stable-scatter trial on main showed about 1.78 occupied CPUs for Whitefoot versus 4.04 for the native oneTBB chain at W8. This PR investigates which execution phase and work-supply mechanism account for that gap, then tests a narrowly supported change using fixed-source Whitefoot A/B controls.

The initial commit records discriminating criteria before measurement. The investigation includes partitioning, count tally, packing, final copy, allocation and initialization, with the existing independent oracle and native controls. Compiler lowering is inspected before extending the existing trace, and observed images are checked for perturbation. General scheduling/profile/PGO, broad representation redesign and I/O remain separate work.

  • Scope: Ongoing investigation from main 277a1844; no completion/DCR review yet. Adds a separate coarse-phase diagnostic image with unchanged joins and a generated-module check wired to programs-check. Candidate 70c86e5b routes buffer/slice aggregate transfers through the existing snapshot-copy path, with a pending storage-placement amendment and premeasurement retention criterion. No specification or live-tree change.
  • Checks: Before the compiler candidate: ordinary scatter verification across all available forms/widths and the null twin; diagnostic image and missing-input check; diagnostic WF/oneTBB oracle at W1/2/4/8 (3,466,725 values each): pass. Candidate gate-profile compiler build, formatting and git diff --check: pass. Candidate correctness tests and images are building under severe local contention. Full make check has not run for this revision.
  • Findings: Fresh null run at W8: WF wall 3.961 ms / CPU 6.804 ms, oneTBB chain 1.573 / 6.111 ms; paired null WF median wall ratio 1.006. A subsequent phase/perturbation run encountered severe external load (system load over 200 while another worktree ran tests); ordinary WF wall rose to ~96 ms. That run is unsuitable for performance selection and will be retained as contaminated evidence. No runtime improvement is claimed. Static inspection finds an adequately expanded outer map, a budget-limited packing continuation, and large aggregate transfers in serial tally/packing; phase evidence and a causal intervention remain pending.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant