You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
CUDA kernels: profile the misaligned (atomicAdd) kernels and tune their split separately #91
#85 profiled only the aligned kernels (matmul_delta_csr, rmatmul_delta_csc) with ncu. The misaligned, atomicAdd kernels are unprofiled, and they are now the slower pair relative to cuSPARSE (kernel time over cuSPARSE, benchmarks.run --set cuda):
Each call issues nnz x width atomic adds (~96M at width 16). The chunk split from #85 helped VCSC @ B at widths 16 to 32 (0.91x and 0.85x of the previous time) but was neutral at 64 and above (1.01 to 1.02x) and measured 1.07x slower at width 1. The last one is not explained, since width 1 does not split. I tuned one split policy for all four kernels; the misaligned ones may want their own.
To investigate
ncu on matmul_delta_csc and rmatmul_delta_csr: atomic throughput, L2 contention, where the time actually goes.
Warp-level aggregation when several lanes in a lane-group target the same output element (a scattered cell id rarely repeats, but a scattered gene id does, in B @ VCSR).
Investigated on the RTX 5080 (sm_120), against the kernels after #85/#97/#95. There is no significant speedup left in the misaligned kernels, so I'm closing this.
Profile (ncu).VCSC @ B, width 16, uniform 60,000 x 2,000 at 5%: 314 us, L2 throughput 93% of peak (L1 hit 37%, L2 hit 97%, DRAM 6%). It is bound by L2 atomic throughput, with nothing left on the SM side to tune. B @ VCSR on a Zipf-skewed matrix (hot genes in most cells): 594 us at only 33% L2 throughput. There the time goes to same-address serialization on the hot genes' output elements, not to bandwidth.
Warp-level aggregation: not applicable by construction. A warp covers one slice (or one chunk of one), and the minor indices inside a slice are distinct. A cell appears once per gene in VCSC @ B, and a gene once per row in B @ VCSR. So lanes of a warp never target the same output element. The contention in B @ VCSR is across warps (many rows hitting the same hot gene), which warp aggregation cannot reduce.
Vector atomics (atomicAdd(float4*), sm_90+): implemented and measured, ~0-9% at best. Uniform: 0.95-1.01x at widths 1-64. Skewed: VCSC @ B 0.76x at width 4 and 0.91-0.99x elsewhere. B @ VCSR 1.28x slower at width 4 and 0.96-1.00x elsewhere. This matches the earlier note in _VECTOR_WIDTH: the L2 atomic units are limited by elements updated, not instructions issued. It would also need an sm_90+ gate, so I did not keep it.
Splitting the misaligned kernels. The chunk split is neutral at width 64+ (VCSC @ B 920 us before and after). The width-1 "1.07x slower" is within run-to-run noise: width 1 does not split, and the whole-slice kernel is unchanged. Re-measured 139 vs 149 us across separate runs. The real problem for these kernels was skew, which is fixed in #100 (#92): skewed VCSC @ B goes 537 -> 113 us at width 1, 1499 -> 151 us at width 4, and 1073 -> 271 us at width 16.
Not tried: shared-memory privatization of the output for B @ VCSR (output indexed by gene, so hot-gene contention would go away). It only fits when n_genes * width * 4 bytes is under ~48-100 KB (width <= ~6 at 2,000 genes), so it would help narrow widths only. If skewed B @ VCSR at those widths matters downstream, that is the lead to open a new issue for.
Activity
aarmey commentedon Nov 15, 2026
Investigated on the RTX 5080 (sm_120), against the kernels after #85/#97/#95. There is no significant speedup left in the misaligned kernels, so I'm closing this.
Profile (ncu).
VCSC @ B, width 16, uniform 60,000 x 2,000 at 5%: 314 us, L2 throughput 93% of peak (L1 hit 37%, L2 hit 97%, DRAM 6%). It is bound by L2 atomic throughput, with nothing left on the SM side to tune.B @ VCSRon a Zipf-skewed matrix (hot genes in most cells): 594 us at only 33% L2 throughput. There the time goes to same-address serialization on the hot genes' output elements, not to bandwidth.Warp-level aggregation: not applicable by construction. A warp covers one slice (or one chunk of one), and the minor indices inside a slice are distinct. A cell appears once per gene in
VCSC @ B, and a gene once per row inB @ VCSR. So lanes of a warp never target the same output element. The contention inB @ VCSRis across warps (many rows hitting the same hot gene), which warp aggregation cannot reduce.Vector atomics (
atomicAdd(float4*), sm_90+): implemented and measured, ~0-9% at best. Uniform: 0.95-1.01x at widths 1-64. Skewed:VCSC @ B0.76x at width 4 and 0.91-0.99x elsewhere.B @ VCSR1.28x slower at width 4 and 0.96-1.00x elsewhere. This matches the earlier note in_VECTOR_WIDTH: the L2 atomic units are limited by elements updated, not instructions issued. It would also need an sm_90+ gate, so I did not keep it.Splitting the misaligned kernels. The chunk split is neutral at width 64+ (
VCSC @ B920 us before and after). The width-1 "1.07x slower" is within run-to-run noise: width 1 does not split, and the whole-slice kernel is unchanged. Re-measured 139 vs 149 us across separate runs. The real problem for these kernels was skew, which is fixed in #100 (#92): skewedVCSC @ Bgoes 537 -> 113 us at width 1, 1499 -> 151 us at width 4, and 1073 -> 271 us at width 16.Not tried: shared-memory privatization of the output for
B @ VCSR(output indexed by gene, so hot-gene contention would go away). It only fits whenn_genes * width * 4bytes is under ~48-100 KB (width <= ~6 at 2,000 genes), so it would help narrow widths only. If skewedB @ VCSRat those widths matters downstream, that is the lead to open a new issue for.