SM90_H100 DRAM clock gives 3.98 TB/s; both H100 variants report 2619 MHz

[gregoryjr-dev]

configs/tested-cfgs/SM90_H100/gpgpusim.config sets:

-gpgpu_n_mem 40
-gpgpu_clock_domains 1980:3960:3960:3106

I read in #516 that DRAM capacity is not modelled and that the channel count is scaled to match the target bandwidth rather than the real geometry, so I am asking about the bandwidth target specifically.

The fourth clock domain is the DRAM clock in MHz (gpu-sim.cc, init_clock_domains), and one DRAM atom is burst length x bus width (gpu-sim.h), here BL 4 x busW 8 x 1 = 32 B. With CCD=1 in -gpgpu_dram_timing_opt a column command can issue every cycle, so the ceiling is n_mem x 32 B x DRAM clock:

config	n_mem	DRAM clock	peak
SM90_H100 as shipped	40	3106 MHz	3.98 TB/s
SM90_H200 as shipped	48	3106 MHz	4.77 TB/s

NVIDIA specifies 3.35 TB/s for the H100 SXM5 80GB, and such a board reports clocks.max.memory = 2619 MHz, HBM3 at 5.238 Gbps. At 40 partitions, 2619 MHz gives 3.35 TB/s exactly. For the H200 the shipped 3106 lands within 1 percent of its 4.8 TB/s, so the value looks right there.

If the target is the SXM5, the clock looks about 19 percent high:

--gpgpu_clock_domains 1980:3960:3960:3106
+-gpgpu_clock_domains 1980:3960:3960:2619

If the target is the H100 NVL at 3.9 TB/s, then 3.98 is within 2 percent and the value is already doing its job, and a comment naming the part would settle it for the next reader. I ask only because the file is named SM90_H100, where an unqualified H100 usually reads as the SXM5.

For what it is worth, an H100 NVL reports the same clocks.max.memory = 2619 MHz as the SXM5: both are HBM3 at 5.2 Gbps, and the NVL's higher bandwidth comes from more channels rather than a faster clock. Its SM clock is 1785 MHz, where the config has 1980, which matches the SXM5, and -gpgpu_n_mem 40 matches the SXM5's five stacks where an NVL would want six. So the clock looks like the only value in the file pointing away from an SXM5.

Happy to send a PR for whichever is intended.

Related: SM90_H100 and SM90_H200 are the only configs in tested-cfgs that set -gpgpu_simple_dram_model 1. Is that deliberate for Hopper, and was the Hopper performance model validated with it? Asking because #516 points to Ramulator for detailed DRAM, and it would help to know which model the SM90 constants were tuned against before swapping one in.

[JRPan]

Okay, thanks for pointing that out. I'll correlate with the H100 clock you suggested and let you know. At the time of fixing it, I'll add you as a co-author.

Regarding the simple dram model, yes, it was validated on Hopper. The config included are the exact configs used in the paper. The old HBM model does not model HBM3, and a simple latency-bandwidth model is enough for non-HBM-related research. Our ultimate plan is to have both Ramulator and the simple DRAM model, configurable:
simple DRAM for non-HBM-related research, swap to Ramulator as needed.

The reason is mainly simulation speed. I believe Ramulator, being a detailed DRAM simulator, would take longer to simulate. Right now, we just don't have any motivation to work on that immediately, but it's planned. But we are happy to accept a PR for merging Ramulator.