[dnikolaev-amd]

Summary

This branch continues meta-pytorch/KernelAgent#119 and is rebased onto current main. #119 is the base for the ROCm/AMD path: --target-platform rocm, rocprof profiling in place of NCU, heuristic roofline analysis, Instinct GPU specs (MI300X, MI300A, MI350X, MI250X, MI250, MI210, MI100), and platform-registry components under rocm.

The full #119 conversation was read, and every requested change from that thread is applied here. Further changes came from testing and benchmarking on MI300X, MI350X, and H100. The commit history has the per-change detail.

ROCm 10.1 is the supported floor. Profiling uses rocprofv3 only. GPU listing and the MI250/MI300 chip id use amd-smi only.

• The worker no longer falls back to rocprof or rocprofv2. That removes the ROCm 6.4 rocprof --stats path, which reported a completed profile with empty counters.
• amd-smi is taken from $PATH, from beside the unresolved Python (a venv interpreter is often a symlink whose target has no sibling amd-smi), then from /opt/rocm/bin. HIP ids come from amd-smi list -e. DEVICE_ID and NUM_COMPUTE_UNITS come from amd-smi static --asic.
• The README installs torch==2.14.0+rocm10.1.0 from stable.repo.amd.com inside a venv. That wheel installs ROCm 10.1.0 with it, including rocprofv3 and amd-smi beside that Python.
• A run uses one vendor. NVIDIA is the default when nvidia-smi lists a GPU and no HIP or ROCR mask selects an amd-smi HIP id. AMD is used when nvidia-smi lists nothing, when the mask leaves a HIP id, or when a ROCm platform was requested and both masks are unset. An empty mask disables that vendor. A non-empty CUDA_VISIBLE_DEVICES together with a non-empty HIP_VISIBLE_DEVICES or ROCR_VISIBLE_DEVICES is an error. An explicit platform that does not get its own devices raises, and nothing listed raises. A ROCm worker removes CUDA_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES and sets HIP_VISIBLE_DEVICES to that HIP id. The integer in both ROCm masks is the HIP id, the same index as unmasked cuda:N, and the chip lookup reads the ASIC record from the same amd-smi listing that produced the id.
• duration_ms is the last kernel-trace dispatch of the non-at:: kernel with the largest TotalDurationNs. AverageNs is the mean of every launch in the process, including warmup. A later helper, such as a reduce after the main kernel, does not replace that time. A stats file with no AverageNs, or a trace with no matching dispatch, is not a profile. When a PMC file names kernels, each counter keeps the greatest Dispatch_Id of that same kernel. at:: rows are left out. A file with no kernel column still adds every value.

Review requests from #119

• examples/configs/amd.yaml for out-of-the-box --strategy amd runs (Jack Khuu). Strategy selection discovers configs by filename, so the strategy name is amd.
• Usage comments use --strategy amd and --strategy nvidia. The --config flag is not documented, matching the CLI (Jack Khuu, Z Jin).
• The README requires ROCm 10.1 or later and documents rocprofv3, amd-smi, and the amd quickstart (Jack Khuu).
• platform_kwargs.rocprof_bin_path selects the profiler binary, on the same pattern as ncu_bin_path. The manager baseline profiler and the workers both receive it (Jack Khuu). The binary is rocprofv3.
• rocprof_profiler.py uses dict, list, and | None. Each PMC group is its own rocprofv3 -i pass. The stats pass adds --kernel-trace and --output-format csv (Jack Khuu).
• The standalone synthetic bottleneck analyzer stays out of this PR, as requested. A narrower fallback remains: after rocprof fails, that session stays on synthetic analysis instead of profiling again (Jack Khuu, Kaiming Cheng).

Changes from testing and benchmarking

• rocprofv3 --pmc no longer hangs on gfx942. FETCH_SIZE and WRITE_SIZE share a PMC block, so each counter group is its own pass. gfx942 and gfx950 use TCC_EA0_* names. gfx90a uses unprefixed TCC_EA_*. Stats output is CSV, and the parser reads suffixed files and long-format Counter_Name rows (Kaiming Cheng). A pass that exits 0 but writes no CSV is not a completed profile. A missing PMC column is omitted rather than stored as 0. A non-zero exit or an empty CSV on TCC_EA0_* tries TCC_EA_*.
• Instinct spec rows follow the CDNA whitepapers. MI350X is 256 CUs, FP32 144.2, and FP16/BF16 2300, with no specs_confidence. L2 is the package total: 4 MB per XCD on CDNA 3 and 4 (MI300X and MI350X 32, MI300A 24) and 8 MB per GCD on CDNA 2 (MI250 and MI250X 16, MI210 8). MI300A peak bandwidth is 5.3 TB/s. MI100 FP32 is 46.1, the matrix headline, the same convention as MI300X. CDNA 2 stays on the vector FP32 numbers.
• ROCm kernel prompts no longer hard-code a SKU's bandwidth or CU count. Those numbers come from the spec row. The optimization prompt says CU on CDNA and SM on NVIDIA. Refinement includes the same ROCm kernel guidance. Where that guidance conflicts with the Triton programming guidelines, it wins: no tensor descriptors, no TMA, and no warp_specialize. CUDA kernel guidance is empty.
• After the rebase onto main, the ROCm path uses the platform attribute, accepts both ncu_metrics and rocm_metrics, and keeps a flat rocprof dict intact. A SOL of 0 does not turn profiling off for the rest of the session. Workers round-robin GPUs, drain the result queue while joining, and forward cached baseline_metrics.
• ROCm test timeout is 300s. CUDA and XPU stay at 30s.
• The Anthropic client sends temperature on the JSON body, and high_reasoning_effort sets output-config effort.
• On a machine that lists both vendors, an explicit ROCm request with no HIP or ROCR mask uses the amd-smi HIP ids. A mask that matches nothing raises ROCm platform was requested but no devices are visible. An explicit NVIDIA request with a matching HIP id raises NVIDIA platform was requested but ROCm devices were selected.

Z Jin's later note about illegal addresses in KernelBench level-1 SDPA at 20 rounds on MI300A was read. It asks whether the failure reproduces and does not request a code change. This branch does not include a fix for that report.

Benchmarks

Matvec C = A @ x, M=2048, K=1048576, BF16. RMSNorm, shape 112 x 64 x 512 x 512, BF16. All three runs used claude-opus-4-6, 8 rounds, and 4 workers.

Matvec

Version	MI300X	MI350X	H100
Eager torch.matmul	1.2031 ms	1.5973 ms	2.4036 ms
torch.compile	1.1794 ms	1.6011 ms	2.4507 ms
Starting Triton kernel	9.8626 ms	12.7146 ms	7.7503 ms
KernelAgent best	1.2108 ms	0.9114 ms	2.2109 ms

MI300X: 8.15x vs start, 0.99x vs eager, 0.97x vs compile. MI350X: 13.95x vs start, 1.75x vs eager, 1.76x vs compile. H100: 3.51x vs start, 1.09x vs eager, 1.11x vs compile.

RMSNorm

Version	MI300X	MI350X	H100
Eager	7.2097 ms	5.6796 ms	13.3056 ms
torch.compile	4.3376 ms	3.1459 ms	8.7059 ms
Starting Triton kernel	3.9234 ms	2.9189 ms	6.2523 ms
KernelAgent best	2.1854 ms	1.7616 ms	6.2024 ms

MI300X: 1.80x vs start, 3.30x vs eager, 1.99x vs compile. MI350X: 1.66x vs start, 3.22x vs eager, 1.79x vs compile. H100: 1.01x vs start, 2.15x vs eager, 1.40x vs compile.

Tests

One new file, tests/test_rocm_platform.py. No existing test file was changed.

• test_registry_lists_rocm_for_every_component
• test_registry_profiler_is_the_wrapper
• test_ncu_bin_path_is_accepted_as_rocprof_binary
• test_roofline_accepts_shared_ncu_metrics_keyword
• test_roofline_rejects_scalar_metrics
• test_flat_rocprof_metrics_are_not_unwrapped_to_a_scalar
• test_synthetic_mode_is_rocm_only
• test_timeout_default_is_platform_specific
• test_gfx_isa_names_map_onto_spec_database_keys
• test_gfx90a_chip_id_separates_mi210_and_mi250
• test_combined_mi250_name_uses_chip_id
• test_amd_smi_asic_follows_hip_id
• test_gfx942_chip_id_separates_mi300a_and_mi300x
• test_prompt_uses_cu_labels
• test_rocm_kernel_guidance_skips_sku_numbers
• test_optimization_prompt_uses_cu_labels_for_cdna
• test_refinement_prompt_includes_rocm_kernel_guidance
• test_bottleneck_analyzer_accepts_num_bottlenecks
• test_yaml_platform_kwargs_select_rocprof_binary
• test_missing_rocprof_csv_is_not_a_profile
• test_collected_rocprof_csvs_keep_measured_counters
• test_duration_is_the_last_triton_dispatch
• test_pmc_keeps_the_last_dispatch_of_the_profiled_kernel
• test_stats_csv_without_kernel_trace_is_not_a_profile
• test_stats_csv_without_averagens_is_not_a_profile
• test_rocprofv3_pass_without_csv_is_not_collected
• test_rocprofv3_uses_long_output_format
• test_rocprofv3_retries_ea_counter_after_nonzero_exit
• test_roofline_wrappers_expose_early_stop
• test_find_amd_smi_beside_python_before_resolved_path
• test_find_amd_smi_uses_opt_rocm_bin
• test_mixed_node_uses_nvidia_unless_hip_visible_devices
• test_path_binaries_use_nvidia_unless_hip_visible_devices
• test_no_listed_gpu_stops
• test_amd_smi_print_order_is_kept_when_nvidia_is_absent
• test_unmatched_hip_visible_devices_disables_amd
• test_pin_rocm_worker_unsets_cuda_visible_devices
• test_visible_hip_id_uses_the_detected_list
• test_chip_lookup_uses_the_listing_that_produced_the_hip_id
• test_manager_runs_the_vendor_the_visibility_rules_choose
• test_cuda_visible_devices_without_nvidia_smi_keeps_amd
• test_explicit_nvidia_without_listed_gpus_does_not_invent_one

Test plan

• python -m pytest tests -q --tb=line -o addopts= with CUDA_VISIBLE_DEVICES, HIP_VISIBLE_DEVICES, and ROCR_VISIBLE_DEVICES unset (95 passed, 1 skipped locally). The skip is test_oink_rmsnorm_sm100_correctness.
• The same command on H100 with CUDA_VISIBLE_DEVICES=0,1 and the two ROCm masks unset (95 passed, 1 skipped). The skip is the same oink test.
• Matvec and RMSNorm optimization on MI300X, MI350X, and H100, numbers above
• rocprofv3 PMC passes complete on gfx942 / gfx950 with FETCH_SIZE and WRITE_SIZE in separate groups

Ruff is unchanged. Unpinned ruff check . still fails on the existing tree, and this push does not fix that.

[dnikolaev-amd]

Hi @Jack-Khuu, @kaiming-cheng, @zjin-lcf,
could you please review this PR to enable KernelAgent on ROCm?

[kaiming-cheng]

@dnikolaev-amd thanks for the PR. could you fix the ci and ruff?

[dnikolaev-amd]

Hi @kaiming-cheng,
The test failure is fixed in dnikolaev-amd@d5e168f. Both Ubuntu and macOS failed only test_yaml_platform_kwargs_select_rocprof_binary, because building OptimizationWorker imports Benchmark, which imports torch. CI does not install torch. The test now uses a stand-in Benchmark.

Ruff still fails. Unpinned ruff check . reported 904 errors, and 809 are in files this PR does not change. main last passed that job on 2026-07-15, so a newer ruff is flagging the existing tree.

[zjin-lcf]

Hello @dnikolaev-amd, Will you add specs for MI100 and MI210 GPUs ?

[zjin-lcf]

Hello @dnikolaev-amd,

I ran the test files listed in the test plan (test_rocm_platform.py, test_platform_config.py, test_basic.py, test_config_consistency.py, test_config_injectable.py, tests/fuser/test_auto_agent_cache.py, and test_autopatch.py): 64 passed. Three profiler and roofline paths still look wrong from the code. I reproduced them locally by calling the functions, without a GPU.

1. Fallback rocprof --stats does not write the -o path

On ROCm 6.4, bin/rpl_run.sh is installed as rocprof. The same script in the ROCm 6.4 and 7.0 trees says --stats writes <output name>.stats.csv. tblextr.py builds that name with re.sub(r"\.csv$", ".stats.csv", csvfile), and the script removes the -o file before the run. The command constructed for the non-v3 binary is:

rocprof --stats -o rocprof_round_1_stats.csv -- python rocprof_wrapper.py

The summary file is rocprof_round_1_stats.stats.csv. _resolve_collected_csv looks for the literal -o path and for the rocprofv3 suffix _kernel_stats.csv. _parse_stats_csv then returns {}, and the merged metrics contain duration_ms: 0.0.

This is the fallback taken when rocprofv3 is not on PATH. The rocprofv3 command used for the MI300X and MI350X runs is a separate path and requests -f csv.

2. A missing CSV is still returned as a completed profile

_parse_stats_csv and _parse_pmc_csv return {} when the expected file is absent. _merge_rocprof_outputs fills every missing counter with 0.0, and profile_triton_kernel_rocm writes that JSON and returns. The dict is non-empty, so the worker treats the profile as successful. Merging two missing paths produces compute_sol_pct: 0.0, memory_sol_pct: 0.0, and duration_ms: 0.0. ROCmRooflineAnalyzer.analyze then reports an underutilized kernel at 0% efficiency, with the warning "No compute or memory activity detected in rocprof counters."

_run_rocprofv3_pmc_passes also continues when a pass exits 0 but {prefix}_counter_collection.csv is absent. The remaining passes are merged, and the missing counters stay 0.0. A partial counter set is reported as a measured profile. For example, SQ counters of SQ_INSTS_VALU=900 and SQ_INSTS_SALU=100 with no TCC rows become compute_sol_pct: 90.0, tcc_hit: 0.0, and tcc_miss: 0.0.

3. A multi-round ROCm run raises AttributeError at the stop check

optimization_orchestrator.py evaluates self.roofline_analyzer.config.early_stop after should_stop() returns true. That line is already on main. ROCmRooflineAnalyzerWrapper does not define config; NvidiaRooflineAnalyzer does not either.

The ROCm analyzer sets early_stop to false, but should_stop still returns true after five at-roofline rounds whose efficiency changes by less than 0.1%. The at-roofline threshold is 85%. Holding efficiency at 86% gives:

round 1: should_stop=False
round 2: should_stop=False
round 3: should_stop=False
round 4: should_stop=False
round 5: should_stop=True  reason='Converged (improvement 0.00% < 0.1%)'

The next access raises AttributeError: 'ROCmRooflineAnalyzerWrapper' object has no attribute 'config'.

ROCmWorkerRunner calls optimize_kernel with max_opt_rounds=1, so beam search does not reach this. OptimizationWorker.max_rounds defaults to 10, so a direct multi-round run does.

[dnikolaev-amd]

Hello @zjin-lcf,

MI100, MI210, and MI250 are in e7e26c7. A product name selects the row. A bare gfx90a name uses the PCI chip id, and stays MI250X when that id cannot be read.

2. A missing CSV is still returned as a completed profile:

Fixed in 94b2819. A stats or PMC file that rocprof does not write now raises, including a rocprofv3 pass that exits 0 with no counter CSV.

3. A multi-round ROCm run raises AttributeError at the stop check:

Fixed in 94b2819. Both analyzers expose config. ROCm early_stop stays false.

1. Fallback rocprof --stats does not write the -o path :

Still in progress. v1 and v2 are different tools, and neither matches the rocprofv3 command.

• v1 (rpl_run.sh) rejects --. --hip-trace --stats writes <name>.stats.csv and deletes the -o path. On ROCm 6.2 that path segfaults on Triton JIT kernels. On ROCm 7.0 MI250 the stats duration is real, but PMC exits 0 and writes only Index,KernelName, so every counter is 0.
• v2 rejects --stats and --. Timing is --kernel-trace timestamps in results_<name>.csv. A PMC line runs only if it contains pmc. Results land in pmc_<n>/results_<name>.csv.
• v3 needs --kernel-trace --stats and --. ROCm 7.0 ships rocprofv3, but that build is older and its command line differs: it rejects --version and -f, and requires --output-format csv. ROCm 10.3 accepts -f as the short form of --output-format. On gfx90a, ROCm 7.0 and ROCm 10.3 list TCC_EA_RDREQ_sum / TCC_EA_WRREQ_sum. The gfx942 names TCC_EA0_* are not in that list. On ROCm 7.0 a missing name exits 0 with no CSV.

A successful exit with the wrong filename or an empty counter file looks like a finished profile. That is the case you hit. v1 and v2 are not in the branch yet.

Do we need to support the older ROCm stacks, where rocprof is v1 or v2 and rocprofv3 is not available? ROCm 10.1 has rocprofv3 only. ROCm 7.0 has v1, v2, and an older v3 with different command line options.

[zjin-lcf]

@dnikolaev-amd
rocprofv3 is recommended by ROCm, so support of v1 and v2 is not needed. Thanks.

[dnikolaev-amd]

Hello @zjin-lcf,

This PR supports ROCm 10.1 and later only. Older ROCm can be used without a warranty: rocprofv3 arguments and output can be incompatible with the parser. rocprof and rocprofv2 are no longer called (1841b29). Every ROCm profile is rocprofv3 --kernel-trace --stats --output-format csv

MI100, MI210, and MI250 are in gpu_specs_database.py (e7e26c7). A bare gfx90a name stays MI250X. The chip id and CU count come from amd-smi static --asic.

1. A missing CSV fails the profile (94b2819). The stats pass raises when the CSV is absent or has no Duration(ns) row, and a PMC pass that exits 0 with no counter rows raises too. Those counters are not filled with 0.0.

2. ROCmRooflineAnalyzerWrapper.config forwards to the inner analyzer. early_stop stays false there, so should_stop() returning true does not stop the run. The NVIDIA analyzer still has early_stop true.

[zjin-lcf]

Hello @dnikolaev-amd,

Thanks for the update. MI100, MI210, and MI250 are in the spec table, both roofline wrappers expose config, and profiling calls rocprofv3 only. The stop check on a8406f2 is:

if should_stop and self.roofline_analyzer.config.early_stop:

ROCm early_stop stays false, so five rounds at 86% heuristic SOL no longer raise AttributeError, and the run continues.

Two profiler paths are still wrong. I called the functions on a8406f2 with no GPU.

1. A stats CSV with no AverageNs is still a completed profile

_merge_rocprof_outputs raises only when a path is missing. A file that exists is parsed, and a parse that finds no AverageNs is stored as duration_ms: 0.0. Counter keys absent from a PMC file that does exist are still filled with 0.0.

Input:

Name,Calls,Duration(ns)
kernel,1,2000000

Counter_Name,Counter_Value
SQ_INSTS_VALU,900
SQ_INSTS_SALU,100

_merge_rocprof_outputs returns this and does not raise:

duration_ms: 0.0
sq_insts_valu: 900.0
sq_insts_salu: 100.0
tcc_hit: 0.0
compute_sol_pct: 90.0
memory_sol_pct: 0.0

The same call raises RuntimeError: rocprof completed without a usable CSV when the stats path is absent, so the missing-file case is fixed. The stats pass itself only checks the process exit code. That raise is in the merge, after the PMC passes have already rerun the kernel. There is no check for a Duration(ns) row. The parser reads TotalDurationNs and AverageNs.

2. A non-zero exit on TCC_EA0_* never tries TCC_EA_*

Pass 3 is (["TCC_EA0_RDREQ_sum"], ["TCC_EA_RDREQ_sum"]). The second name runs only when the first exits 0 and writes no counter rows. A non-zero exit raises first.

A fake rocprofv3 that exits 1 on TCC_EA0_RDREQ_sum, and would write TCC_EA_RDREQ_sum,11 for the gfx90a name, produces:

[rocprof] Running PMC pass 3: TCC_EA0_RDREQ_sum
RuntimeError: rocprof PMC pass 3 failed (rc=1)

The only input file for that pass is rocprof_input_pass2_0.txt:

pmc: TCC_EA0_RDREQ_sum

rocprof_input_pass2_1.txt is never written. gfx90a (MI210, MI250, MI250X) needs TCC_EA_RDREQ_sum / TCC_EA_WRREQ_sum. An unknown name exited 0 with no CSV on ROCm 7.0. This branch's floor is ROCm 10.1. If rocprofv3 there returns non-zero for TCC_EA0_* on gfx90a, the new MI210 and MI250 rows cannot finish a profile.

3. MI100 FP32

The MI100 row sets peak_fp32_tflops to 23.1. AMD's MI100 product page lists 46.1 TFLOPS FP32, 184.6 TFLOPS FP16, and 92.3 TFLOPS BF16. 23.1 is the vector rate (120 CUs × 64 lanes × 2 × ~1.5 GHz). The MI300X row in the same table uses AMD's headline 163.4. Please use one convention for both.

[dnikolaev-amd]

Hello @zjin-lcf,

Both profiler cases and the MI100 FP32 row are updated. The profiler change is 717b6dd. The spec change is 1aab87f.

1. A stats CSV with no AverageNs

The two CSVs from the comment now raise RuntimeError: rocprof stats CSV has no AverageNs. The parser reads TotalDurationNs and AverageNs. A Duration(ns) column does not become duration_ms.

Missing PMC columns are left out of the result. They are not stored as 0.0. An empty PMC file raises rocprof PMC CSV has no counter rows. compute_sol_pct is set only when SQ_INSTS_VALU is present, and memory_sol_pct only when a VMEM counter is present.

test_stats_csv_without_averagens_is_not_a_profile uses this input. test_collected_rocprof_csvs_keep_measured_counters checks that absent keys stay absent. The AverageNs check is still in the merge, after the PMC passes.

2. A non-zero exit on TCC_EA0_*

A non-zero exit now tries the next name, as an empty CSV already did. A fake rocprofv3 that exits 1 on TCC_EA0_* writes rocprof_input_pass2_1.txt (pmc: TCC_EA_RDREQ_sum) and rocprof_input_pass3_1.txt (pmc: TCC_EA_WRREQ_sum) and keeps 11 and 7. The last name in a pass still raises rocprof PMC pass N failed (rc=...).

test_rocprofv3_retries_ea_counter_after_nonzero_exit covers that case.

3. MI100 FP32

peak_fp32_tflops is 46.1, the matrix headline from the MI100 product page and the CDNA whitepaper. FP16 stays 184.6 and BF16 stays 92.3. Vector FP32, 23.1, is only in the comment. That is the same headline convention as the MI300X row, 163.4.

Every Instinct row in the table was checked against the ROCm CDNA whitepapers: MI100, MI210, MI250, MI250X, MI300A, MI300X, and MI350X. The database matches those sources. CDNA 2 FP32 stays on the vector column: MI250X 47.9, MI250 45.3, MI210 22.6. The matrix headlines there are 95.7, 90.5, and 45.3. I can switch those three if you want the headline everywhere.

I am thinking of adding l3_cache_mb later. That is the 256 MB Infinity Cache on MI300/MI350X.

The suite was run again with no GPU visible: 82 passed on MI300X, MI250, and MI350.

[zjin-lcf]

Hello @dnikolaev-amd,

Thanks for the updates. I rechecked 717b6dd. The earlier items are fixed: a missing CSV or a stats file with no AverageNs now fails, a non-zero exit on TCC_EA0_* tries TCC_EA_*, both roofline wrappers expose config, and MI100 FP32 is 46.1. I then reproduced the remaining cases locally, without a GPU.

1. A completed profile still describes every dispatch in the process

The wrapper runs the kernel three times and then once more, and both rocprofv3 commands trace that whole process. There is no kernel filter or dispatch range.

_parse_pmc_csv adds every Counter_Value with the same name. _parse_stats_csv selects the row with the largest TotalDurationNs. I merged these two CSVs with _merge_rocprof_outputs:

Name,Calls,TotalDurationNs,AverageNs,Percentage
at::native::vectorized_elementwise,4,8000000,2000000,80
triton_kernel,4,2000000,500000,20

Dispatch_ID,Kernel_Name,Counter_Name,Counter_Value
1,triton_kernel,SQ_INSTS_VALU,100
1,triton_kernel,SQ_INSTS_SALU,100
2,triton_kernel,SQ_INSTS_VALU,100
2,triton_kernel,SQ_INSTS_SALU,100
3,at::native::vectorized_elementwise,SQ_INSTS_VALU,900
3,at::native::vectorized_elementwise,SQ_INSTS_SALU,0

The result is duration_ms: 2.0 and compute_sol_pct: 84.6, with sq_insts_valu: 1100. The duration is the PyTorch kernel, and the instruction mix includes that kernel. The Triton rows alone are 0.5 ms and 50% VALU. Those ratios become the roofline classification and are sent to the model. The first wrapper call is also the call that runs autotune, so slow configurations are included in AverageNs and in the summed counters.

Please keep one post-warmup dispatch of the Triton kernel, and exclude at:: kernels, for both the timing row and the counters.

2. An exit of 0 can reuse the previous kernel's CSV

Output names depend only on rocprof_round_{round_num} inside one artifact directory. The current kernel and the post-benchmark SOL profile use that same prefix. In the manager baseline directory, every parent profiled during one round does too. The new existence check accepts whatever file is already there.

I planted pmc.csv_pass0_0_counter_collection.csv containing TCC_EA0_RDREQ_sum,99, then ran a fake rocprofv3 that exits 0 and writes nothing for TCC_EA0_*. The pass completed, the merged CSV still contained 99, and rocprof_input_pass0_1.txt was never written, so the TCC_EA_* fallback did not run.

The same fake binary for the stats pass, with an existing rocprof_round_1_stats.csv_kernel_stats.csv for kernel_a, returned duration_ms: 5.0 for that old kernel. The stats pass checks only the exit code.

Please remove the expected stats and per-pass CSVs before each invocation, or give each kernel its own prefix, and accept a file only when that invocation created it.

3. Claude seed generation raises TypeError when temperature is set

get_response passes kwargs.get("temperature", 0.7) and then **kwargs. Seed generation calls _call_llm(..., temperature=0.8), and AnthropicProvider.supports_multiple_completions() is false, so this is the path it uses. Calling _create_kwargs(..., 0.8, temperature=0.8) raises:

TypeError: _create_kwargs() got multiple values for argument 'temperature'

The call fails before the API request. A call that does not include temperature, such as the reflexion request, still builds the parameters. Drop temperature from the forwarded keywords. get_multiple_responses has the same collision because it passes temperature= into get_response.

4. The ROCm 300-second test timeout does not apply to the dispatch CLI

resolve_test_timeout_s returns 300 only when the argument is None. Fuser/dispatch_kernel_agent.py defaults --test-timeout-s to 30 and passes that value through, so --target-platform rocm still uses 30 seconds. Default the flag to None.

TritonKernelAgent() and the UI omit it, so they do get 300 on ROCm. The pipeline and auto-router CLIs pass args.run_timeout_s (default 1200) and do not pass --test-timeout-s, so that flag currently has no effect there.

5. The optimization prompt does not receive the ROCm restrictions

Generation and refinement render kernel_guidance, including the prohibitions on tensor descriptors, TMA, and warp_specialize. render_kernel_optimization_prompt does not pass that text, and kernel_optimization.j2 has no place for it. The CU/SM labels in that template are present. This is the prompt used on every optimization round; refinement runs only after verification fails.

6. --strategy all now runs the AMD config on every host

examples/run_opt_manager.py treats every file in examples/configs/ as a strategy. amd.yaml selects the ROCm components and sets the GPU name to MI300X, so --strategy all on an NVIDIA host runs that configuration as well.

One timeout-budget note: one profile is one stats command plus six PMC commands, and each command can wait 600 seconds. That is longer than the 30-minute worker kill. A failed baseline profile also leaves profiling enabled, so the next round starts another profile. A ROCm baseline failure should disable profiling for that session, without changing the CUDA behavior.

[zjin-lcf]

Hello @dnikolaev-amd,

I ran the detection and stats paths on a mixed node: AMD Instinct MI100 (HIP_ID 0, DEVICE_ID 0x738c, 120 CUs, gfx908), Tesla M40 (nvidia-smi index 0). ROCm here is 7.1.1 and rocprofv3 is 1.0.0. No visibility variables were set in the environment.

_detect_gpus returns [0] from nvidia-smi and does not call amd-smi. The ROCm PyTorch build nevertheless sees the MI100 with CUDA_VISIBLE_DEVICES=0, because that is also its HIP id. CUDA_VISIBLE_DEVICES=1 makes that build report no device. A second NVIDIA index would therefore pin a ROCm worker to a device the ROCm runtime does not have. Detection needs to be platform-specific: an ROCm run should use the HIP ids even when nvidia-smi succeeds.

HIP_VISIBLE_DEVICES=9 and ROCR_VISIBLE_DEVICES=9 each make ROCm PyTorch report no device. amd-smi list -e still prints HIP_ID: 0 under both masks, and adding CUDA_VISIBLE_DEVICES=0 does not bring the device back. The detector never reads either ROCm mask, so it still schedules a worker.

The MI100 name path is right. detect_amd_gpu_name() returns AMD Instinct MI100, and amd-smi static --asic reports DEVICE_ID 0x738c and 120 CUs. The marketing name selects the spec row; 0x738c is not in the chip-id tables.

A real stats run confirms the dispatch aggregation. The process did x + y three times and then once more. rocprofv3 --kernel-trace --stats --output-format csv exited 0 and wrote stats.csv_kernel_stats.csv. The add kernel has Calls 4, TotalDurationNs 204960, and AverageNs 51240. The parser turns that into duration_ms 0.05124 for all four launches. The kernel trace has six dispatches: two fill kernels and four adds. The individual add durations are not the same, so the average is not the final launch.

[dnikolaev-amd]

Hi @zjin-lcf,

amd-smi ignores HIP_VISIBLE_DEVICES and ROCCR_VISIBLE_DEVICES. I'm going to apply these masks to amd-smi output.

You’re the first person I know who has Nvidia and AMD GPU installed together :) Unfortunately, I don’t have that kind of setup to test with, but I’ll try to come up with something. Is this mixed node available in the CI environment so that I can run tests?

[zjin-lcf]

Hello @dnikolaev-amd,

The mixed node is not in CI. I applied the detection change from this side and checked it on the machine with the MI100 and the M40: dnikolaev-amd#1

The variable is ROCR_VISIBLE_DEVICES. amd-smi ignores it and HIP_VISIBLE_DEVICES, and both name a HIP id. On this node that id is 0. ROCR_VISIBLE_DEVICES=2 is the HSA_ID printed by amd-smi, and it hides the MI100. An empty mask hides it too. nvidia-smi still reports the M40 as index 0, so a ROCm run now lists HIP ids and does not use that index.

HIP_VISIBLE_DEVICES=0 keeps the MI100 visible when CUDA_VISIBLE_DEVICES=1. CUDA_VISIBLE_DEVICES=1 alone hides it, and setting CUDA_VISIBLE_DEVICES=0 does not bring back a device that HIP_VISIBLE_DEVICES=9 hid. The worker clears an integer mask and then sets CUDA_VISIBLE_DEVICES to the HIP id. With HIP_VISIBLE_DEVICES=9, a ROCm OptimizationManager raises instead of scheduling a worker. An NVIDIA run still returns [0] from nvidia-smi.

tests/test_rocm_platform.py: 37 passed. The commit is 32158ca.

[dnikolaev-amd]

Hello @zjin-lcf,

Thanks for running this on the NVIDIA and AMD GPUs, and for dnikolaev-amd#1. That machine is not in CI, so the measurements are useful.

Detection is in dnikolaev-amd@e6054b0. A run uses one vendor. NVIDIA is the default when nvidia-smi lists a GPU and no HIP or ROCR mask selects an amd-smi HIP id. AMD is used when nvidia-smi lists nothing, or when the mask leaves a HIP id. An empty mask disables that vendor. HIP_VISIBLE_DEVICES=9 and ROCR_VISIBLE_DEVICES=9 select nothing, and a ROCm OptimizationManager raises instead of scheduling a worker. An explicit NVIDIA request with a matching HIP id raises as well. Nothing listed raises. There is no GPU 0 fallback.

On a mixed node with no mask, an explicit ROCm request raises because NVIDIA devices were selected. It does not switch over to the HIP ids.

amd-smi ignores both masks. The integer is a HIP id, not the HSA_ID, so ROCR_VISIBLE_DEVICES=2 hides the AMD GPU (HIP id 0). A non-empty CUDA_VISIBLE_DEVICES together with a non-empty HIP or ROCR list is an error here. On your machine HIP_VISIBLE_DEVICES=0 keeps the AMD GPU visible when CUDA_VISIBLE_DEVICES=1. This tree raises on that pair.

On your machine the AMD GPU is HIP id 0 and the NVIDIA GPU is nvidia-smi index 0. Leave the other mask unset.

NVIDIA:

unset HIP_VISIBLE_DEVICES ROCR_VISIBLE_DEVICES
CUDA_VISIBLE_DEVICES=0 python examples/run_opt_manager.py \
    --kernel-dir examples/optimize_01_matvec \
    --strategy nvidia

AMD, through HIP_VISIBLE_DEVICES:

unset CUDA_VISIBLE_DEVICES ROCR_VISIBLE_DEVICES
HIP_VISIBLE_DEVICES=0 python examples/run_opt_manager.py \
    --kernel-dir examples/optimize_01_matvec \
    --strategy amd

AMD, through ROCR_VISIBLE_DEVICES. The value is the HIP id:

unset CUDA_VISIBLE_DEVICES HIP_VISIBLE_DEVICES
ROCR_VISIBLE_DEVICES=0 python examples/run_opt_manager.py \
    --kernel-dir examples/optimize_01_matvec \
    --strategy amd

AMD, with both set. ROCR_VISIBLE_DEVICES selects HIP id 0. HIP_VISIBLE_DEVICES then indexes that list, so 0 is its first entry and the run stays on the AMD GPU:

unset CUDA_VISIBLE_DEVICES
ROCR_VISIBLE_DEVICES=0 HIP_VISIBLE_DEVICES=0 python examples/run_opt_manager.py \
    --kernel-dir examples/optimize_01_matvec \
    --strategy amd

HIP_VISIBLE_DEVICES=1 with that same ROCR list is past the end, so it selects nothing. --strategy amd then raises because the NVIDIA GPUs were selected.

With both ROCm masks unset, --strategy nvidia runs on the NVIDIA GPUs. The same command with --strategy amd raises "ROCm platform was requested but NVIDIA devices were selected":

unset HIP_VISIBLE_DEVICES ROCR_VISIBLE_DEVICES
python examples/run_opt_manager.py \
    --kernel-dir examples/optimize_01_matvec \
    --strategy amd

A matching AMD mask with --strategy nvidia raises "NVIDIA platform was requested but ROCm devices were selected":

unset CUDA_VISIBLE_DEVICES ROCR_VISIBLE_DEVICES
HIP_VISIBLE_DEVICES=0 python examples/run_opt_manager.py \
    --kernel-dir examples/optimize_01_matvec \
    --strategy nvidia

CUDA_VISIBLE_DEVICES=1 together with HIP_VISIBLE_DEVICES=0 raises for either strategy. That check happens before the strategy is read:

CUDA_VISIBLE_DEVICES=1 HIP_VISIBLE_DEVICES=0 python examples/run_opt_manager.py \
    --kernel-dir examples/optimize_01_matvec \
    --strategy amd

examples/configs/amd.yaml still sets gpu_name to AMD Instinct MI300X, and examples/configs/nvidia.yaml still sets it to NVIDIA H100 NVL 94GB. The mask chooses the device and does not change that name. For the AMD run, set gpu_name to AMD Instinct MI100.

The worker removes CUDA_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES and sets HIP_VISIBLE_DEVICES to the HIP id. The pin in dnikolaev-amd#1 clears the integer masks and then sets CUDA_VISIBLE_DEVICES to that id.

Stats duration is in dnikolaev-amd@8cbcd64. AverageNs is the mean of every launch in the process, including the three warmup calls. duration_ms is the last kernel-trace dispatch of the non-at:: kernel with the largest TotalDurationNs. A later reduce does not replace that time, so the four add launches are no longer stored as their average.

tests/test_rocm_platform.py passed. The full suite was 94 passed, 1 skipped on MI300X and on H100. The skipped test is the Blackwell oink check.

[zjin-lcf]

Hello @dnikolaev-amd,

Thanks for e6054b0 and 8cbcd64. The mask rules, the missing-GPU check, and the worker pin should stay. One selection path still refuses a ROCm run on a mixed node, and the counter pass still sums every dispatch.

An explicit ROCm platform should use the HIP ids when no mask is set

_select_gpus returns NVIDIA before it reads amd-smi when nvidia-smi lists a GPU and neither HIP_VISIBLE_DEVICES nor ROCR_VISIBLE_DEVICES is set:

nvidia_ids = _nvidia_gpu_ids()
if nvidia_ids and not rocm_names:
    return "nvidia", nvidia_ids

OptimizationManager then raises if platform is "rocm", or if the platform dict has worker_runner: rocm. examples/configs/amd.yaml has that key, so --strategy amd with no mask raises ROCm platform was requested but NVIDIA devices were selected. test_manager_runs_the_vendor_the_visibility_rules_choose locks that in. target_platform is not the signal: after a vendor is chosen, the constructor overwrites worker_kwargs["target_platform"].

On this machine both devices are index 0, so the failure is the raise, not a wrong integer. The MI100 is HIP id 0 and the M40 is nvidia-smi index 0. test_mixed_node_uses_nvidia_unless_hip_visible_devices is the case that separates them: nvidia-smi returns [0, 1] and amd-smi returns HIP ids [1, 2, 3, 4]. With no mask, _detect_gpus() calls only nvidia-smi and returns [0, 1]. A requested ROCm platform should call amd-smi and return [1, 2, 3, 4], which is what visible_hip_ids already returns when both ROCm masks are unset.

Please keep the rest of e6054b0:

• No mask and no requested ROCm platform stays on NVIDIA. _detect_gpus() with no platform still returns the nvidia-smi indexes, and it does not need to call amd-smi.
• A HIP or ROCR mask still goes through visible_hip_ids. ROCR_VISIBLE_DEVICES selects HIP ids, and HIP_VISIBLE_DEVICES indexes that result. ROCR_VISIBLE_DEVICES=1,2,3 and HIP_VISIBLE_DEVICES=2 stays [3]. A mask that matches nothing still does not switch a requested ROCm run onto the NVIDIA ids.
• An empty mask still disables that vendor. Nothing listed still raises no GPU visible, with no GPU 0 fallback. CUDA_VISIBLE_DEVICES still does not invent an NVIDIA GPU when nvidia-smi lists none. A non-empty CUDA_VISIBLE_DEVICES together with a non-empty HIP or ROCR list still raises the conflict.
• The worker pin stays _pin_rocm_visible_devices: remove CUDA_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES, and set HIP_VISIBLE_DEVICES to the HIP id. On this machine HIP_VISIBLE_DEVICES=0 shows the MI100. An empty CUDA_VISIBLE_DEVICES hides it, so removing the variable is the right pin.

The change is only the requested-ROCm, no-mask path. The strategy has already chosen the vendor. The mask should restrict that list, not be required to select AMD. On a host with several AMD GPUs and one NVIDIA GPU, --strategy amd would otherwise need every HIP id written into the environment.

The counters still add every dispatch

8cbcd64 fixes the duration. _busiest_kernel_name skips a stats row whose Name contains at:: and keeps the largest TotalDurationNs. A tie keeps the later row. _last_profiled_dispatch_ns then reads {stats path}_kernel_trace.csv and uses the last KERNEL_DISPATCH of that exact name, End_Timestamp - Start_Timestamp. duration_ms is that one launch, so the four x + y launches are no longer stored as their AverageNs. TotalDurationNs still includes warmup, so a slow autotune name can still be the one whose last launch is kept.

_parse_pmc_csv does not apply that choice. A file with Counter_Name and Counter_Value sums every Counter_Value for that name. Kernel_Name is ignored, including the review fixture:

Dispatch_ID,Kernel_Name,Counter_Name,Counter_Value
1,triton_kernel,SQ_INSTS_VALU,100
2,triton_kernel,SQ_INSTS_VALU,100
3,at::native::vectorized_elementwise,SQ_INSTS_VALU,900

The merged sq_insts_valu is 1100. The wide row format sums the same way: Kernel_Name is excluded from the counter columns, and the rows are not filtered. The duration can be the last Triton launch while compute_sol_pct still includes the PyTorch dispatches and the warmup launches. The same kernel filter used for the trace needs to apply to the counters: one post-warmup dispatch of that kernel, and no at:: row.

[dnikolaev-amd]

Hello @zjin-lcf,

dnikolaev-amd@1f633a4

--strategy amd with both ROCm masks unset now calls amd-smi and uses the HIP ids. On the mixed-node fixture that is [1, 2, 3, 4]. This command should no longer raise on mixed node:

unset HIP_VISIBLE_DEVICES ROCR_VISIBLE_DEVICES
python examples/run_opt_manager.py \
    --kernel-dir examples/optimize_01_matvec \
    --strategy amd

A mask that matches nothing stays empty and raises ROCm platform was requested but no devices are visible.

For the kernel chosen in dnikolaev-amd@8cbcd64, each PMC counter keeps the row with the greatest Dispatch_Id or Dispatch_ID. In the review fixture sq_insts_valu is 100. A file with no kernel column still adds every value.

[zjin-lcf]

Hello @dnikolaev-amd,

Thanks for 1f633a4. A requested ROCm run with both masks unset now uses the amd-smi HIP ids, and the PMC parser keeps one dispatch of the profiled kernel. On this head, python -m pytest tests -q -o addopts= gives 95 passed and 1 skipped, as in the test plan.

I checked the rest of the branch against main (e064717). Four issues remain. I reproduced each one by calling the code without a GPU. GPU listings, counters, and the LLM client are stand-ins.

1. GPU detection replaces the platform the caller passed

When gpu_ids is omitted, OptimizationManager.__init__ assigns the detected vendor to platform:

self.gpu_ids = detected
if detected:
    platform = vendor

That discards platform="noop" and every per-component dict. When nothing is listed, the constructor raises no GPU visible. main fell back to [0]. I built the manager on both branches with a stubbed nvidia-smi and no amd-smi:

Case	main	This PR
platform="noop", no GPU tool	NoOpVerifier, NoOpWorkerRunner, gpu_ids=[0]	RuntimeError: no GPU visible
platform="noop", nvidia-smi lists 0 and 1	NoOpVerifier, NoOpWorkerRunner, noop worker components	NvidiaVerifier, NvidiaWorkerRunner, profiler: nvidia
dict with nvidia verifier/benchmarker/worker_runner/specs_provider and noop profiler/roofline_analyzer/bottleneck_analyzer/rag_prescriber	dict kept	every component becomes nvidia
no platform, no GPU tool	NVIDIA, gpu_ids=[0]	RuntimeError: no GPU visible

examples/configs/noop.yaml describes itself as a "dry-run without GPU hardware". It sets platform: noop and no gpu_ids. With this PR, --strategy noop raises on a host with no GPU. On an NVIDIA host it runs real verification, benchmarking, and NCU. The detection tests pass only "nvidia", "rocm", or no platform. test_yaml_platform_kwargs_select_rocprof_binary uses a noop/rocm dict, but it passes gpu_ids=[0] and skips detection. None of them catch this.

Please apply the detected vendor only when platform is None. A noop platform should not need a GPU: skip detection, or keep [0] for it. A dict that names nvidia or rocm can still be checked against the detected vendor, as now, without rewriting its other entries.

2. The ROCm "SOL" values measure instruction mix, not percent of peak

_merge_rocprof_outputs sets compute_sol_pct to the VALU share of counted instructions and memory_sol_pct to the VMEM share. ROCmRooflineAnalyzer scales the memory value by the L2 miss rate. efficiency_pct is the larger of the two. Neither value is compared with the device's peak.

Take two matvec kernels from the PR benchmark shape (M=2048, K=1048576, BF16, 4 GiB read), on MI300X (peak_memory_bw_gbps 5300). Both read the same bytes, with TCC_HIT_sum=10 and TCC_MISS_sum=90. Kernel A uses dword loads: 8M SQ_INSTS_VMEM_RD, 2M SQ_INSTS_VALU, 2.4 ms. Kernel B uses dwordx4 loads: 2M SQ_INSTS_VMEM_RD, 2M SQ_INSTS_VALU, 1.2 ms. Through _merge_rocprof_outputs and ROCmRooflineAnalyzer().analyze:

Kernel	Time	compute_sol_pct	memory_sol_pct	efficiency_pct	at_roofline	Achieved bandwidth from FETCH_SIZE and duration_ms
A	2.4 ms	20.0	100.0	100.0	yes	1,790 GB/s, 33.8% of peak
B	1.2 ms	50.0	72.5	72.5	no	3,579 GB/s, 67.5% of peak

The slower kernel reports 100% SOL. kernel_optimization.j2 gives the model Memory SOL: 100.0% and At Roofline: Yes for A. The judger's roofline section prints the same values. _update_kernels keeps best_sol_kernel by this number, and new_sol > best_sol_sol can set best_round_num. early_stop=False prevents a stop, but the prompts and the SOL tracking still use the number.

The profiler already collects FETCH_SIZE, WRITE_SIZE, and duration_ms, and the spec row has peak_memory_bw_gbps. Memory SOL could be (FETCH_SIZE + WRITE_SIZE) / duration divided by peak bandwidth. If the instruction-mix values stay, please keep them out of efficiency_pct, at_roofline, and the * SOL lines of the prompt templates. Report them under their real names, as _ROCM_COUNTER_FIELDS already does ("instruction mix").

3. output_config breaks the Anthropic provider on older SDKs

The temperature change is right for the current SDK. anthropic 2.4.0 has no temperature parameter on Messages.create, so extra_body is the way to send it.

The same commit adds params["output_config"] = {"effort": "high"} whenever high_reasoning_effort is set. VerificationWorker._call_llm sets it whenever the worker's high_reasoning_effort is true, and both strategy configs set high_reasoning_effort: true. pyproject.toml lists anthropic without a version. With anthropic==0.49.0:

get_response(..., high_reasoning_effort=True)
TypeError: Messages.create() got an unexpected keyword argument 'output_config'

With high_reasoning_effort=False, the same call builds and sends the request. This change affects every Claude user, not only ROCm. Please move it to a separate PR. There, either pin a minimum anthropic version that accepts output_config, or send effort through extra_body like temperature.

4. CUDA no longer profiles the final best kernel when its SOL profile failed

The final roofline pass now requires best_ncu_metrics is not None:

if best_round_num > 0 and best_ncu_metrics is not None:

best_ncu_metrics is set only from new_kernel_metrics. If _profile_kernel_for_sol returns None (a transient NCU failure), a round can still become the best by runtime, and then the final pass is skipped. I ran optimize_kernel on an NVIDIA-style profiler (no platform attribute) for one round. The new kernel benchmarks at 1.0 ms against a 2.0 ms baseline, and _profile_kernel_for_sol returns None:

	best_round	profile_kernel(kernel_round_1.py) for the final roofline
main	1	called once
This PR	1	not called

_finalize_results then gets best_ncu_metrics=None on CUDA. Please keep the old condition on NVIDIA and skip the final pass only when profiler_is_rocm(self.profiler) and the session switched to synthetic mode.

Smaller items

• The per-round roofline log is now wrapped in except Exception: pass. On CUDA, that hides any NCU roofline error. The comment says the block is a no-op on ROCm, but it runs there too. Please catch only the expected case, or log the exception.
• The synthetic BottleneckResult text appears three times in optimization_orchestrator.py. One helper would keep the three copies the same.
• ROCmWorkerRunner and _rocm_worker_process copy NvidiaWorkerRunner and its worker function except for the device pin and target_platform. A shared runner with a pin hook would keep the two in sync.
• On ROCm, OptimizationManager.__init__ calls os.environ.pop("CUDA_VISIBLE_DEVICES") in the manager process. That changes the environment of whatever process created the manager. Workers already remove it in _pin_rocm_visible_devices.
• # A missing baseline SOL used to force synthetic mode, which also disabled NCU on CUDA describes the Add ROCm/AMD GPU support for kernel generation and optimization #119 branch. main never had that behavior.

Please keep the mask rules, visible_hip_ids, the CUDA/ROCm mask conflict check, the worker pin, the platform-specific timeout default, and platform_kwargs.rocprof_bin_path reaching both the manager's baseline profiler and the workers.