[zohaibm-amd]

Overview

Ports vllm-project#60710 (merged upstream as 94ad4df) to gfx11. It replaces the gfx11-only --w4a16-prefill-dequant engine option with the upstream KernelConfig.rdna_hybrid_w4a16_prefill_dequant option. See vllm-project#60710 for the design, the M >= 256 threshold rationale, and the performance and accuracy data.

Breaking change: --w4a16-prefill-dequant is removed

The old option and ModelConfig.w4a16_prefill_dequant no longer exist. Passing --w4a16-prefill-dequant now fails argument.

Old	New
--w4a16-prefill-dequant off (default)	nothing (default is "disable")
--w4a16-prefill-dequant hard	--kernel-config '{"rdna_hybrid_w4a16_prefill_dequant": "enable"}'
--w4a16-prefill-dequant soft	no equivalent; use enable or leave it off

The dotted form --kernel-config.rdna_hybrid_w4a16_prefill_dequant enable also works. From Python, pass kernel_config={"rdna_hybrid_w4a16_prefill_dequant": "enable"}. Any scripts or CI configs that pass the old option must switch to the new one before picking up this change.

Behaviour differences vs the old hard mode

• Dense path only for M >= 256. It used to apply to every M > 5. Batches of 6–255 tokens (including spec-decode verify) now stay on the int4 Triton kernel, where dense was no faster (see [ROCm][Perf] W4A16: opt-in dense weight cache for RDNA hybrid prefill vllm-project/vllm#60710). Performance should be the same or better.
• Same memory use. The dense copy is the same size as before, padding included, so KV cache capacity is unchanged when the model fits.
• Stricter startup check. Before caching, the budget now also reserves 4 GiB of activation headroom plus KV for one max_model_len request. A config that only barely fit under hard can now fail at startup. The error names the option. Raise gpu_memory_utilization or lower max_model_len.

[roberteg16]

Don't forget to update the model settings configuration! LGTM

[eble-amd]

Thanks. I believe that this leaves less work for me when I reach the point of merging your upstream contribution into gfx11.