[afriedri]

Clean up gfx1250 smoke test and bench serve params

Summary

Prunes stale env vars and serve args from vllm_smoketest.sh and vllm_benchserve.sh. Turns on AITER RMSNorm and torch.compile where accuracy holds. Makes the two scripts use identical model recipes and fixes two teardown/timeout bugs found while testing.

Removed

Env vars (all models):

• HSA_ENABLE_SDMA=0 USE_SVM=0 HSA_XNACK=0: no longer needed.

Env vars (no-ops or defaults):

• VLLM_ROCM_USE_AITER_UNIFIED_ATTENTION=1: dsr1 only has TRITON_MLA, and minimax is forced to ROCM_ATTN, so it never takes effect.
• VLLM_FORCE_TORCH_BLOCK_FP8=1: nothing in vLLM reads it.
• VLLM_ROCM_USE_AITER_TRITON_GEMM=1 and VLLM_ROCM_USE_AITER_FP8BMM=0: these are already the defaults.
• VLLM_DISABLE_COMPILE_CACHE=1 (dsr1, minimax) and VLLM_ROCM_USE_AITER_MLA=0 (minimax): no effect on these models.

VLLM_ROCM_USE_AITER_RMSNORM=0:

• Removed from gptoss, dsr1, dsv4f, dsv41f, minimax and oaigptoss, so AITER RMSNorm is now on.
• Verified in the logs that the aiter kernel is selected and gsm8k accuracy holds (see below).
• dsv4pro keeps it because it hasn't been tested.

Env vars (removed one at a time; gsm8k and bench serve hold):

• VLLM_ROCM_USE_SKINNY_GEMM=0: dsr1, minimax, dsv41f.
• TRITON_HIP_USE_ASYNC_COPY=0: dsr1.
• VLLM_USE_BREAKABLE_CUDAGRAPH=1: dsv41f.

Env vars (removed together; gsm8k holds, bench serve is faster):

• oaigptoss: TRITON_HIP_USE_ASYNC_COPY=0 and VLLM_ROCM_USE_SKINNY_GEMM=0. Bench throughput goes from 4855 / 4807 to 4936 / 4911 tok/s (+2%), and gsm8k is 0.90.
• dsv4f: VLLM_ROCM_USE_AITER_LINEAR=0 and VLLM_ROCM_USE_SKINNY_GEMM=0. Bench throughput goes from 872 / 914 to 1300 / 1437 tok/s (+53%), median TPOT drops from 66 to 43 ms, and gsm8k is 0.95.

Serve args:

• --dtype auto and --enable-prefix-caching: both are vLLM defaults.
• dsr1 "cudagraph_mode":"FULL_AND_PIECEWISE": vLLM overrides it to FULL_DECODE_ONLY anyway.
• bench serve dsr1 "mode":0: this disabled torch.compile. Compile is now on in both scripts.
• --trust-remote-code (dsr1, minimax): both load with built-in vLLM model classes. dsr1 was tested with it removed (gsm8k 0.95).
• minimax --max-num-batched-tokens 32768: with chunked prefill off, vLLM already sets it to max_model_len (32768).
• qwen --no-enable-flashinfer-autotune: the autotune only runs when flashinfer is installed and on CUDA, so this does nothing on ROCm.

Changed

• --tensor-parallel-size N becomes -tp N. It's a default for tp=1 but kept as a visual check.
• dsv4pro path: /data/model/DeepSeek-V4-Pro becomes /data/models/DeepSeek-V4-Pro.
• The headers now list all 9 models and describe the current behaviour (bench: concurrency 64 by default, full sweep with --long).
• The MODEL_PATHS, MODEL_ENV and MODEL_SERVE tables are now identical between the two scripts.

Fixed

• stop_server: in images without rocm-smi, it only slept 8 s, so the next model could load while the previous model's VLLM:: workers still held GPU memory. dsv41f ran out of memory this way. It now waits for those processes to exit first.
• /health wait: raised from ~20 to ~40 minutes. dsv41f takes more than 20 minutes to load.

Kept

• dsr1 VLLM_ROCM_USE_AITER_MLA=0: without it, the server crashes on A0 silicon (mla_decode_stage1_asm_fwd asm is only supported on gfx1250 B0+).
• dsv41f VLLM_ROCM_USE_AITER_LINEAR=0: without it, host memory pressure hits 94% during load.
• gptoss VLLM_ROCM_USE_SKINNY_GEMM=0: accuracy is the same without it, but bench throughput was mixed: 3176 / 4223 / 3530 tok/s without it vs 4287 / 5817 with it. Time per token was the same either way (~9.5 ms), but runs without it had more generation stalls, so it stays.
• Settings with real effects: VLLM_ENGINE_READY_TIMEOUT_S, --gpu-memory-utilization, dsr1 --max-num-batched-tokens 8192, --max-num-seqs, minimax's multimodal flags, and --distributed-executor-backend mp.
• dsv4pro: its env is unchanged. It needs 4 GPUs, so it wasn't tested.

Test results

• Image: rocm/vllm-dev:nightly_455_wip__torch__1021_b353
• Hardware: 1x gfx1250 (tp=1)
• Eval: lm_eval gsm8k, 5-shot, limit 100, flexible-extract

Model	Before	After (RMSNorm on)
gptoss	0.92	0.91
dsv4f	0.93	0.93
dsr1	0.94	0.95
minimax	0.92	0.91
dsv41f	FAIL (OOM / load timeout)	0.90
qwen	0.94	0.88
oaigptoss	0.917 (RMSNorm off, mean of 3)	0.900 (mean of 3)

Generation samples, so run-to-run noise is about ±0.03–0.08.

• qwen: it never had RMSNORM=0, and its resolved engine config and compile-cache hash are identical before and after, so nothing in this PR changes its behaviour. The original 0.94 was a single run; the three later runs scored 0.87–0.88.
• oaigptoss: RMSNorm really changes here, so it was run 3 times each way. Off: 0.90 / 0.92 / 0.93. On: 0.87 / 0.91 / 0.92. The gap between the means (0.017) is well under one standard error.

Final run of this PR's scripts (one back-to-back pass, gsm8k):

Model	gsm8k
gptoss	0.93
dsr1	0.95
minimax	0.90
dsv41f	0.92
qwen	0.91
oaigptoss	0.90
dsv4f	0.95

Bench serve (random 1k/1k, concurrency 64, no failed requests):

Model	Output tok/s (before the latest removals)	Output tok/s (final scripts)	Median TTFT	Median TPOT
gptoss	4256	4287 / 5817 (two runs)	0.80 s	9.4 ms
qwen	1505	(not rerun; only change is a no-op flag)	2.45 s	39.1 ms
dsv41f	1385	1393	1.47 s	42.1 ms
minimax	948	969	37.0 s	29.8 ms
dsr1	509	505	63.7 s	59.9 ms
oaigptoss	4855 / 4807	4936 / 4911	0.60 s	10.1 ms
dsv4f	872 / 914	1300 / 1437	2.2 s	42.8 ms

gptoss throughput moves around a lot between runs on this shared box (3.2k–5.8k tok/s) because of occasional generation stalls. Median TPOT stays at about 9.5 ms in every run.

dsr1 and minimax have high TTFT because they use --max-num-seqs 32, which is below concurrency 64.

Not tested

• dsv4pro: tp=4, and the model isn't on the test box.
• llama: the model isn't on the test box.
• The stop_server wait fix has since been exercised in several back-to-back multi-model runs (smoke test followed by bench serve), with clean server handoffs every time.

🤖 Generated with Claude Code

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀