[JadenMathias]

Enables the correct preshuffling for A4W16 moe on gfx1250

gpt-oss-120b on gfx1250 — vLLM + aiter fix validation

Tested 2026-09-23/24 in a fresh container (jm-test) from rocm/vllm-dev:nightly_455_wip,
with the image's vLLM and aiter wheels uninstalled and rebuilt from the fix branches.

Environment

component	version
GPU	gfx1250 (AMD Eng Sample 100-000001046-04), single GPU, TP=1
host kernel / amdgpu	Linux 6.14.0-37-generic / 7.1.1.31300009
image	rocm/vllm-dev:nightly_455_wip (ubuntu:24.04, Python 3.12)
ROCm SDK	10.1.0a20260822 (rocm-sdk-core/devel/libraries/device-gfx1250, from https://rocm.nightlies.amd.com/whl-multi-arch/)
torch	2.11.0+rocm10.1.0a20260822 (HIP 7.16.26332)
triton	3.8.0+git78420176
vLLM	ROCm/vllm branch jm/oai-oss-path @ fd0f802a ("Add A4W16 routing for gfx1250"), python setup.py develop --no-deps
aiter	ROCm/aiter branch jm/1250-tdm-gemm-fix @ 78ee6783 ("Fix gfx1250 TDM grouped GEMM bias index missing the N-tile offset"), AITER_USE_SYSTEM_TRITON=1 GPU_ARCHS=gfx1250 python3 setup.py develop --no-deps
lm_eval	0.4.13

Note: the image has no Rust compiler, so setup.py develop silently skips vLLM's Rust
extensions (vllm._rust_tool_parser). Only --tool-call-parser needs them; MiniMax was
therefore run with the smoke test recipe minus --tool-call-parser minimax_m3 --enable-auto-tool-choice (gsm8k makes no tool calls). Everything else is unaffected.

openai/gpt-oss-120b (/data/models/gpt-oss-120b, MXFP4)

Server:

export HIP_VISIBLE_DEVICES=0 HSA_ENABLE_SDMA=0 USE_SVM=0 HSA_XNACK=0 HSA_COREDUMP_PATTERN=/dev/null
export VLLM_ROCM_USE_AITER=1 VLLM_ROCM_USE_SKINNY_GEMM=0 VLLM_ROCM_USE_AITER_RMSNORM=0
export VLLM_ROCM_AITER_FUSED_MOE_TRITON_GEMM_A4W4=1
vllm serve /data/models/gpt-oss-120b --host localhost --port 8000 --tensor-parallel-size 1 \
    --gpu_memory_utilization 0.7 --attention-backend TRITON_ATTN

Selected path: AITER_MXFP4_BF16 MoE backend, aiter gfx1250 grouped TDM kernel with MXFP8
activations (AITER_FORCE_A8W4=1 is set by the vLLM branch; the kernel has no bf16-activation
variant, so this is effectively W4A8).

Eval:

lm_eval --model local-completions \
  --model_args "base_url=http://localhost:8000/v1/completions,pretrained=/data/models/gpt-oss-120b,tensor_parallel_size=1,add_bos_token=true,trust_remote_code=true,max_gen_toks=2048" \
  --apply_chat_template --tasks gsm8k --num_fewshot 5 --batch_size 64

gsm8k 5-shot (1319 q)	before fixes	after
flexible-extract	0.0008	0.8961 ± 0.0084
strict-match	0.0000	0.2100 ± 0.0112 (harmony output rarely emits #### N)

Other models — /app/vllm_smoketest.sh (gsm8k 5-shot, limit 100, recipes verbatim from the image)

model	flexible-extract	strict-match
gpt-oss-120b-w-mxfp4-a-fp8	0.93	0.22
DeepSeek-R1-0528-MXFP4	0.80	0.88
DeepSeek-V4-Flash	0.96	0.96
MiniMax-M3-MXFP4 (without --tool-call-parser, see note)	0.94	0.94

oai-gpt-oss is good only after this is merged into aiter ROCm/aiter#5801