[Engine] run Mirai's Qwen3.8-Flash-Next-S (QNS trellis) packs

[alesha-pro]

Problem and result

Mirai published Qwen3.8-Flash-Next-S, a trellis build of Flash-Next (routed experts 1-5 bits each, 2.4 on average; backbone and head 5 bits), packed for Strata: trymirai/Qwen3.8-Flash-Next-S-strata-experimental. Until now it runs only on v0.1.43 with Mirai's strata-qns.patch. This PR ports that patch to 0.2.0: the S and S-small packs load and run on one card and on a layer split, with MTP and the refusal projection. Every other model takes the same path as before, checked on IQ3_S below.

Related discussion

Scope: I am writing to @Niko1221 directly to agree it. The QNS format, the CUDA and AVX2 kernels and the packer support are Mirai's work, from their strata-qns.patch.

Approach

Mirai's code is taken unchanged from their HF repo (2026-10-15, which already has the layer-split rotation fix). What the merge with 0.1.44-0.2.0 needed:

• Pack version. 0.2.0 uses native_experts.txt v5 for Hadamard-folded experts, and the patch used v5 for QNS layers. QNS packs are v6 here. v5 keeps its meaning (hadamard.txt is required for v5 only). A v5 QNS pack from the old patch is refused with "repack with tools/iq_pack.py"; repacking takes ~20 s and gives the same files except the header line. The packer skips the Hadamard spec for QNS.
• MTP head on another card (0.1.44 moved the drafter to the first stage): the trellis head's subset rows and their row scales are gathered on the head's card and copied over, like the non-QNS path.
• New side-stream paths that read dense weights directly now take the trellis GEMV for kQnsDenseType: the GDN gate projection (side_proj) and the QSA query chain on a branch stream.
• Expert cache sizing uses slot_stride(expert_bytes(l, e), slot_guard); expert_bytes is the layer's blob for non-QNS packs.
• The CPU pool keeps 0.2.0's per-layer widths and rounds QNS row ranges to whole 8-row blocks.
• docs/QNS.md: how to pack and run.

All new code is behind expert_layout().qns() or kQnsDenseType.

Validation

Ubuntu 22.04, CUDA 12.8, 4x RTX 3090 at 300 W, EPYC 7642 (AVX2), 125 GB RAM. Branch head 8e75f6f (engine sources identical to the tested build).

Passed

• CUDA build: cmake -DSTRATA_ENABLE_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES=86 -DSTRATA_BUILD_TESTS=OFF, --target strata. No new warnings in the touched files.
• tools/iq_pack.py on S and S-small: v6 packs; qns_offsets.bin, qns_rotation.bin, dense.bin, index.txt and every layer row byte-identical to the v0.1.43 patch's packs.
• Six greedy probes through /v1/chat/completions, thinking off, 32K context, MTP on, the refusal vector on: a sum, a Python function, a Russian answer, a tool call, a 6,210-token document with a hidden code, a lock-picking question.
  • S-small, 1 card and 2 cards (layer split): all answers correct; the code is found; the tool call is right; no refusal. Against v0.1.43 + Mirai's patch on the same cards: 5 of 6 answers identical, the sixth parts on a near-tie after 10-55 words (v0.1.43 itself gives a different text on 1 and 2 cards there, from about 120 words on).
  • S (full), 1 card and 2 cards: all six correct.
  • On 2 cards MTP drafts are accepted (117 of 170 on a 200-token answer), no errors in the engine log.
• Default path: IQ3_S (ISTA GSQ-RCO) on 2 cards, the same flags, stock 0.2.0-beta (3d1fb48) against this branch: all six greedy answers byte-identical, same token counts; decode per request within 3.1%.
• Decode, server tok/s per request (one run each, indicative): S-small 47-106 (1 card) / 83-166 (2 cards); S 35-81 / 60-125. v0.1.43 + patch on the same S-small requests: 45-92 / 75-143.

Failed: none observed.

Not run

• HIP build: no ROCm toolchain here. The new .cu files are compiled for HIP by the existing glob, so this needs a HIP build (or a CUDA-only guard) before review. Help welcome.
• SYCL and Metal: not touched. Windows / MSVC: not built.
• Long contexts (above 32K) and vision on QNS: not run in this PR (vision ran on v0.1.43 with the patch).

Limits

• NVIDIA CUDA and the AVX2 CPU pool only, as in Mirai's patch.
• Setup and the Terminal do not offer these models; they start from a hand-written config (docs/QNS.md).
• Kept as a draft until the HIP build is checked.