MXFP4 is documented as requiring compute capability >= 9.0 — Hopper (H100) onwards, which also takes in Blackwell. The V100 is 7.0: Volta, four architecture generations below that floor (Turing, Ampere, Ada, Hopper). It runs MXFP4 at 143 t/s with correct output.
Not yet reported upstream. See Status.
llama-bench, upstream llama.cpp at c550d2f60 (b11096), CUDA 12.9.86,
CUDA_VISIBLE_DEVICES set to the V100's UUID so nothing else is visible.
| test | t/s | flags |
|---|---|---|
| pp512 | 2460.97 ± 47.86 | -ngl 99 -fa 1 -r 5 |
| tg128 | 144.34 ± 0.32 | -ngl 99 -fa 1 -r 5 |
| pp2048 | 3940.38 ± 12.47 | -ngl 99 -fa 1 -b 4096 -ub 2048 -r 5 |
| tg128 | 143.24 ± 0.46 | -ngl 99 -fa 1 -b 4096 -ub 2048 -r 5 |
Decode came out at 144.34 with default batching and 143.24 with large batching — two independent runs agreeing within 1.1 t/s, which is the expected insensitivity of batch-1 decode to batch flags, and a check that both runs measured what they claim to.
Model loads at 11.27 GiB, 20.91 B params, all layers offloaded.
The 4090 row is published, not measured here, from the llama.cpp community
thread (discussion #15396).
Our V100 row was re-run under the same flags (-b 4096 -ub 2048 -p 2048) so
the comparison is like-for-like.
| pp2048 | tg128 | |
|---|---|---|
| Tesla V100-PCIE-32GB (measured) | 3940.4 ± 12.5 | 143.2 ± 0.5 |
| RTX 4090 (published) | 8022.3 ± 161.3 | 222.0 ± 6.3 |
The 4090 is 2.04x faster at prefill and 1.55x at decode. The V100 lands at 49% of its prefill and 65% of its decode.
That asymmetry is the same one seen with Bonsai (see the volta-bonsai repo):
Volta holds up on memory-bound decode, where HBM2 is still competitive, and
loses on compute-bound prefill, where the architecture gap shows.
Caveat that cuts against the V100: the published 4090 row is from build 6210; ours is 11096, roughly 4,900 builds later, and CUDA decode optimisation work has landed in between. A 4090 on current master is likely faster than 222, so 65% is an optimistic ceiling, not a floor. Closing that properly needs a 4090 re-run on current master.
11.27 GiB plus context does not fit in the 4070's 11.86 GB beside the desktop;
llama-bench fails to load it. The V100's 32 GB runs a model the newer,
faster card cannot — and leaves far more room for KV cache, which matters more
than peak speed for long-context work.
Unreported. The MXFP4 hardware-floor claim is about the format, and is independent of the llama.cpp sm_70 build issue recorded in upstream-sm70-build-record.md — that one was already fixed before we built it, and that file exists mainly as a record of a near-miss.
Before reporting anything here: read the whole thread first, and check whether the claim has already been addressed.
Same host as volta-bonsai — see that repo's ENVIRONMENT.md. Built from a
clean clone of ggml-org/llama.cpp, not the PrismML fork:
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON \
-DCMAKE_CUDA_COMPILER=/usr/local/cuda-12.9/bin/nvcc \
-DCMAKE_CUDA_ARCHITECTURES="70"
cmake --build build -j 4
Model: ggml-org/gpt-oss-20b-GGUF, gpt-oss-20b-MXFP4.gguf, 12.11 GB.