[Other] FP4-Local prepare: the pack step's memmap sweep charges its full 64 GB of reads to the process working set on Windows #3377

Area

[Other] (tools/nvfp4_local_pack.py, the FP4-Local prepare step)

Desktop app, Terminal or source?

Source install (git clone, SETUP.bat) on Windows

What happened

nvfp4_local_pack.py prepare fails mid-pack with no traceback:

  layer 24  NVFP4   /NVFP4   blob  2764816  at 31.64 GiB
prepare: D:\...\app\tools\nvfp4_local_pack.py failed (exit 1)

The pack python dies between its layer-24 and layer-32 prints (50–67 % of the 68 GB experts write). On my 187 GB machine with the serving model loaded (135 GB baseline), system "used RAM" climbs +41 GB in five minutes while the process's own committed memory is ~3.5 GB — and the run dies at ~11 GB free. Unload the model first and the same prepare completes in ~10 minutes, exit 0. The step's real RAM demand is a few GB; it presents as ~64 GB to every free-RAM observer (Task Manager, psutil, os.freemem), so prepare cannot run next to a loaded model on any machine without ~60 GB to spare.

Timeline (manager diag sampler, ram_used = total - available, ~10 s cadence; pack started 12:38:43):

12:37  135 GB  baseline        12:41  158 GB  rd 250 MB/s
12:39  143 GB  wr 892 MB/s     12:42  166 GB  wr 963 MB/s
12:40  150 GB                  12:43  176 GB  -> run killed by watchdog (~11 GB free left)
12:44  135 GB  -> all 41 GB back within a minute: nothing leaked

(The watchdog that ended the run is a personal tool of mine, unrelated to this repo — beside the point: every observer of free RAM saw the climb as real usage.)

The cause

Checkpoint.raw() (tools/nvfp4_local_pack.py:86-88) maps each shard with np.memmap(..., mode="r") and caches the mapping for the process's life (self._mm, never closed). The pack pass reads 64.36 GB of distinct expert bytes (header parse: 48 layers x 3 roles x 512 experts x weight + CSF pair + scale_2 + input_scale; expert->shard assignment rotates per layer, so every layer touches 13–14 of the 15 expert shards). On Windows every faulted page is charged to the process working set, and ullAvailPhys (= psutil available = os.freemem = our own ram_used, serve/telemetry.py:624-625) counts it as used until the OS trims under pressure. The sweep inflates "used RAM" ~1:1 with the read rate: when the run died, 50–67 % of 64.36 GB = 32–43 GB faulted — bracketing the +41 GB (private commits ~3.5 GB). Reclaimable, but "reclaimable" is not "available".

A/B on the live checkpoint, the pack's exact read order (layers 0–2 x gate/up/down x experts 0–511 = 4.03 GB): np.memmap grows the process working set +3.75 GB (93 % of bytes touched; charges even from a warm cache) in 2.7 s; cached sequential readinto of the same bytes grows it +0.00 GB in 2.3 s — both hash-identical. PR #3378 does this: same bytes, ~1.2x faster, flat working set.

Workaround that works now

Run the FP4-Local setup with the serving model unloaded (or ~60 GB free: observed 41 GB climb + margin).

Reproduction status

Reproduced on this PC: the failed run with the diag sampler timeline (2026-10-20, source install - nvfp4_local_pack.py there is byte-identical to the revision below; no newer upstream commit touches it), the A/B working-set measurements above run at the revision below, and the fix verified end to end (full pack, byte-identical output, flat working set).

Strata version

e88ee39 (v0.2.0-beta-12)

Graphics card, memory and system

Windows 11 26200, Ryzen 7 9800X3D, 187 GB RAM, RTX 5090 (driver 617.42), NVMe, model nvidia/Qwen3.8-Flash-Next-FP4-Local

Not verified / limits

• The death point (50–67 % written) is bracketed between two progress prints, not measured.
• Working-set inflation from mapped clean pages is Windows accounting behavior (GlobalMemoryStatusEx semantics); the timeline arithmetic is consistent with it but was not decomposed with RAMMap-level counters. The fix covers the checkpoint reader only; pack still memmaps dense.gguf (~6 GB file, ~1.5–2.5 GB faulted, iq_pack.py:161,218,324) and convert ~4 GB via LocalTensor.mmap_bytes (gguf/utility.py:286) — bounded residuals.