[keylimesoda]

Problem and result

With a CPU picture reader (vision.gpu: false), vision.elastic: true is currently disabled unless the text engine also enables vram_elastic. As a result, starting the text model starts the CPU encoder too, even when every request is text. Enabling elastic VRAM merely to defer a CPU process is unnecessary and changes the text engine's memory policy.

This allows the existing opt-in on-demand lifecycle for a CPU encoder without elastic VRAM: text leaves it unloaded, the first picture starts it, and idle unload stops its process. Set "elastic": true in an existing CPU "vision" section; "elastic_idle_s" keeps its existing default of 120 seconds.

Related discussion

Searched open and closed issues and PRs for CPU vision, elastic encoders and on-demand loading; found no duplicate CPU-specific fix. Related GPU lifecycle work: #2196. The tool-returned-image path in #3174 is separate and is not changed here. No changes to defaults or engine architecture; this is a focused extension of the existing opt-in lifecycle.

Approach

Allow CPU elastic startup independently of vram_elastic, and skip the VRAM reserve and free-VRAM probes when starting that CPU reader. Reuse the existing FIFO, idle timer and unload/restart paths. GPU readers still require vram_elastic and retain their existing reserve behavior. Without vision.elastic: true, eager and lazy startup remain unchanged.

Regression checks cover text-first loading, first/repeated pictures, idle unload and reload, no CPU VRAM calls, eager startup, and GPU eligibility/reserve behavior. The docs explain the CPU settings and load-time tradeoff. No personal configuration, model files, engine code or dependencies are included.

Validation

Tested revision: 875e21358194909fbbfcbde10c6f8f11f7a2840f, rebased onto current main at 9de04cd4 (v0.2.1.4). The results below are from the rebased revision.
Environment: x86-64 Linux (Omarchy/Arch), Python 3.14.8, Ryzen 9 9950X, 60 GiB RAM, RTX 4090 24 GB. The hardware smoke used the released 0.1.45.3 picture reader in CPU mode, existing IQ3_S model shards and BF16 mmproj, 8 encoder threads, and synthetic 128×128 PNGs.

Passed:

• python -m unittest serve.test_vision_elastic serve.test_lifecycle serve.test_vram: 26 tests, OK.
• python -m unittest serve.test_server serve.test_runconfig: 353 tests, OK.
• git diff --check: clean.
• python /tmp/strata-cpu-vision-http-smoke.py, with STRATA_TEST_CONFIG pointing to the existing CPU-vision config: a local HTTP harness using the real Vision reader and the existing VramEngine text-engine fixture. A text request left Vision.proc unset; a red-square image started the real CPU reader and encoded; advancing last_image_at beyond the configured 120-second threshold and invoking the idle check stopped that process; another text request left it unloaded; a distinct blue-square image started a new process and encoded. Both engine.vram() and Service.free_vram_mib() were patched to raise on any call. Neither was called. All test processes were shut down.

Observed smoke output (request/timing lines omitted):

PASS: first HTTP text request left CPU encoder unloaded
PASS: first HTTP image started and encoded on CPU {'first_image_start_and_encode_s': 0.67, ...}
[strata] images: the CPU image encoder stopped after 120 s without a picture
[strata] images: the image encoder started in 0.5 s
"first_text_keeps_vision_unloaded": true
"idle_stops_process": true
"vram_resize_and_telemetry_calls": 0
"encoder_stopped_at_end": true

The unittest runs emitted ResourceWarnings for telemetry fixture files, HTTP errors and sockets; both runs completed with OK.

Failed: none observed.

Not run: full text-model image-answer generation or quality checks; GPU vision hardware execution; CUDA/HIP/SYCL/Metal builds (no engine or compiled-backend sources changed). The real smoke verifies CPU encoding and lifecycle with a fake text engine, not backend hardware compatibility or answer quality. No performance claim is made.

Limits

The first picture after unload pays the encoder's load time. With the existing idle polling, a 120-second timeout normally unloads after about 120–150 seconds, later if a request owns the FIFO. Unloading ends the encoder process; existing embedding caches and reclaimable OS file cache may remain. This does not change tool-returned-image handling or fix the separate issue in #3174.

[keylimesoda]

For VRAM-limited systems, it makes sense to keep the vision decoder on CPU.

If the system is also RAM-constrained, it makes further sense to keep the vision model on disk until needed.

[keylimesoda]

Rebased onto main at 9de04cd4 (v0.2.1.4) and force-pushed as 875e2135.

Re-ran on the rebased revision: 26 focused tests (test_vision_elastic, test_lifecycle, test_vram), 353 server/config tests (test_server, test_runconfig), the commit's git diff --check, and the CPU HTTP smoke with the real reader — all OK. The smoke started the encoder in 0.67 s, reloaded in 0.58 s, and made zero VRAM calls.

The change is unchanged in scope: CPU vision.elastic no longer requires vram_elastic; GPU readers keep the existing requirement and reserve behavior. No engine or backend sources touched.