[jerry78424]

Problem and result

An elastic image encoder is woken (seconds of start, VRAM the expert cache gives up) on every turn that re-sends pictures it has already encoded - which is what most clients do on every turn. A turn whose images are all in the hash cache needs no encoder process at all.

Measured on RTX 5090 32 GB, Windows 11 (26200), engine 0.1.44.1, nvfp4, elastic_idle_s: 10 (table and verbatim log lines in #2454): before - a text-only turn with a re-sent picture cost a 2.3 s start + 2085 MiB taken from the expert cache; after - 722 ms with no start line. A re-sent-picture turn: 1277 ms, no start. New pictures unchanged (3979-4213 ms, still start).

Related discussion

Resolves #2454. Independent of #2641 (the tool-picture wake lands in this same method - textual overlap only; the two PRs conflict in neither order, verified by cherry-pick).

Approach

• serve/server.py: Vision.key_of() / Vision.cached() - a hash-cache lookup that answers with the process down (the cache and its .sve files survive an unload, as before) and marks a hit most recently used, as encode() does on its hit path. prepare() maps the turn's images through the cache first, for an elastic encoder only: when every one is a hit the turn skips the encode FIFO, _vision_up() and the encode. A miss - alone or alongside hits - keeps today's path byte-for-byte: FIFO, make-room, start, encode.
• serve/test_vision_cache_wake.py (new, CPU-only, no engine): a re-sent picture does not wake a stopped encoder; a new picture alongside cached ones still wakes it; the lookup answers with the process down and keeps the re-sent picture from aging out of the 64-entry cache; a non-elastic turn keeps today's path even when every picture is cached.
• docs/VRAM_ELASTIC.md, docs/DETAILS.md: both halves of the elastic rule - a picture's turn starts the encoder, a turn of pictures the cache knows takes neither the turn nor a start.

Validation

Passed:

• Every new behavior fails on a tree without the code that provides it.
• python -m unittest serve.test_vision_elastic serve.test_vision_busy serve.test_vision_cache_wake serve.test_images: the vision four 36 OK - at e88ee39d (v0.2.0-beta-12), rebased 2026-10-20, Windows 11 26200, Python 3.12 venv, CPU-only tests.
• Full serve suite: 791 OK (15 skipped).
• With [Other] serve: a tool's picture starts the elastic encoder, and its turn counts #2641 cherry-picked on top: the files stay 19 + 19 OK.

Failed: none observed.

Not run: AMD/HIP and layer-split machines (a layer split refuses elastic anyway; its start failure is #2435). No native files are touched, so no native builds are required.

Limits

• What it leaves alone: non-elastic vision, the start/stop/idle lifecycle itself, the encode path, and every native file (no CUDA/HIP/SYCL build needed).
• The measured numbers are one machine (RTX 5090, nvfp4); the mechanism - no process start for an all-cached turn - is hardware-independent, but the saved seconds are not claimed for other machines.

[qiuzan2001]

Two small points from reading the branch (tests serve.test_vision_cache_wake, test_vision_elastic, test_vision_busy, test_images pass here):

1. Vision.cached() (serve/server.py:2409-2417) reads self.cache.get(key) without the move-to-end that encode() does on a hit (self.cache[key] = self.cache.pop(key), line 2437). Pictures that a client re-sends every turn now take the all-cached path and are never marked as recently used, so once newer images fill the 64-entry cache they are the first evicted, and the next turn that re-sends them has to encode them again and pays the elastic start. Refreshing the order inside cached() keeps the old LRU behaviour.
2. The description says the branch "is only entered when vision.elastic", but the new if all(c is not None for c in cached) in prepare() has no such check: a non-elastic encoder also skips _turn() and the last_image_at update on an all-hit turn. Probably harmless, but it is a change outside the elastic case.

[jerry78424]

Both points are right - fixed in bf73c58.

1. Agreed, and it regressed the PR's own hot path: the re-sent pictures this lookup answers for were the oldest entries in the dict, so encode()'s oldest-first eviction took exactly them once the cache filled past 64, and the turn after paid the encode and the elastic start. cached() now moves the hit to the end inside the same lock, as encode()'s hit path does. New test: with three stored pictures the lookup flips the eviction order to the re-sent one's benefit.

2. Agreed, the sentence did not match the code. The lookup is now computed only for an elastic encoder - a non-elastic turn takes today's path byte-for-byte (FIFO, encode, cache hits inside encode()) and does not pay the key_of() hashing either. New test: an elastic=False encoder sees encode_all on every turn, hits included, with no make-room request.

Docs (VRAM_ELASTIC/DETAILS) carried the wake rule only one way; they now say both halves (bdbb0e6).

Suite on the branch: serve 791 OK (15 skipped), the vision four 36 OK.

[Niko1221]

Thanks for the pull request.

We're resetting the tracker for 0.2.0-beta. There are more than 1,300 open issues and pull requests, too many for us to review properly, so we're closing everything that isn't being worked on right now. Closed here does not mean rejected.

To reopen it takes about two minutes:

1. Rebase on main if it no longer applies, and re-run your checks on 0.2.0-beta.
2. Start the title with its area: [Engine], [GUI], [TUI], [Benchmark], [Docs] or [Other].
3. Fill in the pull request template: the problem and result, and the validation (what you ran, on which revision, what you saw).
4. Comment /reopen. A check runs and either reopens the pull request or lists what is still missing.

The rules are in CONTRIBUTING.md.

[jerry78424]

/reopen

[github-actions]

The check passed, but GitHub would not reopen it (Validation Failed: ). If the branch was deleted, restore it or open a new pull request and link this one.

[jerry78424]

Reopened as #3380 (GitHub could not reopen this one after the rebase force-push; same branch, checks re-run on e88ee39).