Repository navigation
Conversation
… image
_note_unreadable_tool_images() encodes a picture a tool returned (Claude Code's
Read of an image file, an MCP tool returning a plot) with self.vision.encode(),
but unlike prepare()'s own image path it never calls self._vision_up(). With the
elastic encoder (vram_elastic + vision "elastic") the encoder is not running
between pictures, so Vision.proc is None and encode()'s `self.proc.stdin.write`
raises AttributeError. The request then fails:
400 {"type": "invalid_request_error",
"message": "the request is malformed (AttributeError: 'NoneType' object
has no attribute 'stdin')"}
That function's own except (ValueError, OSError) does not catch it, so the
exception escapes prepare() and is reported as a client error.
Reproduced on 0.1.44.1 with a fresh elastic session (the encoder never started,
so no waiting for elastic_idle_s is needed): one /v1/messages request with a
tool_result image gives the 400 above; the same request succeeds with
"elastic": false. Fixes Niko1221#3171.
_vision_up() is a no-op when the encoder is not elastic or is already running, so
non-elastic setups are unaffected.
|
Confirmed on real hardware, and still broken on current
Two notes for whoever reviews, both minor:
+1 from a Windows/5090 box where this was the single most-hit bug of the week — glad to retest any revision. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3171.
The bug
_note_unreadable_tool_images()encodes a picture that a tool returned (Claude Code'sReadof an image file, an MCP tool returning a plot) withself.vision.encode(src)— but unlikeprepare()'s own image path it never callsself._vision_up():With the elastic encoder (
vram_elastic: true+ vision"elastic": true) the encoder is not running between pictures, soVision.procisNoneandencode()'sself.proc.stdin.write(...)raisesAttributeError. That function'sexcept (ValueError, OSError)does not catch it, so it leavesprepare()and lands in_post's generic handler, which reports any of those as a client error:So an agent doing a normal tool call breaks the turn — and, because clients re-send history, possibly the turns after it as well.
Repro (0.1.44.1, no waiting needed)
A fresh elastic session never starts the encoder (it starts with the first picture), so the first request with a tool image fails:
"vram_elastic": trueand"vision": {"gpu": true, "elastic": true, ...}; start the server —strata-visionis not running (expected).user→assistant tool_use→user tool_resultwith animageblock).VRAM 3584 MiB kept free: the expert cache N -> M of N slotsin the engine log)."elastic": falsedoes not hit it (the encoder runs from the start), and a user-sent image was always fine — only the tool-returned path is affected.The change
Two lines, the same call the sibling path makes.
_vision_up()returns immediately when the encoder is not elastic or is already running (if not getattr(v, "elastic", False) or v.alive(): return), so nothing changes for non-elastic setups.\ No newline at end of file— no; the file's LF endings are unchanged (2 lines added, 0 removed).Two things I did not change (see #3171)
[image omitted: ...]instead of a 400; widening itsexcepttoAttributeErrorwould keep that promise for unexpected failures too. Kept out of this PR to keep it to the defect._postmaps anyAttributeErrorraised while preparing a request to400 invalid_request_error. That is why this looked like a client bug; an internal error arguably deserves a 500.Verified
--mmap-experts,--vram-elastic,elastic_mib3584)./v1/chat/completions), a user image via/v1/chat/completionsand via/v1/messages, and the tool image that used to fail.