[silencelamb]

Motivation

POST /v1/responses rejects user input_image content with HTTP 400 (Unsupported Responses input content part type: 'input_image'.) before model preprocessing, even when the same image works through /v1/chat/completions on the same vision model.

Fix this request-conversion gap by forwarding user images through the existing multimodal execution path.

Modification

• Convert user input_image blocks into Chat-compatible image_url parts, preserving interleaved text/image order, multiple images, and conversation turns.
• Accept HTTP(S) URLs and image data URLs, and forward detail and the existing dynamic-patch image parameters. Preserve raw JSON input dictionaries so SDK TypedDict validation does not discard image extensions; SDK model objects remain supported.
• Preserve text-only concatenation and existing function-call history behavior. Return validation errors for unsupported image sources, invalid image fields, and unsupported content types.
• Add request-conversion, real-image-decoding, and HTTP/SSE regression tests, focused on the Responses endpoint. Update the English and Chinese Responses API documentation with image examples and limitations.

This change covers images in user messages. Image file IDs, images in other roles, and image-bearing function_call_output remain unsupported. Tool-image ownership/order is handled separately in #5044. Router forwarding changes are outside this PR. Image parameter effects remain model-dependent.

BC-breaking (Optional)

No intended breaking changes. Existing supported text and tool-call inputs retain their behavior; valid user image inputs that previously returned HTTP 400 are now accepted for vision-capable models.

Use cases (Optional)

Send a user message containing input_text and input_image to /v1/responses, using either a remote image URL or an image data URL. Both non-streaming and streaming requests use the existing vision-model execution path. See the updated English and Chinese API documentation for complete examples.

Validation

• Complete serving test suite at 6a5a32a4: 486 passed (python -m pytest tests/test_lmdeploy/serve -q).
• Responses-specific tests cover conversion, field validation, SDK objects, image parameters, PNG/JPEG/WEBP decoding, and eight HTTP/SSE cases (multiple images, image-only input, multi-turn history, and remote image fetching). Generation and remote HTTP transport are simulated; image decoding uses production code.
• Live Qwen3-VL-2B service running implementation commit 50754852 (subsequent changes affect tests only): 17/17 HTTP checks passed, covering PNG/JPEG/WEBP, image-only input, image extensions, two images, multiple turns, remote image fetching, SSE completion, and expected 400 responses for corrupt image data, file IDs, and unknown content types.
• Actual GLM-5.3-Flash tokenizer, chat template, and CPU vision preprocessing: 12/12 checks passed (Chat/Messages/Responses × single image, image-only input, multiple images, and multi-turn history). Equivalent requests produce identical complete input IDs, multimodal tensors, and offsets across the three protocols.
• Real GLM-5.3-Flash GPU inference at 168158a4: 42/42 ordinary-image HTTP checks passed across Chat, Messages, and Responses, both non-streaming and SSE. Coverage includes PNG/JPEG/WEBP data URLs, image-only input, multiple images, multi-turn image history, and an actual remotely fetched image. Checks compare the final recognition answer and verify successful response/stream completion. The model ran locally on 8 H200 GPUs using the existing Python environment and its actual vision path.
• 14/14 image-related rejection checks passed: corrupt base64 for all three protocols, unsupported Responses file IDs, unsupported Responses content-only blocks, and mixed image/unknown-content blocks in Chat and Responses, each in both stream modes.
• A broader validation probe exposed a separate existing Chat input-validation gap: an unknown-only content array with no image is silently converted into an empty message and returns HTTP 200 (2 failing checks: non-streaming/SSE). Mixed image/unknown-content arrays correctly return 400. This is tracked separately; the entire exploratory suite is not claimed to pass.
• The PR1 GPU service has been stopped. Router integration validation remains pending.
• All applicable pre-commit hooks and git diff --check passed.

Test scope

tests/test_lmdeploy/serve/openai/responses/test_images.py contains the focused image regressions for this fix and reuses the existing Responses endpoint fixture. The shared 24-case Chat/Messages/Responses suite is submitted separately in #5046. Tool-result image handling and its three-endpoint tests remain in #5044.

The GPU results above were collected at the explicitly recorded earlier commits. The current test-only revision leaves all production code and documentation identical to 168158a4; the serving tests and applicable pre-commit hooks were rerun after the split. No GPU service was started for this revision.

Checklist

• Pre-commit checks passed.
• Regression unit tests added and the Responses suite passed.
• No new dependency or downstream minimum-version requirements introduced.
• English and Chinese documentation updated.

[caikun-pjlab]

LGTM

[CUHKSZzxy]

Overall LGTM, one small nit about UT.
This PR only modify /v1/respponses related, how about removing tests/test_lmdeploy/serve/test_user_images.py, or putting simplified logic in tests/test_lmdeploy/serve/openai/responses/test_images.py, which makes the PR focused and clean.

[silencelamb]

Thanks @CUHKSZzxy, agreed. Addressed in 6a5a32a: removed tests/test_lmdeploy/serve/test_user_images.py from this PR and kept the focused image regressions in tests/test_lmdeploy/serve/openai/responses/test_images.py, reusing the existing Responses endpoint fixture. The HTTP/SSE checks cover multiple images, image-only input, multi-turn history, and remote image fetching.

The broader three-endpoint regression suite is now submitted separately in #5046, so it can be reviewed independently. Production code is unchanged by this revision. The complete serving suite passes (486 tests), and the applicable pre-commit hooks pass.