Skip to content

fix(serve): reject unsupported chat content blocks - #5047

Open
silencelamb wants to merge 1 commit into
InternLM:mainfrom
silencelamb:fix/chat-content-validation
Open

silencelamb wants to merge 1 commit into
InternLM:mainfrom
silencelamb:fix/chat-content-validation

Conversation

@silencelamb

@silencelamb silencelamb commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

A Chat request containing only an unknown content block, such as {"role": "user", "content": [{"type": "unsupported_probe"}]}, returns HTTP 200 in both streaming and non-streaming modes. Multimodal detection ignores the block, and text preprocessing then silently removes it.

Reject unsupported blocks with their message/content indexes before template rendering, media loading, or generation.

Modification

  • Validate content-array objects and block types while selecting text or multimodal preprocessing. Scan the complete history so an earlier image cannot hide a later invalid block.
  • Reject unknown/missing types and non-object blocks with a ValueError that the existing engine error handling maps to HTTP 400. Error messages identify locations such as messages[1].content[0].type.
  • Preserve supported text/media types and the existing missing/null-content normalization used by assistant tool-call messages.
  • Retain regression coverage for the original content-part location in Responses' unsupported-type error, already provided by merged #5043.
  • Add regression checks for all three APIs. Responses continues to reject unknown types; Messages continues to preserve ordinary unknown extension blocks as text under its permissive compatibility behavior.
  • Document Chat content validation in the English and Chinese API guides.

BC-breaking (Optional)

Malformed or unsupported Chat blocks that were previously silently discarded now return HTTP 400, including for stream=true. The shared processor also rejects these blocks for direct Python callers. Supported text/media inputs and Anthropic extension-block handling retain their behavior.

Use cases (Optional)

For an unknown block at the second position of the second message, the error message is:

Unsupported content block type 'unsupported_probe' at `messages[1].content[1].type`.

Streaming requests receive the JSON error before an SSE stream starts.

Validation

  • Reproduced HTTP 200 for the original unknown-only Chat input in both stream modes before the fix, using the production AsyncEngine.preprocess and MultimodalProcessor with simulated generation.
  • After rebasing onto merged [Fix] Handle user image inputs in Responses API #5043, at 66a519fc, 538 tests passed: python -m pytest tests/test_lmdeploy/serve tests/test_lmdeploy/test_content_merge.py -q (2026-10-21). Applicable pre-commit hooks also passed on this revision.
  • Chat HTTP regressions cover unknown-only, text/unknown, image/unknown, an image in an earlier turn, non-object blocks, and missing types in both stream modes. They check HTTP 400, precise error location, no generation, no media loading, and session cleanup. Valid text controls pass.
  • Responses HTTP/SSE controls check rejection and the original input location before preprocessing. Messages HTTP/SSE controls verify that unknown extension data reaches preprocessing as text instead of disappearing.
  • Content-processor regressions cover user, assistant, system, developer, and tool roles; existing supported-media and content-normalization tests pass.
  • A local integration of this fix with [Fix] Handle user image inputs in Responses API #5043, fix(serve): preserve tool-result image ownership #5044, and test(serve): cover user images across three HTTP APIs #5046 passed the same test command: 617 passed.
  • Applicable pre-commit hooks and git diff --check passed.

Fresh real-model validation on 2026-10-20: the local integration with #5043, #5044, and #5046 at 501c5899 passed 106/106 serial HTTP cases against GLM-5.3-Flash on 8 H200 GPUs (PyTorch TP8, MP executor, eager mode, batch size 2, session length 8192, no MTP). The request fixtures and expectations are identical to the earlier 104/106 run except for the served model alias.

  • User images: 42/42; tool-result images: 48/48; expected rejection cases: 16/16. All three APIs and streaming/non-streaming requests are represented. All 90 successful responses also passed a separate answer/completion audit.
  • The original two unknown-only Chat failures now return HTTP 400 with JSON in both stream modes. The error message contains messages[0].content[0].type; no SSE stream is opened.
  • A separate full Router/Direct comparison ran 56 cases per endpoint at concurrency 2: all 112 requests returned, with no transport, SSE, or service failures. Existing Router request-rewriting differences remain outside this PR's scope. This run did not reproduce the previously tracked inference stall; it does not establish that the intermittent issue is fixed or identify its cause.

The temporary GPU service and Router were stopped after validation.

Dependency and review scope

This fix is based directly on upstream main at 52e57d8f, which includes merged #5043. The rebase preserves that PR's image-aware Responses converter and error locations unchanged; the overlapping Responses production-code change is no longer part of this diff. This PR does not depend on the pending #5044 or #5046. The real-model results above were obtained on the explicitly identified earlier integration revision; the current rebase was validated with CPU automated tests.

Checklist

  • Applicable pre-commit hooks and whitespace checks passed.
  • Serving and content-processing regressions passed.
  • No new dependency or minimum-version requirement.
  • English and Chinese API documentation updated.

@silencelamb silencelamb changed the title [Fix] Reject unsupported chat content blocks fix(serve): reject unsupported chat content blocks Oct 20, 2026
@silencelamb
silencelamb force-pushed the fix/chat-content-validation branch from b49ceef to 66a519f Compare October 20, 2026 23:15

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant