[Bug]: DeepSeek V4.1 / V3.2 tool parsers cut a parameter value at a literal DSML closing tag and drop or leak the rest of the call #21014

System Info

• TensorRT-LLM main (3614f68). tensorrt_llm/serve/tool_parser/deepseekv41_parser.py and deepseekv32_parser.py have not changed since [TRTLLM-16621][feat] Support DeepSeek-V4.1-Flash #19727, so earlier builds with V4.1 support behave the same.
• CPU only: the repro calls the parser directly and does not need a model or a GPU.

Who can help?

@NVIDIA/trt-llm-runtime-devs

Information

• The official example scripts
• My own modified scripts

Reproduction

A tool-call argument can legitimately contain DSML closing-tag text. This happens, for example, when an agent writes a file or a note that quotes the tool-call format, or edits code that handles it. The parsers end the parameter, the invoke, or the whole call block at the first matching closing tag, even when that tag is inside the value.

import random

from tensorrt_llm.serve.openai_protocol import ChatCompletionToolsParam, FunctionDefinition
from tensorrt_llm.serve.tool_parser.deepseekv41_parser import DeepSeekV41Parser

TOOLS = [
    ChatCompletionToolsParam(function=FunctionDefinition(name="Write")),
    ChatCompletionToolsParam(function=FunctionDefinition(name="Bash")),
]
D = "｜DSML｜"


def param(name, value, string="true"):
    return f'<{D} parameter name="{name}" string="{string}">{value}</{D} parameter>\n'


def invoke(name, *params):
    return f'<{D} invoke name="{name}">\n' + "".join(params) + f"</{D} invoke>\n"


def output(*invokes):
    return "Sure.\n\n" + f"<{D} calls>\n" + "".join(invokes) + f"</{D} calls>"


CASES = {
    "parameter_close_in_value": output(
        invoke("Write", param("path", "/tmp/n.md"), param("content", f"tag is </{D} parameter> ok"))
    ),
    "invoke_and_calls_close_in_value": output(
        invoke("Write", param("path", "/tmp/n.md"), param("content", f"tag is </{D} invoke> and </{D} calls> ok")),
        invoke("Bash", param("command", "ls")),
    ),
}


def streamed(text):
    parser = DeepSeekV41Parser()
    random.seed(0)
    normal, calls, i = "", {}, 0
    while i < len(text):
        k = random.randint(1, 6)
        result = parser.parse_streaming_increment(text[i : i + k], TOOLS)
        i += k
        normal += result.normal_text or ""
        for c in result.calls:
            entry = calls.setdefault(c.tool_index, {"name": "", "arguments": ""})
            entry["name"] += c.name or ""
            entry["arguments"] += c.parameters or ""
    return normal, calls


for label, text in CASES.items():
    result = DeepSeekV41Parser().detect_and_parse(text, TOOLS)
    print(f"== {label}")
    print("  non-streaming:", repr(result.normal_text), [(c.name, c.parameters) for c in result.calls])
    normal, calls = streamed(text)
    print("  streaming:    ", repr(normal), list(calls.values()))

Output:

== parameter_close_in_value
  non-streaming: 'Sure.' [('Write', '{"path": "/tmp/n.md", "content": "tag is"}')]
  streaming:     'Sure.\n\n\n' [{'name': 'Write', 'arguments': '{"path": "/tmp/n.md", "content": "tag is"}'}]
== invoke_and_calls_close_in_value
  non-streaming: 'Sure.' [('Write', '{"path": "/tmp/n.md"}')]
  streaming:     'Sure.\n\n and  ok</｜DSML｜ parameter>\n\n\n' [{'name': 'Write', 'arguments': '{"path": "/tmp/n.md"}'}, {'name': 'Bash', 'arguments': '{"command": "ls"}'}]

DeepSeekV32Parser behaves the same with the V3.2 tags (</｜DSML｜parameter> etc.). With content = tag is </｜DSML｜parameter> ok, it returns {"path": "/tmp/n.md", "content": "tag is"}.

Expected behavior

Both modes return content unchanged (tag is </｜DSML｜ parameter> ok and tag is </｜DSML｜ invoke> and </｜DSML｜ calls> ok), together with both calls of the second case. No DSML text appears in the content.

Actual behavior

• </｜DSML｜ parameter> inside a value ends the parameter. The value is truncated with no error ("tag is"), and the rest of it is discarded.
• </｜DSML｜ invoke> inside a value ends the invoke.
  • Non-streaming: the content parameter disappears, and the following Bash call is dropped as well, because the call block also ends at the literal </｜DSML｜ calls>.
  • Streaming: the rest of the value, ending in a dangling </｜DSML｜ parameter>, is emitted to the client as assistant content.
• The two modes disagree with each other. For the same output, non-streaming returns one call and streaming returns two.

Additional notes

The cause is that every closing tag is found with a first-match search:

• the parameter regex (.*?)</｜DSML｜ parameter>;
• the invoke regex (.*?)</｜DSML｜ invoke>;
• the call-block regex (.*?)</｜DSML｜ calls>;
• the streaming search for the same tokens.

The model's output is structured, so a closing tag can be accepted only where the structure allows it to end something:

• </｜DSML｜ parameter> closes a parameter only when the next non-whitespace text is another <｜DSML｜ parameter or </｜DSML｜ invoke>;
• </｜DSML｜ invoke> closes an invoke only when it is followed by another <｜DSML｜ invoke, by </｜DSML｜ calls>, or by the end of the output.

In streaming, a closing tag whose follower has not arrived yet is held back until the next increment decides it. This keeps both modes consistent. Values that merely contain a closing tag, the common case, then parse whole.

One case stays ambiguous under any rule: a value that contains the full closing sequence </｜DSML｜ parameter>\n</｜DSML｜ invoke>\n</｜DSML｜ calls> cannot be told apart from a real end of the call block.

The open PRs #20873 (newlines between tool calls) and #20990 (incremental scanning) do not change how closing tags are matched. I have a patch along these lines in DeepSeekV32Parser, so DeepSeekV4Parser and DeepSeekV41Parser inherit it, and will open a PR.