You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: MiniMax-M2 streaming tool parser takes cubic time on arguments that contain > #20955
TensorRT-LLM main at fe180a9 (tensorrt_llm/serve/tool_parser/minimax_m2_parser.py), pure Python, CPU is enough to reproduce
Python 3.12
Serving path: trtllm-serve ... --tool_parser minimax_m2, streaming chat with tools
Who can help?
@NVIDIA/trt-llm-runtime-devs (CODEOWNERS for tensorrt_llm/serve)
Reproduction
Stream one write_file tool call whose content argument is code with > characters, 4 characters per increment:
fromtensorrt_llm.serve.tool_parser.minimax_m2_parserimportMiniMaxM2ToolParservalue="if (a > b) { p->x = q; } // ok\n"*132# about 4k characterstext= ('<minimax:tool_call>\n<invoke name="write_file">\n''<parameter name="path">a.c</parameter>\n'f'<parameter name="content">{value}</parameter>\n</invoke>\n</minimax:tool_call>')
p=MiniMaxM2ToolParser()
foriinrange(0, len(text), 4):
p.parse_streaming_increment(text[i:i+4], tools)
value length
stream time, value with >
without >
1024
0.39 s
0.09 s
2048
2.6 s
0.33 s
4096
18.9 s
1.2 s
Expected behavior
Streaming parse time stays close to linear in the output length, independent of how many > the argument contains.
actual behavior
Time grows with the cube of the value length when the value contains >. The parser runs in post-processing on the server event loop (default num_postprocess_workers=0), so one such response stalls every other request on the server.
additional notes
invoke_regex, param_regex and the partial-invoke pattern capture the name with a lazy name=["']?(.*?)["']?\s*>. While the invoke or parameter is still open, every > in the value is another candidate end for the name, and from each one the body group scans to the end of the buffer. A name never contains >, so ([^>]*?) keeps the same matches with one candidate end. I have a fix with a test ready and will open a PR referencing this issue.
Activity
[NVIDIA#20955][fix] MiniMax-M2 tool parser: name the angle-bracket te…