Repository navigation
[Bug]: Kimi K3 and MiniMax-M3 tool parsers raise on deeply nested or very long numeric JSON arguments #20993
Description
System Info
- TensorRT-LLM main at 687359e (
tensorrt_llm/serve/tool_parser/kimi_k3_tool_parser.py,tensorrt_llm/serve/tool_parser/minimax_m3_parser.py), pure Python. A CPU is enough to reproduce. - Python 3.12
- Serving path:
trtllm-serve ... --tool_parser kimi_k3or--tool_parser minimax_m3, chat completions with tools
Who can help?
@NVIDIA/trt-llm-runtime-devs (CODEOWNERS for tensorrt_llm/serve)
Reproduction
from tensorrt_llm.serve.openai_protocol import ChatCompletionToolsParam
from tensorrt_llm.serve.tool_parser.kimi_k3_tool_parser import KimiK3ToolParser
from tensorrt_llm.serve.tool_parser.minimax_m3_parser import MiniMaxM3ToolParser
tools = [ChatCompletionToolsParam(function={
"name": "f", "parameters": {"type": "object", "properties": {"v": {"type": "object"}}}})]
deep = "[" * 100000 + "]" * 100000
k3 = ('<|open|>tools<|sep|><|open|>call tool="f" index="1"<|sep|>'
'<|open|>argument key="v" type="object"<|sep|>{}<|close|>argument<|sep|>'
'<|close|>call<|sep|><|close|>tools<|sep|>')
KimiK3ToolParser().detect_and_parse(k3.format(deep), tools) # RecursionError
KimiK3ToolParser().detect_and_parse(k3.format("9" * 4400), tools) # ValueError (int digit limit)
m3 = (']<]minimax[>[<tool_call>\n]<]minimax[>[<invoke name="f">\n'
']<]minimax[>[<v>{}]<]minimax[>[</v>\n]<]minimax[>[</invoke>\n]<]minimax[>[</tool_call>')
MiniMaxM3ToolParser().detect_and_parse(m3.format(deep), tools) # RecursionErrorExpected behavior
An argument that json.loads cannot decode keeps its raw text, which is what both parsers already do for plain invalid JSON.
actual behavior
| parser, argument form | value | result on main |
|---|---|---|
kimi_k3, argument type="object" |
100000 nested [ |
RecursionError at kimi_k3_tool_parser.py:210 |
kimi_k3, argument type="object" |
9997 nested [ |
RecursionError at kimi_k3_tool_parser.py:237 (json.dumps of the arguments object) |
kimi_k3, argument type="object" |
4400 digits | ValueError at kimi_k3_tool_parser.py:210 |
kimi_k3, json block |
100000 nested [ |
RecursionError at kimi_k3_tool_parser.py:224 |
kimi_k3, json block |
4400 digits | ValueError at kimi_k3_tool_parser.py:224 |
| minimax_m3, object parameter | 100000 nested [ |
RecursionError at minimax_m3_parser.py:58 |
| minimax_m3, object parameter | 9997 nested [ |
RecursionError at minimax_m3_parser.py:176 (json.dumps of the arguments object) |
For kimi_k3 the error leaves both detect_and_parse and the streaming path (finish and the section close call detect_and_parse). For minimax_m3 it leaves detect_and_parse, which the non-streaming chat path calls directly, so the request fails with HTTP 400 and the generated text is lost. The minimax_m3 streaming path catches it and returns the raw text.
additional notes
KimiK3ToolParser._coerce_value and _parse_call_arguments catch only json.JSONDecodeError. A number with more than 4300 digits makes json.loads raise a plain ValueError (the CPython integer string limit), and deep nesting raises RecursionError. minimax_m3_parser._coerce_value catches ValueError and lets RecursionError through. A value just under the decoder's nesting limit decodes, and then the json.dumps of the arguments object, one level deeper, raises RecursionError. The 9997 depth is the boundary on CPython 3.12 and can differ on other versions. I have a fix with tests ready and will open a PR that references this issue.
Activity
[NVIDIA#20993][test] Cover the json.dumps probe in Kimi K3 and MiniMa…
[NVIDIA#20993][fix] Kimi K3 and MiniMax-M3 tool parsers: keep the tex…
[NVIDIA#20993][test] Cover the json.dumps probe in Kimi K3 and MiniMa…
[NVIDIA#20993][fix] Emit only finalized Gemma4 streaming arguments
[NVIDIA#20993][fix] Gemma4 streaming: report a call only once its arg…
[NVIDIA#20993][fix] Gemma4 streaming: report every complete call in o…
[NVIDIA#20993][fix] Gemma4 tool parser: keep text after a call as con…