[Bug]: Streaming with n > 1 repeats decoded text in the outputs #21309

With streaming=True and n > 1, the final text of the outputs can contain repeated pieces of decoded text so the text no longer matches the token_ids (1x H200, Qwen/Qwen3-30B-A3B: 6 of 6 outputs over 3 prompts). Non-streaming n > 1 and streaming n = 1 are correct, so a client that streams several completions per request can get corrupted text.

System Info

• 1x H200, nvcr.io/nvidia/tensorrt-llm/release:1.4.0rc2.dev202610260007, LLM API
• Qwen/Qwen3-30B-A3B
• tensorrt_llm/executor/result.py in this image is identical to main; the links below are at main bba275b1ab.

Who can help?

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Save as repro.py and run python repro.py. For each output it prints the tail of text and the tail of the decoded token_ids, which should be equal.

import asyncio
from tensorrt_llm import LLM, SamplingParams
from tensorrt_llm.llmapi import KvCacheConfig


async def main():
    # host_cache_size=0 works around a separate host-memory startup failure on the test host.
    llm = LLM(model="Qwen/Qwen3-30B-A3B",
              kv_cache_config=KvCacheConfig(free_gpu_memory_fraction=0.6, host_cache_size=0))
    sp = SamplingParams(n=2, temperature=0.8, top_k=20, max_tokens=48, seed=1234)
    final = None
    async for out in llm.generate_async("Write two sentences about the sea.", sampling_params=sp, streaming=True):
        final = out
    for c in final.outputs:
        print(repr(c.text[-40:]))
        print(repr(llm.tokenizer.decode(c.token_ids, skip_special_tokens=True)[-40:]))


if __name__ == "__main__":
    asyncio.run(main())

Expected behavior

Each output's text equals its decoded token_ids, as it does without streaming or with n=1.

actual behavior

Pieces of the text are repeated, for example:

text:    '.\n\nTheThe sea sea spark sparklesles with'
decoded: 'sound of the sea.\n\nThe sea sparkles with'

Over 3 prompts, all 6 streaming n=2 outputs were affected. Non-streaming n=2 (6 outputs) and streaming n=1 (3 outputs) matched their tokens.

additional notes

Without beam search each response carries one sequence (result.py#L695), and _handle_sequence advances only that output's _last_token_ids_len (#L371). The detokenization loop then runs over every output (#L1024) and decodes token_ids_diff (#L1030), so an output that is not in the current response decodes its previous diff again. The OpenAI streaming handlers send output.text_diff for every output (postprocess_handlers.py#L475, #L882), so trtllm-serve with stream: true and n > 1 should be affected the same way; that path was not run.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and checked the documentation and examples for answers to frequently asked questions.