[ConstBob]

Summary

Add Cosmos3-Edge to the TensorRT-LLM Reasoner cookbook.

Cosmos3-Nano and Cosmos3-Super stay on the pinned nvcr.io/nvidia/tensorrt-llm/release:1.3.0rc22 image. That image does not include Cosmos3EdgeForConditionalGeneration, so Cosmos3-Edge is documented as a single-GPU server started from a TensorRT-LLM source build that includes NVIDIA/TensorRT-LLM#20307.

Changes

• cookbooks/cosmos3/README.md
  • Separates the Nano/Super release-image path from the Edge source-build path.
  • Adds the single-GPU Cosmos3-Edge trtllm-serve command on port 8001.
• cookbooks/cosmos3/reasoner/README.md
  • Lists Cosmos3-Edge in the TensorRT-LLM quickstart and notebook walkthrough.
• cookbooks/cosmos3/reasoner/run_with_tensorrt_llm.ipynb
  • Keeps the Nano and Super Docker cells on release:1.3.0rc22.
  • Adds an Edge cell that starts trtllm-serve nvidia/Cosmos3-Edge from the current environment and exits with a clear message when trtllm-serve is unavailable.
  • Updates the readiness cell to follow either the Docker container log or the Edge server log, and to ignore a missing Docker daemon when Edge is running outside Docker.

Testing

• Validated the notebook JSON and ran bash -n on every bash cell.
• Exercised the readiness cell locally with a stub /health server:
  • the Docker container path reaches ready;
  • the Edge process path reaches ready;
  • an exited Edge process fails instead of waiting until timeout.
• Ran the Edge server and readiness cells on one L40S with the nvidia/Cosmos3-Edge checkpoint, using a TensorRT-LLM environment that includes the Edge architecture.
  • trtllm-serve nvidia/Cosmos3-Edge --host 0.0.0.0 --port 8001 --max_num_tokens 32768 started successfully.
  • /health became ready.
  • /v1/models returned Cosmos3-Edge.
  • An OpenAI-compatible image request with assets/robot_153.jpg returned a non-empty caption.

The run used max_tokens=128 for the image request; the cookbook example keeps max_tokens=4096. Nano and Super launch commands are unchanged and were not rerun.

[pengcuo]

Adding to @ishovkun's comments (PID reuse and wait in a fresh shell, missing Edge shutdown, unconditional docker pull, request parity), all of which I agree with. Two more:

1. The Edge path depends on an unmerged TensorRT-LLM PR. NVIDIA/TensorRT-LLM#20307 is still open (head ConstBob:cosmos3-edge-reasoner, mergeable state blocked as of today), and no release up to v1.4.0rc1 contains Cosmos3EdgeForConditionalGeneration. The README tells readers to install "from a checkout that includes PR 20307" without naming a branch or commit, so the instructions cannot be followed from upstream main or from any image. Either hold the Edge section until it ships in a release image (then pin that image like Nano/Super), or name the exact fork branch and commit and label the section as a preview.
2. The Edge launch cell runs python -m pip install opencv-python-headless with whatever python is first on the %%bash PATH, which is not necessarily the environment that owns trtllm-serve (the Nano/Super cells install it inside the container, where that is unambiguous). Install into the interpreter that owns the entry point, e.g. "$(dirname "$(command -v trtllm-serve)")/python" -m pip install ..., or move it to the prerequisites.

The rest of the readiness cell reads correctly to me (Docker-absent branch, Edge PID liveness check).

[ishovkun]

LGTM

[yaoxu-crypto]

closing this PR. will merge 299 instead

[yaoxu-crypto]

@ConstBob , very sorry, I closed this one by accident. I meant to close the one in the cosmos repo. Could you please reopen this tomorrow? We're making some changes to the cookbook, see PR 324, your changes need to be on top of that one.

[ConstBob]

Hi @yaoxu-crypto , I tried to reopen this, but the Reopen pull request button does not show up on the page. gh pr reopen also failed with "Could not open the pull request". GitHub returned HTTP 422 Validation Failed and did not give a reason. Could you reopen this? Thanks!

[ConstBob]

Hi @yaoxu-crypto , since this pull request still cannot be reopened, I opened a new one on current main, on top of the changes you mentioned: #347