diff --git a/.agents/skills/cookbooks/cosmos3/README.md b/.agents/skills/cookbooks/cosmos3/README.md index 000fe8dff..28c117329 100644 --- a/.agents/skills/cookbooks/cosmos3/README.md +++ b/.agents/skills/cookbooks/cosmos3/README.md @@ -191,7 +191,7 @@ git lfs install git clone https://github.com/NVIDIA/TensorRT-LLM.git cd TensorRT-LLM -git checkout bca6761ab84fbcd58fc7f914eade7de48b32e35e +git checkout cc63e66daf52644f1441e117bb7ba67d20566016 git submodule update --init --recursive git lfs pull @@ -219,58 +219,18 @@ Then install the Cosmos3 guardrail package in the same environment unless you explicitly disable guardrails before starting the server: ```bash -pip install cosmos_guardrail==0.3.0 +pip install cosmos_guardrail==0.3.2 # On headless servers without libGL.so.1, replace the OpenCV wheel pulled in by # cosmos_guardrail with the matching headless build: pip uninstall -y opencv-python pip install opencv-python-headless==5.0.0.93 ``` -#### Server-side NLTK data setup - -With `cosmos_guardrail==0.3.0`, NLTK's path checks can reject tokenizer or -dictionary files that are symlinks from a Hugging Face snapshot into its -`blobs/` directory. Prepare a separate copy of the NLTK data as regular files -**on the server, in the same container and shell used to launch TensorRT-LLM**: - -```bash -export NLTK_DATA="$(mktemp -d "${TMPDIR:-/tmp}/cosmos3-nltk.XXXXXX")" -python3 - <<'PY' -import os -import shutil -from pathlib import Path - -import nltk -from huggingface_hub import snapshot_download - -snapshot = snapshot_download( - "nvidia/Cosmos-1.0-Guardrail", - revision="cf03c0395fac8c4de386c0bdab12cc4fc8d66362", - allow_patterns=["blocklist/**"], -) -source = Path(snapshot) / "blocklist" / "nltk_data" -destination = Path(os.environ["NLTK_DATA"]).resolve() -shutil.copytree(source, destination, symlinks=False, dirs_exist_ok=True) -assert not any(path.is_symlink() for path in destination.rglob("*")) - -# Check both resource lookups used by the text blocklist before starting a GPU server. -nltk.data.path[:] = [str(destination)] -tokens = nltk.word_tokenize("You are an autonomous vehicle planning system.") -assert nltk.WordNetLemmatizer().lemmatize("vehicles") == "vehicle" -print("Guardrail NLTK data ready:", destination, tokens) -PY -``` - -Keep `NLTK_DATA` exported when starting the server below. Repeat this setup -after recreating the container or removing the temporary directory. Running it -only in the client notebook's environment does not configure a remote server. -This workaround does not rewrite cached files or symlinks, and keeps NLTK -path security and `use_guardrails=True` enabled. - -The separate `No safety models found, returning safe` warning in guardrail -0.3.0 refers to its intentionally empty video-content classifier list. Text -checks and face blurring remain configured; the NLTK workaround does not enable -video-content classification or suppress that warning. +`cosmos_guardrail` 0.3.2 copies its NLTK data out of the Hugging Face cache +on first use, so the server needs no separate NLTK data setup. Its +`No safety models found, returning safe` warning refers to the intentionally +empty video-content classifier list; text checks and face blurring remain +configured. Set the TensorRT-LLM source root for the shared VisualGen config YAMLs. Run this from inside the TensorRT-LLM checkout — the directory the `git clone` above diff --git a/.agents/skills/cookbooks/cosmos3/generator/action/README.md b/.agents/skills/cookbooks/cosmos3/generator/action/README.md index 8400c6d08..7b9aefae1 100644 --- a/.agents/skills/cookbooks/cosmos3/generator/action/README.md +++ b/.agents/skills/cookbooks/cosmos3/generator/action/README.md @@ -176,16 +176,14 @@ have available. ### Quickstart Set up and launch the VisualGen server with the one-GPU Nano config: -[TensorRT-LLM setup](../../README.md#tensorrt-llm-generator), including the -[server-side NLTK data setup](../../README.md#server-side-nltk-data-setup). -Keep `NLTK_DATA` exported in the server's shell. Action requests +[TensorRT-LLM setup](../../README.md#tensorrt-llm-generator). Action requests upload a conditioning image as multipart `image_reference` or a video as `video_reference`, and put the mode-specific fields under `extra_params`. Use the same trained AV prompt as the Cosmos Framework reference; the legacy `view_point` field is ignored by current TensorRT-LLM. Both notebooks' current multipart fields were exercised against TensorRT-LLM -source revision `bca6761ab84fbcd58fc7f914eade7de48b32e35e`. Follow the source +source revision `cc63e66daf52644f1441e117bb7ba67d20566016`. Follow the source pin in the shared setup. ```python diff --git a/.agents/skills/cookbooks/cosmos3/generator/action/run_fd_with_trt_llm.ipynb b/.agents/skills/cookbooks/cosmos3/generator/action/run_fd_with_trt_llm.ipynb index 1f241facd..488853a85 100644 --- a/.agents/skills/cookbooks/cosmos3/generator/action/run_fd_with_trt_llm.ipynb +++ b/.agents/skills/cookbooks/cosmos3/generator/action/run_fd_with_trt_llm.ipynb @@ -30,7 +30,6 @@ "\n", "Follow the shared [TensorRT-LLM setup](../../README.md#tensorrt-llm-generator), then run this\n", "from the TensorRT-LLM checkout. This notebook is a client; keep the server in a separate terminal.\n", - "Complete the [server-side NLTK data setup](../../README.md#server-side-nltk-data-setup) there first, and keep `NLTK_DATA` exported in that server terminal.\n", "\n", "```bash\n", "export TRTLLM_ROOT=\"${TRTLLM_ROOT:-$PWD}\"\n", diff --git a/.agents/skills/cookbooks/cosmos3/generator/action/run_id_with_trt_llm.ipynb b/.agents/skills/cookbooks/cosmos3/generator/action/run_id_with_trt_llm.ipynb index eb66355f8..ae70cf4c2 100644 --- a/.agents/skills/cookbooks/cosmos3/generator/action/run_id_with_trt_llm.ipynb +++ b/.agents/skills/cookbooks/cosmos3/generator/action/run_id_with_trt_llm.ipynb @@ -29,7 +29,6 @@ "\n", "Follow the shared [TensorRT-LLM setup](../../README.md#tensorrt-llm-generator), then run this\n", "from the TensorRT-LLM checkout. This notebook is a client; keep the server in a separate terminal.\n", - "Complete the [server-side NLTK data setup](../../README.md#server-side-nltk-data-setup) there first, and keep `NLTK_DATA` exported in that server terminal.\n", "\n", "```bash\n", "export TRTLLM_ROOT=\"${TRTLLM_ROOT:-$PWD}\"\n", diff --git a/.agents/skills/cookbooks/cosmos3/generator/transfer/README.md b/.agents/skills/cookbooks/cosmos3/generator/transfer/README.md index f479243d8..8bdc5b23e 100644 --- a/.agents/skills/cookbooks/cosmos3/generator/transfer/README.md +++ b/.agents/skills/cookbooks/cosmos3/generator/transfer/README.md @@ -199,9 +199,7 @@ reuses the previews from [`preview_helpers.py`](./preview_helpers.py), writing o ### Quickstart Set up and launch Nano or Super as described in the shared -[TensorRT-LLM setup](../../README.md#tensorrt-llm-generator), including the -[server-side NLTK data setup](../../README.md#server-side-nltk-data-setup). -Keep `NLTK_DATA` exported in the server's shell. TensorRT-LLM +[TensorRT-LLM setup](../../README.md#tensorrt-llm-generator). TensorRT-LLM accepts a raw source video as multipart `video_reference` when edge or blur will be computed on the server. The checked-in assets are already precomputed controls, so this example base64-encodes the control inside its hint and does diff --git a/.agents/skills/cookbooks/cosmos3/generator/transfer/run_video_transfer_with_trt_llm.ipynb b/.agents/skills/cookbooks/cosmos3/generator/transfer/run_video_transfer_with_trt_llm.ipynb index 124cedf7c..0b08f90a5 100644 --- a/.agents/skills/cookbooks/cosmos3/generator/transfer/run_video_transfer_with_trt_llm.ipynb +++ b/.agents/skills/cookbooks/cosmos3/generator/transfer/run_video_transfer_with_trt_llm.ipynb @@ -30,7 +30,6 @@ "\n", "Follow the shared [TensorRT-LLM setup](../../README.md#tensorrt-llm-generator), then launch one\n", "server from the TensorRT-LLM checkout.\n", - "Complete the [server-side NLTK data setup](../../README.md#server-side-nltk-data-setup) there first, and keep `NLTK_DATA` exported in that server terminal.\n", "\n", "```bash\n", "export TRTLLM_ROOT=\"${TRTLLM_ROOT:-$PWD}\"\n",