[ConstBob]

Do not merge. Wait for legal approval on the FP8 Hugging Face model cards. The model cards need to be approved and published on Hugging Face before this GitHub PR is merged.

Summary

Ports the Diffusers FP8 cookbook support from NVIDIA/cosmos#304 to
NVIDIA/cosmos-framework following the cookbook repository reorganization.
Updates the audiovisual, action forward/inverse dynamics, and transfer notebooks to:

• select BF16 or FP8 through COSMOS3_DIFFUSERS_PRECISION
• load Nano, Super, and Edge from their dedicated FP8 repositories
• load distilled T2I/I2V checkpoints from their fp8 revisions
• enable ModelOpt checkpoint restoration and FP8 GEMM support
• explicitly fetch transformer/modelopt_state.pth for a fresh Hub cache
• preload modular transformers where required by ModelOpt 0.44
• validate FP8 weights, quantizers, and unresolved meta tensors after loading
  The existing BF16 behavior remains the default.

Repository migration

The original implementation and review history are in NVIDIA/cosmos#304.
After NVIDIA/cosmos#384 and #292 moved cookbook
content into .agents/skills/, the changes were rebased onto the new
Framework locations while preserving the current Framework setup,
uv 0.11.29 guidance, and updated action imports.

Validation

Validated on an NVIDIA RTX PRO 6000 Blackwell:

• Nano FP8 from an empty Hub cache:
  • ModelOpt state restored
  • 504 FP8 weights, 1533 quantizers, 0 meta tensors
  • complete 189-frame, 1280×720 T2V generation
• Super FP8 from an independent empty cache:
  • 896 FP8 weights, 2709 quantizers, 0 meta tensors
• Distilled Super T2I FP8:
  • complete 768×768 image generation
• Distilled Super I2V FP8:
  • complete 189-frame, 832×480 video generation
• Nano action forward dynamics, inverse dynamics, and transfer paths were
  exercised end to end in the original implementation.
• Generated videos were decoded and checked for expected frame count and
  resolution.

[pengcuo]

Reviewed the four notebooks at cell level. The loading logic looks right to me; everything below is non-blocking except the first point, which is your own hold.

• Hold stands. nvidia/Cosmos3-{Nano,Super,Edge}-FP8 and the fp8 revisions of the two 4-Step cards resolve with an NVIDIA token but return 401 anonymously as of today, so the notebooks would fail for external users until the cards are public. Please keep the "Do not merge" note until then.
• I checked the concern that load_components() after update_components(transformer=...) would reload the transformer from repo+subfolder (where ModelOpt ignores the subfolder): diffusers ModularPipeline.load_components(names=None) only loads components whose attribute is still None, so the preloaded FP8 transformer is kept. A one-line comment saying so next to the preload would save the next reader the same check.
• validate_fp8_transformer counts float8_e4m3fn parameters and modules with "quantizer" in their name. That shows ModelOpt restored its graph, not that the W8A8 GEMM path is active (the fp8_per_tensor_gemm import is what enables it). Fine as a smoke check; just do not describe it as verifying FP8 GEMM support.
• The same helpers (precision_mode, resolve_pretrained_source, ensure_modelopt_state, validate_fp8_transformer) now live in four notebooks with slightly different signatures (materialize_checkpoint_dir(src, from_kwargs) vs (src, token)). That is the cookbook convention, so no change requested, but it is where drift will come from.

[yzhautouskay]

FYI I have created small PR to diffusers upstream to filter out weight_scale/input_scale warnings of "unused weights" when initializing Cosmos3OmniTransformer. They are supposed to be ignored by Diffusers/Transformers since this is vLLM-Omni/SGLang format, and these warnings pollute the logs

[yzhautouskay]

Hold stands. nvidia/Cosmos3-{Nano,Super,Edge}-FP8 and the fp8 revisions of the two 4-Step cards resolve with an NVIDIA token but return 401 anonymously as of today, so the notebooks would fail for external users until the cards are public. Please keep the "Do not merge" note until then.

Additionally, Edge-FP8 with the access to the private model repo still won't work, because checkpoint wasn't yet published to its HF repo main

[ConstBob]

Thanks for the review. The code comments are addressed in 9956d47.

• The "Do not merge" note stays until the FP8 Hugging Face model cards are public.
• The audiovisual FP8 support check now runs inside resolve_pretrained_source, before release_pipe(), matching the action notebooks. An unsupported model raises without dropping a pipeline that is already loaded.
• Noted next to the distilled and transfer preloads that load_components(names=None) only loads components that are still unset, so the preloaded FP8 transformer is kept.
• validate_fp8_transformer is documented as a smoke check of FP8 weights, quantizer modules, and meta tensors. It does not verify that the W8A8 GEMM path is active. That path is enabled by importing fp8_per_tensor_gemm.
• Left the duplicated helpers as they are.
• Noted the unused-weight warning filter in Fix: filter Cosmos3 warnings for auxiliary FP8 export scales huggingface/diffusers#15107; no change needed in this PR for that.