Public sync - 2026-07-07

.gitignore

@@ -44,12 +44,18 @@ wandb/
 *.wav
 *.webp
 
-# HDR IC-LoRA e2e test baseline (checked in via Git LFS)
-!packages/ltx-pipelines/tests/assets/expected_hdr_ic_lora_exr/frame_*.exr
+# Full-params expected results for the --full-e2e quality lane (checked in via Git LFS)
+!packages/ltx-pipelines/tests/assets/full_e2e/
+!packages/ltx-pipelines/tests/assets/full_e2e/*.mp4
+!packages/ltx-pipelines/tests/assets/full_e2e/*.wav
+!packages/ltx-pipelines/tests/assets/full_e2e/expected_hdr_ic_lora_exr/frame_*.exr
+
+# HDR IC-LoRA e2e test input clip (checked in via Git LFS)
 !packages/ltx-pipelines/tests/assets/hdr_ic_lora_test_input.mp4
 
-# Text-to-audio (T2A) e2e test baseline (checked in via Git LFS)
-!packages/ltx-pipelines/tests/assets/expected_t2a_one_stage_ltx2_3.wav
+# Fast integration-profile goldens (bit-exact decoded video/audio; checked in via Git LFS)
+!packages/ltx-pipelines/tests/assets/integration/
+!packages/ltx-pipelines/tests/assets/integration/*.safetensors
 
 # ltx-bench Grafana dashboards (source of truth in the repo)
 !packages/ltx-bench/grafana/*.json

README.md

@@ -14,19 +14,43 @@
 
 ## 🚀 Quick Start
 
+Clone the repo
+
 ```bash
-# Clone the repository
 git clone https://github.com/Lightricks/LTX-2.git
 cd LTX-2
+```
+
+Download the relevant [models](https://huggingface.co/Lightricks/LTX-2.3) or use the [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli)
 
-# Set up the environment
-uv sync --frozen
-source .venv/bin/activate
+```bash
+hf auth login
+hf download Lightricks/LTX-2.3 \
+    ltx-2.3-22b-distilled-1.1.safetensors ltx-2.3-spatial-upscaler-x2-1.1.safetensors --local-dir models/ltx-2.3
+hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
 ```
 
-### Required Models
+If you get a 401/403, accept the model terms on Hugging Face and log in with a **Read** token (fine-grained tokens need the "read gated repos" scope enabled).
+
+Generate
+
+```bash
+uv run python -m ltx_pipelines.distilled \
+    --distilled-checkpoint-path models/ltx-2.3/ltx-2.3-22b-distilled-1.1.safetensors \
+    --spatial-upsampler-path    models/ltx-2.3/ltx-2.3-spatial-upscaler-x2-1.1.safetensors \
+    --gemma-root models/gemma-3-12b \
+    --seed 42 \
+    --output-path output.mp4 \
+    --prompt "A medium close-up shot features a Caucasian man with a beard, wearing a green and white baseball cap without any letters on the front, and a light blue shirt over a white t-shirt. He is positioned in the center of the frame, looking intently directly at the camera, his eyes focused on camera. His facial expression is one of deep concentration, with his brow slightly raised. As he looks straight at the camera, a quick sniff sound is heard, and then he speaks with a deep male voice and a satisfied tone, saying, 'I think it's so good.' The camera remains static throughout, maintaining a shallow depth of field, which keeps the man in sharp focus while the background is softly blurred, showing a beige wall behind him. After a brief pause, another short, audible sniff is heard. The man then continues to speak, his voice maintaining the same quality, as he states, 'So good. So good.' He elaborates further, emphasizing his point with a final statement, 'This got to be, it's got to be the best tool I've ever seen.'"
+```
+
+In cases of GPU memory constraints, consider `--quantization fp8-cast --offload {cpu, disk}`. See [additional flags](packages/ltx-pipelines/docs/installation.md#common-cli-flags).
+
+This uses the distilled model and pipeline for fast results. For better quality or other capabilities, see [Models](#full-model-list) and [Pipelines](#available-pipelines).
+
+### Full Model List
 
-Download the following models from the [LTX-2.3 HuggingFace repository](https://huggingface.co/Lightricks/LTX-2.3):
+For pipelines beyond the quickstart, download the relevant models from the [LTX-2.3 HuggingFace repository](https://huggingface.co/Lightricks/LTX-2.3):
 
 **LTX-2.3 Model Checkpoint** (choose and download one of the following)
   * [`ltx-2.3-22b-dev.safetensors`](https://huggingface.co/Lightricks/LTX-2.3/blob/main/ltx-2.3-22b-dev.safetensors) - [Download](https://huggingface.co/Lightricks/LTX-2.3/resolve/main/ltx-2.3-22b-dev.safetensors)
@@ -76,9 +100,9 @@ Download the following models from the [LTX-2.3 HuggingFace repository](https://
 ### ⚡ Optimization Tips
 
 * **Use DistilledPipeline** - Fastest inference with only 8 predefined sigmas (8 steps stage 1, 4 steps stage 2)
-* **Enable FP8 quantization** - Enables lower memory footprint: `--quantization fp8-cast` (CLI) or `quantization=QuantizationPolicy.fp8_cast()` (Python). Fp8-cast should be used with bf16 checkpoints, it shall downcast them on the fly. For Hopper GPUs with TensorRT-LLM, use `--quantization fp8-scaled-mm` for FP8 scaled matrix multiplication. Fp8-scaled-mm should be used with fp8 checkpoints.
-* **Install attention optimizations** - On datacenter Blackwell GPUs (B200), install FlashAttention 4 manually: `uv pip install 'flash-attn-4==4.0.0b9'` (this specific revision is the one we have verified against torch 2.9.1+cu128; newer betas have known issues on consumer Blackwell). On other CUDA GPUs (including Hopper), use xFormers (`uv sync --extra xformers`).
-* **Use gradient estimation** - Reduce inference steps from 40 to 20-30 while maintaining quality (see [pipeline documentation](packages/ltx-pipelines/README.md#denoising-loop-optimization))
+* **Enable FP8 quantization** - Enables lower memory footprint: `--quantization fp8-cast` (CLI) or `quantization=QuantizationPolicy.fp8_cast()` (Python). Fp8-cast should be used with bf16 checkpoints, it shall downcast them on the fly. On Hopper+ GPUs with native FP8 support, use `--quantization fp8-scaled-mm` for FP8 scaled matrix multiplication. Fp8-scaled-mm should be used with fp8 checkpoints.
+* **Install attention optimizations** - On datacenter Blackwell GPUs (B200), install FlashAttention 4 manually: `uv pip install 'flash-attn-4==4.0.0b9'` (this specific revision is the one we have verified against torch 2.9.1+cu128; newer betas have known issues on consumer Blackwell). On Hopper GPUs, install the FlashAttention 3 wheel. On other CUDA GPUs, PyTorch SDPA is used automatically. An installed backend is selected automatically at runtime; forcing a specific one is a Python-API option (`AttentionFunction.FLASH_ATTENTION_3`/`FLASH_ATTENTION_4`), not a CLI flag.
+* **Use gradient estimation** - Reduce inference steps from 40 to 20-30 while maintaining quality (see [pipeline documentation](packages/ltx-pipelines/docs/optimization.md#denoising-loop-optimization))
 * **Skip memory cleanup** - If you have sufficient VRAM, disable automatic memory cleanup between stages for faster processing
 * **Choose single-stage pipeline** - Use `TI2VidOneStagePipeline` for faster generation when high resolution isn't required
 
@@ -94,7 +118,7 @@ When writing prompts, focus on detailed, chronological descriptions of actions a
 - Describe lighting and colors
 - Note any changes or sudden events
 
-For additional guidance on writing a prompt please refer to <https://ltx.video/blog/how-to-prompt-for-ltx-2>
+For additional guidance on writing a prompt please refer to <https://ltx.io/blog/prompting-guide-for-ltx-2>
 
 ### Automatic Prompt Enhancement
 

packages/ltx-core/README.md

@@ -11,7 +11,7 @@ The foundational library for the LTX-2 Audio-Video generation model. This packag
 - **`block_streaming/`**: Memory-efficient inference that streams transformer blocks through the GPU one at a time (from pinned CPU buffers or directly from disk)
 - **`model/`**: PyTorch implementations of the LTX-2 Transformer, Video VAE, Audio VAE, Vocoder and Upscaler
 - **`text_encoders/gemma`**: Gemma text encoder implementation with tokenizers, feature extractors, and separate encoders for audio-video and video-only generation
-- **`quantization/`**: FP8 quantization backends (FP8-TensorRT-LLM scaled MM, FP8 cast) for reduced memory footprint.
+- **`quantization/`**: FP8 quantization backends (FP8 scaled MM, FP8 cast) for reduced memory footprint.
 
 ## 🚀 Quick Start
 
@@ -118,11 +118,9 @@ model = builder.build(device=torch.device("cuda"))
 
 The `quantization/` module provides FP8 quantization support for the LTX-2 transformer, significantly reducing memory usage while maintaining quality. Two backends are available:
 
-#### FP8 Scaled MM (TensorRT-LLM)
+#### FP8 Scaled MM
 
-Uses NVIDIA TensorRT-LLM's `cublas_scaled_mm` for efficient FP8 matrix multiplication. Weights are stored in FP8 format with per-tensor scaling, and inputs are quantized dynamically (or statically with calibration data).
-
-**Requirements**: `uv sync --frozen --extra fp8-trtllm`
+Uses PyTorch's `torch._scaled_mm` for efficient FP8 matrix multiplication. Weights are stored in FP8 format with per-tensor scaling, and inputs are quantized dynamically.
 
 **Usage with QuantizationPolicy:**
 

packages/ltx-core/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "ltx-core"
-version = "1.1.6"
+version = "v1.1.7"
 description = "Core implementation of Lightricks' LTX-2 model"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -13,38 +13,14 @@ dependencies = [
     "safetensors",
     "accelerate",
     "scipy>=1.14",
+    # Apple Silicon only: Apple's fused MPSGraph SDPA, the AUTOMATIC attention
+    # backend on MPS. The marker installs it on Apple Silicon and prunes it
+    # everywhere else (Linux/CUDA), so it is a hard requirement exactly where it
+    # is the only viable attention kernel. Requires torch>=2.11 (within the
+    # torch~=2.7 floor); the resolver forks torch to >=2.11 on macOS.
+    "mps-sdpa>=0.2.0; sys_platform == 'darwin' and platform_machine == 'arm64'",
 ]
 
-[project.optional-dependencies]
-xformers = ["xformers"]
-fp8-trtllm = [
-    "tensorrt-llm==1.0.0",
-    "onnx>=1.16.0,<1.20.0",
-    "openmpi",
-]
-
-[tool.uv]
-conflicts = [
-  [
-    { extra = "xformers" },
-    { extra = "fp8-trtllm" },
-  ],
-]
-
-[tool.uv.sources]
-xformers = { index = "pytorch" }
-tensorrt-llm = { index = "nvidia" }
-
-[[tool.uv.index]]
-name = "pytorch"
-url = "https://download.pytorch.org/whl/cu129"
-explicit = true
-
-[[tool.uv.index]]
-name = "nvidia"
-url = "https://pypi.nvidia.com/"
-explicit = true
-
 [build-system]
 requires = ["uv_build>=0.9.8,<0.10.0"]
 build-backend = "uv_build"

packages/ltx-core/src/ltx_core/batch_split.py

@@ -25,8 +25,12 @@
 
 def _split_perturbations(config: BatchedPerturbationConfig, sizes: list[int]) -> list[BatchedPerturbationConfig]:
     """Split a ``BatchedPerturbationConfig`` along the batch dimension."""
-    it = iter(config.perturbations)
-    return [BatchedPerturbationConfig([next(it) for _ in range(s)]) for s in sizes]
+    chunks = []
+    offset = 0
+    for size in sizes:
+        chunks.append(config.batch_slice(offset, offset + size))
+        offset += size
+    return chunks
 
 
 def _merge_tensors(tensors: list[torch.Tensor | None]) -> torch.Tensor | None:
@@ -54,6 +58,10 @@ def __init__(self, model: nn.Module, max_batch_size: int) -> None:
         self._model = model
         self._max_batch_size = max_batch_size
 
+    @property
+    def num_blocks(self) -> int:
+        return self._model.num_blocks
+
     def _get_chunk_sizes(self, batch_size: int) -> list[int]:
         full, remainder = divmod(batch_size, self._max_batch_size)
         sizes = [self._max_batch_size] * full
@@ -65,7 +73,7 @@ def forward(
         self,
         video: Modality | None,
         audio: Modality | None,
-        perturbations: BatchedPerturbationConfig,
+        perturbations: BatchedPerturbationConfig | None,
     ) -> tuple[torch.Tensor | None, torch.Tensor | None]:
         batch_size = (video or audio).latent.shape[0]
 
@@ -77,7 +85,9 @@ def forward(
 
         v_chunks = video.split(sizes) if video is not None else [None] * n
         a_chunks = audio.split(sizes) if audio is not None else [None] * n
-        p_chunks = _split_perturbations(perturbations, sizes)
+        # A None config means "perturb nothing"; forward it per chunk so the inner model
+        # builds a per-chunk all-keep mask (splitting None has nothing to slice).
+        p_chunks = _split_perturbations(perturbations, sizes) if perturbations is not None else [None] * n
 
         chunk_results = [
             self._model(video=vc, audio=ac, perturbations=pc)

packages/ltx-core/src/ltx_core/block_streaming/builder.py

@@ -17,6 +17,7 @@
 from ltx_core.block_streaming.pool import BufferPool
 from ltx_core.block_streaming.provider import WeightsProvider
 from ltx_core.block_streaming.source import DiskWeightSource, PinnedBlock, PinnedWeightSource, WeightSource
+from ltx_core.block_streaming.stream_sync import create_stream_sync
 from ltx_core.block_streaming.utils import (
     carve_buffer,
     derive_layout,
@@ -25,6 +26,7 @@
     resolve_attr,
 )
 from ltx_core.block_streaming.wrapper import BlockStreamingWrapper
+from ltx_core.devices import synchronize_device
 from ltx_core.loader.fuse_loras import FuseRule, bf16_fuse_rule, fuse_lora_weights
 from ltx_core.loader.helpers import create_meta_model, load_state_dict, read_model_config
 from ltx_core.loader.module_ops import ModuleOps
@@ -72,9 +74,14 @@ class StreamingModelBuilder(Generic[ModelType], ModelBuilderProtocol[ModelType])
             ``"transformer_blocks"``).
         blocks_prefix: State-dict key prefix for block weights
             (e.g. ``"transformer_blocks"``).
+        cpu_slots_count: Default number of pinned CPU buffer slots used by
+            :meth:`build` when it is not given an explicit ``cpu_slots_count``.
+            ``None`` = RAM streaming (all blocks pinned); a small value (e.g.
+            ``DISK_CPU_SLOTS``) selects disk streaming. Lets a builder fully
+            encode its offload behaviour so callers need not re-specify it.
     """
 
-    def __init__(
+    def __init__(  # noqa: PLR0913
         self,
         model_class_configurator: type[ModelConfigurator[ModelType]],
         model_path: str | tuple[str, ...],
@@ -86,6 +93,7 @@ def __init__(
         fuse_rule: FuseRule = bf16_fuse_rule,
         blocks_attr: str = "",
         blocks_prefix: str = "",
+        cpu_slots_count: int | None = None,
     ) -> None:
         # Read-only: typed with the covariant ModelType, so it must not be a mutable attribute.
         self._model_class_configurator: Final = model_class_configurator
@@ -98,6 +106,7 @@ def __init__(
         self._fuse_rule = fuse_rule
         self._blocks_attr = blocks_attr
         self._blocks_prefix = blocks_prefix
+        self._cpu_slots_count = cpu_slots_count
 
     @property
     def model_class_configurator(self) -> type[ModelConfigurator[ModelType]]:
@@ -107,6 +116,10 @@ def model_class_configurator(self) -> type[ModelConfigurator[ModelType]]:
     def model_path(self) -> str | tuple[str, ...]:
         return self._model_path
 
+    @property
+    def checkpoint(self) -> str | tuple[str, ...]:
+        return self._model_path
+
     @property
     def model_sd_ops(self) -> SDOps | None:
         return self._model_sd_ops
@@ -139,6 +152,10 @@ def blocks_attr(self) -> str:
     def blocks_prefix(self) -> str:
         return self._blocks_prefix
 
+    @property
+    def cpu_slots_count(self) -> int | None:
+        return self._cpu_slots_count
+
     def with_sd_ops(self, sd_ops: SDOps | None) -> Self:
         clone = copy.copy(self)
         clone._model_sd_ops = sd_ops
@@ -188,8 +205,10 @@ def build(
         Args:
             device: GPU device for compute. ``None`` defaults to ``cuda``.
             dtype: Weight dtype (e.g. ``torch.bfloat16``). Required.
-            cpu_slots_count: Number of pinned CPU buffer slots.
-                ``None`` = RAM streaming (all blocks pre-loaded with LoRA fusion).
+            cpu_slots_count: Number of pinned CPU buffer slots. ``None`` falls
+                back to the builder's configured ``cpu_slots_count``, and if that
+                is also ``None``, to RAM streaming (all blocks pre-loaded with
+                LoRA fusion).
             gpu_slots_count: Number of GPU buffer slots.
                 ``None`` = ``_DEFAULT_GPU_SLOTS`` (2).
         """
@@ -216,6 +235,7 @@ def build(
                 f"missing indices {missing}, unexpected indices {extra}"
             )
 
+        cpu_slots_count = cpu_slots_count if cpu_slots_count is not None else self._cpu_slots_count
         cpu_slots_count = cpu_slots_count if cpu_slots_count is not None else len(blocks)
         gpu_slots_count = gpu_slots_count if gpu_slots_count is not None else _DEFAULT_GPU_SLOTS
 
@@ -234,16 +254,11 @@ def build(
 
         self._load_non_block_weights(meta_model, non_block_keys, device, dtype, non_block_loras)
 
-        copy_stream = torch.cuda.Stream(device=device)
-        gpu_pool = BufferPool(
-            source.slot_nbytes,
-            gpu_slots_count,
-            device,
-            reuse_barrier=lambda event: copy_stream.wait_event(event),
-        )
+        sync = create_stream_sync(device)
+        gpu_pool = BufferPool(source.slot_nbytes, gpu_slots_count, device, reuse_barrier=sync.reuse_barrier)
         provider = WeightsProvider(
             gpu_pool,
-            copy_stream,
+            sync,
             device,
             source,
             lora_sources,
@@ -327,7 +342,7 @@ def _build_pinned_source(
             block_sd.sd[key] = None
             should_sync = True
         if should_sync:
-            torch.cuda.synchronize()
+            synchronize_device()
 
         # Fill remaining pinned keys from the source state dict.
         for key, view in fill_views.items():

packages/ltx-core/src/ltx_core/block_streaming/disk.py

@@ -153,9 +153,11 @@ def get_ab(
         if pair is None:
             return None
         a, b = pair
-        if device is not None and device.type == "cuda":
-            a = a.to(device=device, non_blocking=True)
-            b = b.to(device=device, non_blocking=True)
+        # Move A/B to a GPU-class target (CUDA/MPS) so the B@A aggregation runs on
+        # the device; on a CPU target they stay put. non_blocking only helps CUDA.
+        if device is not None and device.type in ("cuda", "mps"):
+            a = a.to(device=device, non_blocking=device.type == "cuda")
+            b = b.to(device=device, non_blocking=device.type == "cuda")
         if dtype is not None:
             a = a.to(dtype=dtype)
             b = b.to(dtype=dtype)

packages/ltx-core/src/ltx_core/block_streaming/pool.py

@@ -8,6 +8,7 @@
 import torch
 
 from ltx_core.block_streaming import utils
+from ltx_core.block_streaming.stream_sync import StreamEvent
 
 
 class BufferPool:
@@ -27,13 +28,13 @@ def __init__(
         slot_nbytes: int,
         capacity: int,
         device: torch.device,
-        reuse_barrier: Callable[[torch.cuda.Event], None],
+        reuse_barrier: Callable[[StreamEvent], None],
         pin_memory: bool = False,
     ) -> None:
         self._slot_nbytes = slot_nbytes
         self._capacity = capacity
         self._free: deque[torch.Tensor] = deque()
-        self._events: dict[int, torch.cuda.Event] = {}
+        self._events: dict[int, StreamEvent] = {}
         self._reuse_barrier = reuse_barrier
         buffer = utils.alloc_buffer(max(slot_nbytes * capacity, 1), device, pin_memory)
         for slot in range(capacity):
@@ -59,7 +60,7 @@ def acquire(self) -> torch.Tensor:
             self._reuse_barrier(event)
         return buffer
 
-    def release(self, buffer: torch.Tensor, event: torch.cuda.Event | None = None) -> None:
+    def release(self, buffer: torch.Tensor, event: StreamEvent | None = None) -> None:
         """Return a raw slot to the free list.
         The *buffer* must be the exact tensor object returned by :meth:`acquire`
         (reuse is keyed on its identity). If *event* is given it is waited on the