Conversation
…ndency (does not support Apple Silicon).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backstory
I was attempting to test this model locally on an M5 Pro Mac and ran into issue with the flash-attn package requiring CUDA. flash-attn package was never actually used: every attention path in this codebase resolves to torch.nn.functional.scaled_dot_product_attention, either through diffusers' dispatch_attention_fn (default backend is native SDPA) or through the text encoder's ATTENTION_CLASSES, which maps "eager", "sdpa", and "flash_attention_2" to the same SDPA class. Since flash-attn ships no wheels, pip tried to compile it from source against nvcc and failed on any machine without the CUDA toolkit, making the package uninstallable on Apple Silicon and CPU-only setups.
Changes
device_map), relax torch pin to >=2.8.0
that prefer CUDA, then MPS, then CPU, with explicit overrides
respected
dtype instead of defaulting to CPU-only when device is unset
resident in bf16), and the MoE eager fallback on non-CUDA devices
Testing
Verified: clean-venv install with no flash-attn succeeds; a scaled-down DiT with the real architecture runs correct forward passes on MPS, including the padded/masked ragged-batch attention path, with MPS bf16 output within bf16 rounding of CPU fp32 (max abs diff 0.0148, std 0.58). Full pipeline run against the real ~49 GB checkpoint exercised (on Mac).