Skip to content

Qwen Image 2.1 transformer is incompatible with torch.compile #14821

Description

@abel1502

Describe the bug

QwenImage21Rope features a call to image_pad_mask.tolist(). When the transformer in the Qwen Image 2.1 pipeline is torch.compile-ed and then run, it causes the following error:

Unsupported: Tensor.tolist() with non-integer tensor
  Explanation: Dynamo currently does not support tracing `tolist()` on non-integer tensors.
  Hint: Ensure the input tensor to `tolist()` is an integer type (e.g., int8, int16, int32, int64).

  Developer debug context: call_method TensorVariable() to_list

 For more details about this graph break, please visit: https://meta-pytorch.github.io/compile-graph-break-site/gb/gb0109.html

from user code:
   File "[redacted]\.venv\Lib\site-packages\diffusers\models\transformers\transformer_qwenimage21.py", line 925, in forward
    rotary_emb = self.pos_embed(img_shapes[0], image_pad_mask, device=hidden_states.device)
  File "[redacted]\.venv\Lib\site-packages\diffusers\models\transformers\transformer_qwenimage21.py", line 686, in forward
    is_image_token = image_pad_mask.tolist()

Reproduction

import torch
from diffusers import QwenImage21Pipeline

pipe = QwenImage21Pipeline.from_pretrained(
    "Qwen/Qwen-Image-2.1",
    torch_dtype=torch.bfloat16,
).to("cuda")

# Some of this might be unrelated
torch._inductor.config.conv_1x1_as_mm = True
torch._inductor.config.coordinate_descent_tuning = True
torch._inductor.config.epilogue_fusion = False
torch._inductor.config.coordinate_descent_check_all_directions = True
torch._dynamo.config.capture_scalar_outputs = True

pipe.transformer.compile(
    mode="max-autotune", fullgraph=True,
)

image = pipe(
    prompt="The Mona Lisa or whatever.",
    width=2048, height=2048,
    num_inference_steps=40,
    generator=torch.Generator("cuda").manual_seed(42),
).images[0]

System Info

  • 🤗 Diffusers version: 0.41.0.dev0
  • Platform: Windows-11-10.0.26200-SP0
  • Running on Google Colab?: No
  • Python version: 3.14.2
  • PyTorch version (GPU?): 2.14.0+cu130 (True)
  • Huggingface_hub version: 1.32.0
  • Transformers version: 5.17.0
  • Accelerate version: 1.15.0
  • PEFT version: not installed
  • Safetensors version: 0.8.0
  • xFormers version: not installed
  • Accelerator: NVIDIA GeForce RTX 5090, 32607 MiB
  • Using GPU in script?: yes
  • Using distributed or parallel set-up in script?: no

Who can help?

@naykun @yiyixuxu , based on https://github.com/huggingface/diffusers/blame/main/src/diffusers/models/transformers/transformer_qwenimage21.py

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions