huggingface / huggingface/diffusers

FluxTransformer2DModel does not have config and cannot set_default_attn_processor

Aperta
#11,398 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
bug stale
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

### Describe the bug

```
scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
BFL_REPO, subfolder="scheduler", revision=REVISION
)
text_encoder = CLIPTextModel.from_pretrained(
"openai/clip-vit-large-patch14", torch_dtype=DTYPE
)
tokenizer = CLIPTokenizer.from_pretrained(
"openai/clip-vit-large-patch14", torch_dtype=DTYPE
)
text_encoder_2 = T5EncoderModel.from_pretrained(
BFL_REPO, subfolder="text_encoder_2", torch_dtype=DTYPE, revision=REVISION
)
tokenizer_2 = T5TokenizerFast.from_pretrained(
BFL_REPO, subfolder="tokenizer_2", torch_dtype=DTYPE, revision=REVISION
)
vae = AutoencoderKL.from_pretrained(
BFL_REPO, subfolder="vae", torch_dtype=DTYPE, revision=REVISION
)
transformer = FluxTransformer2DModel.from_pretrained(
BFL_REPO, subfolder="transformer", torch_dtype=DTYPE, revision=REVISION
)

quantize(transformer, weights=qfloat8)
freeze(transformer)

quantize(text_encoder_2, weights=qfloat8)
freeze(text_encoder_2)

pipe = FluxPipeline(
scheduler=scheduler,
text_encoder=text_encoder,
tokenizer=tokenizer,
text_encoder_2=None,
tokenizer_2=tokenizer_2,
vae=vae,
transformer=None,
)
pipe.text_encoder_2 = text_encoder_2
pipe.transformer = transformer
pipe.enable_model_cpu_offload()
```

Running the attention ops without SDPA
```
pipe.transformer.set_default_attn_processor()
```

Returns an error
```
1727 if name in modules:
1728 return modules[name]
-> 1729 raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")

AttributeError: 'FluxTransformer2DModel' object has no attribute 'set_default_attn_processor'
```

And also, you cannot do `pipe.transformer.config`

### Reproduction

Latest diffusers

### Logs

```shell

```

### System Info

Latest diffusion version (0.33.0)
A100
pytorch/pytorch:2.4.1-cuda12.4-cudnn9-devel
accelerate, peft, transformers are also all up to date

Moreover, torch.compile do not work

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start with FluxTransformer2DModel and its use from FluxPipeline, reproducing the reported calls on diffusers 0.33.0. Compare the model's configuration and attention-processor behavior with related transformer models. Done means the reported config access and set_default_attn_processor call work, with the torch.compile concern checked.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
machine-learning
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.