FluxTransformer2DModel does not have config and cannot set_default_attn_processor

### Describe the bug

```
scheduler = FlowMatchEulerDiscreteScheduler.from_pretrained(
BFL_REPO, subfolder="scheduler", revision=REVISION
)
text_encoder = CLIPTextModel.from_pretrained(
"openai/clip-vit-large-patch14", torch_dtype=DTYPE
)
tokenizer = CLIPTokenizer.from_pretrained(
"openai/clip-vit-large-patch14", torch_dtype=DTYPE
)
text_encoder_2 = T5EncoderModel.from_pretrained(
BFL_REPO, subfolder="text_encoder_2", torch_dtype=DTYPE, revision=REVISION
)
tokenizer_2 = T5TokenizerFast.from_pretrained(
BFL_REPO, subfolder="tokenizer_2", torch_dtype=DTYPE, revision=REVISION
)
vae = AutoencoderKL.from_pretrained(
BFL_REPO, subfolder="vae", torch_dtype=DTYPE, revision=REVISION
)
transformer = FluxTransformer2DModel.from_pretrained(
BFL_REPO, subfolder="transformer", torch_dtype=DTYPE, revision=REVISION
)

quantize(transformer, weights=qfloat8)
freeze(transformer)

quantize(text_encoder_2, weights=qfloat8)
freeze(text_encoder_2)

pipe = FluxPipeline(
scheduler=scheduler,
text_encoder=text_encoder,
tokenizer=tokenizer,
text_encoder_2=None,
tokenizer_2=tokenizer_2,
vae=vae,
transformer=None,
)
pipe.text_encoder_2 = text_encoder_2
pipe.transformer = transformer
pipe.enable_model_cpu_offload()
```

Running the attention ops without SDPA
```
pipe.transformer.set_default_attn_processor()
```

Returns an error
```
1727 if name in modules:
1728 return modules[name]
-> 1729 raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")

AttributeError: 'FluxTransformer2DModel' object has no attribute 'set_default_attn_processor'
```

And also, you cannot do `pipe.transformer.config`

### Reproduction

Latest diffusers

### Logs

```shell

```

### System Info

Latest diffusion version (0.33.0)
A100
pytorch/pytorch:2.4.1-cuda12.4-cudnn9-devel
accelerate, peft, transformers are also all up to date

Moreover, torch.compile do not work