huggingface / huggingface/diffusers

FLUX error when loading with low_cpu_mem_usage=False and ignore_mismatched_sizes=True

Offen
#9,343 16 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug stale
Vorherrschende Sprache
Python
Sterne
34.5k
Forks
7.3k
Ø Merge
3 T. 3 Std.
Gemergte PRs (30 T.)
91

Beschreibung

### Describe the bug

I'd like to change the input layers of FLUX for training some img2img stuff, but got:
`TypeError: expected str, bytes or os.PathLike object, not NoneType`
when loading `FluxTransformer2DModel` with `low_cpu_mem_usage=False, ignore_mismatched_sizes=True`.

### Reproduction

```
from diffusers.models import FluxTransformer2DModel

transformer = FluxTransformer2DModel.from_pretrained(
"black-forest-labs/FLUX.1-dev",
subfolder="transformer",
torch_dtype=weight_dtype,
low_cpu_mem_usage=False, ignore_mismatched_sizes=True,
revision=None, variant=None)
```

### Logs

```shell
expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/model_loading_utils.py", line 104, in load_state_dict
file_extension = os.path.basename(checkpoint_file).split(".")[-1]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 142, in basename
TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/xxxx/repos/xxxx/train.py", line xxxx, in
main()
File "/home/xxxx/repos/xxxx/train.py", line xxx, in main
transformer = load_flux(args, weight_dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/xxxx.py", line xx, in load_flux
transformer = FluxTransformer2DModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/modeling_utils.py", line 828, in from_pretrained
state_dict = load_state_dict(model_file, variant=variant)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/model_loading_utils.py", line 116, in load_state_dict
with open(checkpoint_file) as f:
^^^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
```

### System Info

- 🤗 Diffusers version: 0.31.0.dev0
- Platform: Linux-5.15.0-119-generic-x86_64-with-glibc2.31
- Running on Google Colab?: No
- Python version: 3.11.9
- PyTorch version (GPU?): 2.4.0+cu124 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.24.5
- Transformers version: 4.44.0
- Accelerate version: 0.33.0
- PEFT version: 0.12.0
- Bitsandbytes version: 0.43.3
- Safetensors version: 0.4.4
- xFormers version: 0.0.27.post2
- Accelerator: NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help?

@sayakpaul @DN6

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start with diffusers/models/model_loading_utils.py and the FluxTransformer2DModel.from_pretrained path in diffusers/models/modeling_utils.py. Reproduce the failure using the provided FLUX.1-dev call with low_cpu_mem_usage=False and ignore_mismatched_sizes=True, then trace why checkpoint_file is None. Done means the loading call no longer raises this TypeError under those options and the behavior is covered by a regression test.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python, pytorch
Bereich
machine-learning
Issue-Typ
Bug
Schwierigkeit
3/5
Geschätzter Aufwand
1-2 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.