huggingface / huggingface/diffusers
FLUX error when loading with low_cpu_mem_usage=False and ignore_mismatched_sizes=True
- Lingua principale
- Python
- Stelle
- 34.5k
- Fork
- 7.3k
- Merge medio
- 3g 3h
- PR unite (30g)
- 91
Descrizione
### Describe the bug
I'd like to change the input layers of FLUX for training some img2img stuff, but got:
`TypeError: expected str, bytes or os.PathLike object, not NoneType`
when loading `FluxTransformer2DModel` with `low_cpu_mem_usage=False, ignore_mismatched_sizes=True`.
### Reproduction
```
from diffusers.models import FluxTransformer2DModel
transformer = FluxTransformer2DModel.from_pretrained(
"black-forest-labs/FLUX.1-dev",
subfolder="transformer",
torch_dtype=weight_dtype,
low_cpu_mem_usage=False, ignore_mismatched_sizes=True,
revision=None, variant=None)
```
### Logs
```shell
expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/model_loading_utils.py", line 104, in load_state_dict
file_extension = os.path.basename(checkpoint_file).split(".")[-1]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 142, in basename
TypeError: expected str, bytes or os.PathLike object, not NoneType
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/xxxx/repos/xxxx/train.py", line xxxx, in
main()
File "/home/xxxx/repos/xxxx/train.py", line xxx, in main
transformer = load_flux(args, weight_dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/xxxx.py", line xx, in load_flux
transformer = FluxTransformer2DModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/modeling_utils.py", line 828, in from_pretrained
state_dict = load_state_dict(model_file, variant=variant)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/model_loading_utils.py", line 116, in load_state_dict
with open(checkpoint_file) as f:
^^^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
```
### System Info
- 🤗 Diffusers version: 0.31.0.dev0
- Platform: Linux-5.15.0-119-generic-x86_64-with-glibc2.31
- Running on Google Colab?: No
- Python version: 3.11.9
- PyTorch version (GPU?): 2.4.0+cu124 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.24.5
- Transformers version: 4.44.0
- Accelerate version: 0.33.0
- PEFT version: 0.12.0
- Bitsandbytes version: 0.43.3
- Safetensors version: 0.4.4
- xFormers version: 0.0.27.post2
- Accelerator: NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes
### Who can help?
@sayakpaul @DN6
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Start with diffusers/models/model_loading_utils.py and the FluxTransformer2DModel.from_pretrained path in diffusers/models/modeling_utils.py. Reproduce the failure using the provided FLUX.1-dev call with low_cpu_mem_usage=False and ignore_mismatched_sizes=True, then trace why checkpoint_file is None. Done means the loading call no longer raises this TypeError under those options and the behavior is covered by a regression test.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- python, pytorch
- Ambito
- machine-learning
- Tipo di issue
- Bug
- Difficoltà
- 3/5
- Tempo stimato
- 1-2 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100