FLUX error when loading with low_cpu_mem_usage=False and ignore_mismatched_sizes=True

### Describe the bug

I'd like to change the input layers of FLUX for training some img2img stuff, but got:
`TypeError: expected str, bytes or os.PathLike object, not NoneType`
when loading `FluxTransformer2DModel` with `low_cpu_mem_usage=False, ignore_mismatched_sizes=True`.

### Reproduction

```
from diffusers.models import FluxTransformer2DModel

transformer = FluxTransformer2DModel.from_pretrained(
"black-forest-labs/FLUX.1-dev",
subfolder="transformer",
torch_dtype=weight_dtype,
low_cpu_mem_usage=False, ignore_mismatched_sizes=True,
revision=None, variant=None)
```

### Logs

```shell
expected str, bytes or os.PathLike object, not NoneType
Traceback (most recent call last):
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/model_loading_utils.py", line 104, in load_state_dict
file_extension = os.path.basename(checkpoint_file).split(".")[-1]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 142, in basename
TypeError: expected str, bytes or os.PathLike object, not NoneType

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/xxxx/repos/xxxx/train.py", line xxxx, in
main()
File "/home/xxxx/repos/xxxx/train.py", line xxx, in main
transformer = load_flux(args, weight_dtype)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/xxxx.py", line xx, in load_flux
transformer = FluxTransformer2DModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/modeling_utils.py", line 828, in from_pretrained
state_dict = load_state_dict(model_file, variant=variant)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/xxxx/repos/xxxx/.venv/lib/python3.11/site-packages/diffusers/models/model_loading_utils.py", line 116, in load_state_dict
with open(checkpoint_file) as f:
^^^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
```

### System Info

- 🤗 Diffusers version: 0.31.0.dev0
- Platform: Linux-5.15.0-119-generic-x86_64-with-glibc2.31
- Running on Google Colab?: No
- Python version: 3.11.9
- PyTorch version (GPU?): 2.4.0+cu124 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Huggingface_hub version: 0.24.5
- Transformers version: 4.44.0
- Accelerate version: 0.33.0
- PEFT version: 0.12.0
- Bitsandbytes version: 0.43.3
- Safetensors version: 0.4.4
- xFormers version: 0.0.27.post2
- Accelerator: NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
NVIDIA A100 80GB PCIe, 81920 MiB
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help?

@sayakpaul @DN6