ControlNetLoader - KeyError: 'pos_embed.proj.weight'

### Expected Behavior

- The node to load the Flux.1-dev Upscaler ControlNet model from [Link to model on HF](https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Upscaler)
- Worked on older torch Version **(2.4.1+cu121)**

### Actual Behavior

A Traceback happens

### Steps to Reproduce

Using the newest windows standalone package:

- torch 2.5.0+cu124
- with Flux gguf models

### Debug Logs

```powershell
# ComfyUI Error Report
## Error Details
- **Node Type:** ControlNetLoader
- **Exception Type:** KeyError
- **Exception Message:** 'pos_embed.proj.weight'
## Stack Trace

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\nodes.py", line 762, in load_controlnet
controlnet = comfy.controlnet.load_controlnet(controlnet_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\comfy\controlnet.py", line 654, in load_controlnet
cnet = load_controlnet_state_dict(comfy.utils.load_torch_file(ckpt_path, safe_load=True), model=model, model_options=model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\comfy\controlnet.py", line 565, in load_controlnet_state_dict
return load_controlnet_flux_instantx(controlnet_data, model_options=model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\comfy\controlnet.py", line 471, in load_controlnet_flux_instantx
new_sd = comfy.model_detection.convert_diffusers_mmdit(sd, "")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\comfy\model_detection.py", line 554, in convert_diffusers_mmdit
depth = state_dict["pos_embed.proj.weight"].shape[0] // 64
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^

```
## System Information
- **ComfyUI Version:** v0.2.6-1-g09fdb2b
- **Arguments:** ComfyUI\main.py --windows-standalone-build --fast
- **OS:** nt
- **Python Version:** 3.12.7 (tags/v3.12.7:0b05ead, Oct 1 2024, 03:06:41) [MSC v.1941 64 bit (AMD64)]
- **Embedded Python:** true
- **PyTorch Version:** 2.5.0+cu124
## Devices

- **Name:** cuda:0 NVIDIA GeForce RTX 4070 : cudaMallocAsync
- **Type:** cuda
- **VRAM Total:** 12877955072
- **VRAM Free:** 6825841802
- **Torch VRAM Total:** 1811939328
- **Torch VRAM Free:** 1372198026

## Logs
```
2024-10-30 14:33:23,749 - root - INFO - Total VRAM 12281 MB, total RAM 32710 MB
2024-10-30 14:33:23,749 - root - INFO - pytorch version: 2.5.0+cu124
2024-10-30 14:33:23,750 - root - INFO - Set vram state to: NORMAL_VRAM
2024-10-30 14:33:23,750 - root - INFO - Device: cuda:0 NVIDIA GeForce RTX 4070 : cudaMallocAsync
2024-10-30 14:33:25,198 - root - INFO - Using pytorch cross attention
2024-10-30 14:33:26,811 - root - INFO - [Prompt Server] web root: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\web
2024-10-30 14:33:26,813 - root - INFO - Adding extra search path checkpoints A:\StabilityMatrix\Data\Models/StableDiffusion
2024-10-30 14:33:26,814 - root - INFO - Adding extra search path configs A:\StabilityMatrix\Data\Models/configs
2024-10-30 14:33:26,814 - root - INFO - Adding extra search path clip A:\StabilityMatrix\Data\Models/CLIP
2024-10-30 14:33:26,814 - root - INFO - Adding extra search path clip_vision A:\StabilityMatrix\Data\Models/InvokeClipVision
2024-10-30 14:33:26,814 - root - INFO - Adding extra search path vae A:\StabilityMatrix\Data\Models/VAE
2024-10-30 14:33:26,814 - root - INFO - Adding extra search path loras A:\StabilityMatrix\Data\Models/Lora
2024-10-30 14:33:26,814 - root - INFO - Adding extra search path loras A:\StabilityMatrix\Data\Models/LyCORIS
2024-10-30 14:33:26,815 - root - INFO - Adding extra search path upscale_models A:\StabilityMatrix\Data\Models/ESRGAN
2024-10-30 14:33:26,815 - root - INFO - Adding extra search path upscale_models A:\StabilityMatrix\Data\Models/RealESRGAN
2024-10-30 14:33:26,815 - root - INFO - Adding extra search path upscale_models A:\StabilityMatrix\Data\Models/SwinIR
2024-10-30 14:33:26,816 - root - INFO - Adding extra search path embeddings A:\StabilityMatrix\Data\Models\TextualInversion
2024-10-30 14:33:26,816 - root - INFO - Adding extra search path hypernetworks A:\StabilityMatrix\Data\models/Hypernetwork
2024-10-30 14:33:26,816 - root - INFO - Adding extra search path controlnet A:\StabilityMatrix\Data\models/ControlNet
2024-10-30 14:33:29,205 - root - INFO - Total VRAM 12281 MB, total RAM 32710 MB
2024-10-30 14:33:29,205 - root - INFO - pytorch version: 2.5.0+cu124
2024-10-30 14:33:29,206 - root - INFO - Set vram state to: NORMAL_VRAM
2024-10-30 14:33:29,207 - root - INFO - Device: cuda:0 NVIDIA GeForce RTX 4070 : cudaMallocAsync
2024-10-30 14:33:32,585 - root - INFO -
Import times for custom nodes:
2024-10-30 14:33:32,585 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\websocket_image_save.py
2024-10-30 14:33:32,585 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\FRED_Auto_Crop_Image_SDXL_Face_Detect_v1.py
2024-10-30 14:33:32,586 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\cg-use-everywhere
2024-10-30 14:33:32,586 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI-AutomaticCFG
2024-10-30 14:33:32,586 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyLiterals
2024-10-30 14:33:32,587 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI_SLK_joy_caption_two
2024-10-30 14:33:32,587 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\sigmas_tools_and_the_golden_scheduler
2024-10-30 14:33:32,588 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\comfyui-various
2024-10-30 14:33:32,588 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\comfy-image-saver
2024-10-30 14:33:32,588 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI-Custom-Scripts
2024-10-30 14:33:32,589 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI-GGUF
2024-10-30 14:33:32,589 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\rgthree-comfy
2024-10-30 14:33:32,589 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI_ExtraModels
2024-10-30 14:33:32,590 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI_essentials
2024-10-30 14:33:32,590 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI-KJNodes
2024-10-30 14:33:32,590 - root - INFO - 0.0 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI_Comfyroll_CustomNodes
2024-10-30 14:33:32,591 - root - INFO - 0.1 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI_tinyterraNodes
2024-10-30 14:33:32,591 - root - INFO - 0.1 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\comfyui_face_parsing
2024-10-30 14:33:32,591 - root - INFO - 0.4 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI-Manager
2024-10-30 14:33:32,592 - root - INFO - 0.4 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI_FaceAnalysis
2024-10-30 14:33:32,594 - root - INFO - 0.5 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI-Crystools
2024-10-30 14:33:32,594 - root - INFO - 0.5 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI-Impact-Pack
2024-10-30 14:33:32,594 - root - INFO - 0.7 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\comfyui-reactor-node
2024-10-30 14:33:32,595 - root - INFO - 0.8 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\ComfyUI-Easy-Use
2024-10-30 14:33:32,595 - root - INFO - 1.5 seconds: A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\custom_nodes\was-node-suite-comfyui
2024-10-30 14:33:32,595 - root - INFO -
2024-10-30 14:33:32,609 - root - INFO - Starting server

2024-10-30 14:33:32,610 - root - INFO - To see the GUI go to: http://127.0.0.1:8188
2024-10-30 14:33:41,426 - root - INFO - got prompt
2024-10-30 14:33:41,514 - root - INFO - Using pytorch attention in VAE
2024-10-30 14:33:41,517 - root - INFO - Using pytorch attention in VAE
2024-10-30 14:33:41,883 - root - INFO - model weight dtype torch.bfloat16, manual cast: None
2024-10-30 14:33:41,884 - root - INFO - model_type FLUX
2024-10-30 14:33:55,121 - root - INFO - Requested to load AutoencodingEngine
2024-10-30 14:33:55,122 - root - INFO - Loading 1 new model
2024-10-30 14:33:55,238 - root - INFO - loaded completely 0.0 159.87335777282715 True
2024-10-30 14:33:56,113 - root - ERROR - !!! Exception during processing !!! 'pos_embed.proj.weight'
2024-10-30 14:33:56,118 - root - ERROR - Traceback (most recent call last):
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\execution.py", line 323, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\execution.py", line 198, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\execution.py", line 169, in _map_node_over_list
process_inputs(input_dict, i)
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\execution.py", line 158, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\nodes.py", line 762, in load_controlnet
controlnet = comfy.controlnet.load_controlnet(controlnet_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\comfy\controlnet.py", line 654, in load_controlnet
cnet = load_controlnet_state_dict(comfy.utils.load_torch_file(ckpt_path, safe_load=True), model=model, model_options=model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\comfy\controlnet.py", line 565, in load_controlnet_state_dict
return load_controlnet_flux_instantx(controlnet_data, model_options=model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\comfy\controlnet.py", line 471, in load_controlnet_flux_instantx
new_sd = comfy.model_detection.convert_diffusers_mmdit(sd, "")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "A:\ProgramFiles\ComfyUI_windows_portable_new\ComfyUI\comfy\model_detection.py", line 554, in convert_diffusers_mmdit
depth = state_dict["pos_embed.proj.weight"].shape[0] // 64
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'pos_embed.proj.weight'

2024-10-30 14:33:56,120 - root - INFO - Prompt executed in 14.68 seconds
```
## Attached Workflow
Please make sure that workflow does not contain any sensitive information such as API keys or passwords.
```
Workflow too large. Please manually upload the workflow from local file system.
```

## Additional Context
(Please add any additional context or steps to reproduce the error here)
```

### Other

_No response_