Model save error
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 158
Description
### Custom Node Testing
- [ ] I have tried disabling custom nodes and the issue persists (see [how to disable custom nodes](https://docs.comfy.org/troubleshooting/custom-node-issues#step-1%3A-test-with-all-custom-nodes-disabled) if you need help)
### Expected Behavior
A new checkpoint of NoobAI must be saved.
### Actual Behavior
I just did to save a model(NoobAI)+CLIP+VAE by this node.
W/F is the simple one, just 4 nodes only, Load Checkpoint node, load dual CLIP node, Load VAE node and CheckpointSave.
### Steps to Reproduce
Just execute by these 4 nodes.
### Debug Logs
```powershell
[INFO] got prompt 03:36:29
[INFO] Using pytorch attention in VAE
[INFO] Using pytorch attention in VAE
[INFO] VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
[MultiGPU Core Patching] text_encoder_device_patched returning device: cuda:0 (current_text_encoder_device=cuda:0)
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float32
[INFO] Requested to load SDXLClipModel
[INFO] Model SDXLClipModel prepared for dynamic VRAM loading. 3120MB Staged. 0 patches attached. Force pre-loaded 180 weights: 400 KB.
[INFO] Requested to load SDXL
[INFO] Model SDXLClipModel prepared for dynamic VRAM loading. 3120MB Staged. 0 patches attached. Force pre-loaded 180 weights: 400 KB.
[INFO] Model SDXL prepared for dynamic VRAM loading. 4896MB Staged. 0 patches attached. Force pre-loaded 512 weights: 1197 KB.
[ERROR] !!! Exception during processing !!! empty(): argument 'device' must be torch.device, not FakeDevice
[ERROR] Traceback (most recent call last):
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 545, in execute
output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 344, in get_output_data
return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\custom_nodes\comfyui-lora-manager\py\metadata_collector\metadata_hook.py", line 168, in async_map_node_over_list_with_metadata
results = await original_map_node_over_list(
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 318, in _async_map_node_over_list
await process_inputs(input_dict, i)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\execution.py", line 306, in process_inputs
result = f(**inputs)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\comfy_extras\nodes_model_merging.py", line 248, in save
save_checkpoint(model, clip=clip, vae=vae, filename_prefix=filename_prefix, output_dir=self.output_dir, prompt=prompt, extra_pnginfo=extra_pnginfo)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\comfy_extras\nodes_model_merging.py", line 227, in save_checkpoint
comfy.sd.save_checkpoint(output_checkpoint, model, clip, vae, clip_vision, metadata=metadata, extra_keys=extra_keys)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\comfy\sd.py", line 2407, in save_checkpoint
sd = model.state_dict_for_saving(clip_sd, vae_sd, clip_vision_sd)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\comfy\model_patcher.py", line 1743, in state_dict_for_saving
return self.model.state_dict_for_saving(unet_state_dict, clip_state_dict=clip_state_dict, vae_state_dict=vae_state_dict, clip_vision_state_dict=clip_vision_state_dict)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\comfy\model_base.py", line 387, in state_dict_for_saving
extra_sds.append(self.model_config.process_clip_state_dict_for_saving(clip_state_dict))
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\comfy\supported_models.py", line 255, in process_clip_state_dict_for_saving
state_dict_g = diffusers_convert.convert_text_enc_state_dict_v20(state_dict, "clip_g")
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\comfy\diffusers_convert.py", line 177, in convert_text_enc_state_dict_v20
new_state_dict[relabelled_key + ".in_proj_weight"] = cat_tensors(tensors)
File "Z:\WorkSpace\StabilityMatrix\Data\Packages\ComfyUI\comfy\diffusers_convert.py", line 125, in cat_tensors
out = torch.empty(shape, device=tensors[0].device, dtype=tensors[0].dtype)
TypeError: empty(): argument 'device' must be torch.device, not FakeDevice
```
### Other
_No response_
Contributor guide
Research direction
Start by reproducing the four-node workflow with custom nodes disabled, then inspect comfy/diffusers_convert.py around cat_tensors and comfy/supported_models.py around process_clip_state_dict_for_saving. Use the traceback to follow checkpoint saving through comfy_extras/nodes_model_merging.py. Done means the NoobAI checkpoint saves without the FakeDevice TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100