modelscope / modelscope/DiffSynth-Studio
DiffSynth-Studio低显存示例代码中的vram_config参数与ModelConfig不兼容导致报错
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
非常感谢各位大佬做出的贡献,我想尝试在4070Super的显卡推理Qwen-Image-Edit-2509模型,为了节省显存,不至于显存爆炸而使用https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/qwen_image/model_inference_low_vram/Qwen-Image-Edit-2509.py提供的样例代码,直接复制粘贴到pycharm后进行推理但是报错了。我查看了ModelConfig的源码,发现vram_config配置的参数,有很多参数均是无效的,vram_config参数如下:
vram_config = {
"offload_dtype": "disk",
"offload_device": "disk",
"onload_dtype": torch.float8_e4m3fn,
"onload_device": "cpu",
"preparing_dtype": torch.float8_e4m3fn,
"preparing_device": "cuda",
"computation_dtype": torch.bfloat16,
"computation_device": "cuda",
}
ModelConfig如下:
path: Union[str, list[str]] = None
model_id: str = None
origin_file_pattern: Union[str, list[str]] = None
download_resource: str = "ModelScope"
offload_device: Optional[Union[str, torch.device]] = None
offload_dtype: Optional[torch.dtype] = None
local_model_path: str = None
skip_download: bool = False
具体报错如下:
File "E:\Project_Style\Qwen-Image-Edit\Project\demo.py", line 25, in <module>
ModelConfig(model_id="Qwen/Qwen-Image-Edit-2509", origin_file_pattern="transformer/diffusion_pytorch_model*.safetensors", **vram_config),
TypeError: ModelConfig.__init__() got an unexpected keyword argument 'onload_dtype'
我的环境中diffsynth 的Version: 1.1.9,我不知道哪个版本才可以运行官方提供的代码,如果有知道的朋友,劳烦您告诉我一声,非常的感谢!
后来为了解决该问题,并且不明白为什么使用Qwen-Image-Edit-2509进行推理,却还要加载Qwen-Image的权重文件,我在modelscope中提前下载好Qwen-Image-Edit-2509模型。尝试修改代码,但依旧无效并报错。
具体代码如下:
vram_config = {
"offload_dtype": "disk",
"offload_device": "disk",}
model_path=r"E:\Project_Style\Qwen-Image-Edit\Models"
model_id="Qwen/Qwen-Image-Edit-2509"
pipe = QwenImagePipeline.from_pretrained(
torch_dtype=torch.bfloat16,
device="cuda",
model_configs=[
ModelConfig(model_id=model_id,
origin_file_pattern="transformer/",
local_model_path=model_path,
skip_download=True,
**vram_config
),
ModelConfig(model_id=model_id,
origin_file_pattern="text_encoder/",
local_model_path=model_path,
skip_download=True,
**vram_config
),
ModelConfig(model_id=model_id,
origin_file_pattern="vae/",
local_model_path=model_path,
skip_download=True,
**vram_config
),],
processor_config=ModelConfig(model_id=model_id,
origin_file_pattern="processor/",
skip_download=True,
local_model_path=model_path,),
)
报错如下:
Traceback (most recent call last):
File "E:\Project_Style\Qwen-Image-Edit\Project\降低显存的方式.py", line 28, in <module>
pipe = QwenImagePipeline.from_pretrained(
File "D:\Miniconda3\envs\llm\lib\site-packages\diffsynth\pipelines\qwen_image.py", line 331, in from_pretrained
model_manager.load_model(
File "D:\Miniconda3\envs\llm\lib\site-packages\diffsynth\models\model_manager.py", line 408, in load_model
if model_detector.match(file_path, state_dict):
File "D:\Miniconda3\envs\llm\lib\site-packages\diffsynth\models\model_manager.py", line 165, in match
if len(state_dict) == 0:
TypeError: object of type 'NoneType' has no len()
Loading models from: E:\Project_Style\Qwen-Image-Edit\Models\Qwen/Qwen-Image-Edit-2509\transformer/
为了解决这些问题,我耗费了很长的时间依旧没有头绪,有朋友知道如何解决该问题的,我非常的愿意向您请教,在此提前向您表示感谢!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/qwen_image/model_inference_low_vram/Qwen-Image-Edit-2509.py and compare its ModelConfig fields with the installed DiffSynth version 1.1.9. Then trace the loading path through qwen_image.py and model_manager.py, using the reported errors and local model path as reproduction points. Done means the example or documented local-model configuration runs without the reported TypeErrors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100