kohya-ss / kohya-ss/sd-scripts
zero3 train flux, accelerate unwrap_model got tensor(size(0))
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 1.2k
- Avg merge
- 11m
- Merged PRs (30d)
- 2
Description
Hi, I am using deepspeed zero3 to fine tune flux model using the script - flux_train_network.py.
```
flux = accelerator.unwrap_model(flux)
print(f"flux - {flux.state_dict()['single_blocks.7.linear1.weight'].shape}")
print(f"flux - {flux.state_dict()['single_blocks.7.linear1.weight'].device}")
```
I got the below result:

and When I save the trained model, I got this:

Here is my zero config:

Contributor guide
No contributing guide indexed for this repository
Research direction
Start with flux_train_network.py and the reported accelerator.unwrap_model(flux) call, then compare the zero configuration with the printed tensor shape and device. Reproduce the model save using the provided configuration and verify that the saved weights no longer contain the reported size(0) tensor.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100