我已在模型加载中选择剪枝类型torch.float8_e4m3fn 了。为什么还是以32位精度推理呢
Open
Potential Bug
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
### Expected Behavior
正常运行工作流,
### Actual Behavior
got prompt
Using split attention in VAE
Using split attention in VAE
Requested to load FluxClipModel_
loaded completely 9.5367431640625e+25 4777.53759765625 True
clip missing: ['text_projection.weight']
Requested to load AutoencodingEngine
loaded completely 9.5367431640625e+25 319.7467155456543 True
model weight dtype torch.float8_e4m3fn, manual cast: torch.float32
model_type FLUX
Requested to load Flux
loaded completely 9.5367431640625e+25 11351.004943847656 True
torch.Size([1, 1, 1024, 1168])
torch.Size([1, 1, 1024, 1168])
### Steps to Reproduce
1
### Debug Logs
```powershell
1
```
### Other
1
Contributor guide
Assessment
This issue has not been assessed yet.