huggingface / huggingface/diffusers
GGUF + `torch.compile` is not compatible with Hub Kernels
Open
gguf
torch-compile
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
The set checks when using hub kernels with GGUF are not traceable with torch.compile.
https://github.com/huggingface/diffusers/blob/b0dc51da31a112e5cd4e8a3ee9eafff08dab913d/src/diffusers/quantizers/gguf/utils.py#L74-L76
The GGUF Kernels also need to be updated to support the latest version of torch.
Initial Solution has been proposed: #12556
Contributor guide
Assessment
This issue has not been assessed yet.