huggingface / huggingface/diffusers
Add AMD Quark support to Diffusers
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 34.5k
- Forks
- 7.3k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 91
Description
Is your feature request related to a problem? Please describe.
Diffusers cannot load or run AMD Quark-quantized diffusion models through the standard from_pretrained APIs Today the only way to use a Quark-quantized UNet/transformer with Diffusers is to monkey-patch Quark into the auto-quantizer mappings at runtime, which is fragile and breaks when Diffusers internals change.
Describe the solution you'd like.
We would like to add Quark to AUTO_QUANTIZER_MAPPING (like bitsandbytes, GGUF, torchao, quanto, ModelOpt).
This would mean QuantizationMethod.QUARK, QuarkConfig, and QuarkDiffusersQuantizer would be registered in AUTO_QUANTIZER_MAPPING / AUTO_QUANTIZATION_CONFIG_MAPPING. There would be two load paths -- prequantized reload as well as online quantization. This also adds the optional amd-quark dependancy.
Describe alternatives you've considered.
We have tried runtime monkey-patching from Quark-side (currently in place).
Additional context.
This mirrors https://huggingface.co/docs/transformers/quantization/quark (the existing Transformers Quark integration). Implementation is ready at #14077.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the implementation referenced in #14077 and the standard from_pretrained APIs, then inspect AUTO_QUANTIZER_MAPPING and AUTO_QUANTIZATION_CONFIG_MAPPING. Done means QuarkConfig, QuantizationMethod.QUARK, and QuarkDiffusersQuantizer support both prequantized reload and online quantization with the optional amd-quark dependency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100