huggingface / huggingface/diffusers

Add AMD Quark support to Diffusers

Open
#14,149 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

feature-request models quantization
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

Is your feature request related to a problem? Please describe.
Diffusers cannot load or run AMD Quark-quantized diffusion models through the standard from_pretrained APIs Today the only way to use a Quark-quantized UNet/transformer with Diffusers is to monkey-patch Quark into the auto-quantizer mappings at runtime, which is fragile and breaks when Diffusers internals change.
Describe the solution you'd like.
We would like to add Quark to AUTO_QUANTIZER_MAPPING (like bitsandbytes, GGUF, torchao, quanto, ModelOpt).
This would mean QuantizationMethod.QUARK, QuarkConfig, and QuarkDiffusersQuantizer would be registered in AUTO_QUANTIZER_MAPPING / AUTO_QUANTIZATION_CONFIG_MAPPING. There would be two load paths -- prequantized reload as well as online quantization. This also adds the optional amd-quark dependancy.

Describe alternatives you've considered.
We have tried runtime monkey-patching from Quark-side (currently in place).

Additional context.
This mirrors https://huggingface.co/docs/transformers/quantization/quark (the existing Transformers Quark integration). Implementation is ready at #14077.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the implementation referenced in #14077 and the standard from_pretrained APIs, then inspect AUTO_QUANTIZER_MAPPING and AUTO_QUANTIZATION_CONFIG_MAPPING. Done means QuarkConfig, QuantizationMethod.QUARK, and QuarkDiffusersQuantizer support both prequantized reload and online quantization with the optional amd-quark dependency.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.