huggingface / huggingface/diffusers

Add `diffusers-cli` support for quantization via `PipelineQuantizationConfig`

Open
#14,259 0 comments 0 reactions 1 assignee Claimed by @DN6 View on GitHub
Dominant language
Python
Stars
34.5k
Forks
7.3k
Avg merge
3d 3h
Merged PRs (30d)
91

Description

Following up on #13966 (`diffusers-cli` polish for agents), it would be great to make the CLI more agent-optimized for quantization use cases. Exposing quantization through the CLI would reduce the number of tokens an agent has to try out (see the [Is it agentic enough?](https://huggingface.co/blog/is-it-agentic-enough) study).

The quantization API surface is quite large with a lot of variation across quant configs, so the proposal is to expose it through `PipelineQuantizationConfig` via a `--quantize` flag that accepts a JSON config. For example:

```
diffusers-cli run \
--model black-forest-labs/FLUX.1-dev \
--quantize '{"quant_backend": "bitsandbytes_4bit", "quant_kwargs": {"bnb_4bit_compute_dtype": "bfloat16", "bnb_4bit_quant_type": "nf4"}, "components_to_quantize": ["transformer", "text_encoder_2"]}'
```

This doesn't have to be part of #13966 — it can be a separate follow-up.

We should also include a skill alongside it to cover all the variations in the quant configs.

Context: #13966 (discussion https://github.com/huggingface/diffusers/pull/13966#discussion_r3419194082).

---
Requested by Sayak Paul - [Slack thread](https://huggingface.slack.com/archives/C065E480NN9/p1784619207201959?thread_ts=1784619207.201959&cid=C065E480NN9) - [Agent trace](https://huggingface.co/buckets/huggingface/moon-bot-memory/tree/sessions/2026-07-21T08-01-54-357Z_a8d11d4a-cbc1-4c9c-9fe5-b17fcba41537.jsonl)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.