huggingface / huggingface/diffusers

Torchao SD3 int8wo

Aperta
#13,266 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
bug
Lingua principale
Python
Stelle
34.5k
Fork
7.3k
Merge medio
3g 3h
PR unite (30g)
91

Descrizione

### Describe the bug

Exception when click Run, SD3 quantization

### Reproduction

- Mellon + ModularDiffuser setup (https://github.com/cubiq/Mellon/blob/main/modules/ModularDiffusers/README.md)
- uv pip install -U torch torchao
- python main.py

SD3 Text Encoder Loader: Dtype bfloat16, Quantization: TorchAO, Quant Type: int8wo

### Logs

```shell
20260314 21.08.15 [ERR] An unexpected error occurred during exec: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)
20260314 21.08.15 [ERR] Error executing node 50pz3Zy-IBsosDgI_t6gC (modules.Experiments.SD3TextEncodersLoader)
20260314 21.08.15 [ERR] Traceback (most recent call last):
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 216, in __call__
output = getattr(self, self.CALLBACK)(**self.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in execute
t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 465, in mm_exec
return memory_manager.exec(func, device, models, exclude, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 188, in exec
raise e
File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 172, in exec
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in
t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 52, in quantize
torchao(model, quant_type=torchao_quant_type)
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 70, in torchao
dtype = get_torchao_quant_method(quant_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 96, in get_torchao_quant_method
from torchao.quantization import (
ImportError: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1015, in execute_node
self.node_cache[id](**args)
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 220, in __call__
raise RuntimeError(f"Error executing {self.module_name}.{self.class_name}: {e}")
RuntimeError: Error executing modules.Experiments.SD3TextEncodersLoader: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)

20260314 21.08.15 [ERR] Error occurred in Traceback (most recent call last):
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 216, in __call__
output = getattr(self, self.CALLBACK)(**self.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in execute
t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 465, in mm_exec
return memory_manager.exec(func, device, models, exclude, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 188, in exec
raise e
File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 172, in exec
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in
t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 52, in quantize
torchao(model, quant_type=torchao_quant_type)
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 70, in torchao
dtype = get_torchao_quant_method(quant_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 96, in get_torchao_quant_method
from torchao.quantization import (
ImportError: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 318, in _main_worker
result = await self.loop.run_in_executor(None, partial(task, *args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wasami/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/concurrent/futures/thread.py", line 59, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 916, in execute_graph
self.execute_node(id, nodes[id], sid)
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1019, in execute_node
raise e
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1015, in execute_node
self.node_cache[id](**args)
```

### System Info

Ubuntu 24.04
RTX 5090

### Who can help?

_No response_

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start in utils/quantization.py, especially get_torchao_quant_method and the torchao call at line 70, then trace the SD3 path in modules/Experiments/StableDiffusion3.py. Reproduce the failure with the Mellon setup and the SD3 Text Encoder Loader using TorchAO int8wo. Done means the reported ImportError no longer occurs for this configuration and the quantization path executes successfully.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python, pytorch
Ambito
machine-learning
Tipo di issue
Bug
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
38/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.