huggingface / huggingface/diffusers
Torchao SD3 int8wo
- Vorherrschende Sprache
- Python
- Sterne
- 34.5k
- Forks
- 7.3k
- Ø Merge
- 3 T. 3 Std.
- Gemergte PRs (30 T.)
- 91
Beschreibung
### Describe the bug
Exception when click Run, SD3 quantization
### Reproduction
- Mellon + ModularDiffuser setup (https://github.com/cubiq/Mellon/blob/main/modules/ModularDiffusers/README.md)
- uv pip install -U torch torchao
- python main.py
SD3 Text Encoder Loader: Dtype bfloat16, Quantization: TorchAO, Quant Type: int8wo
### Logs
```shell
20260314 21.08.15 [ERR] An unexpected error occurred during exec: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)
20260314 21.08.15 [ERR] Error executing node 50pz3Zy-IBsosDgI_t6gC (modules.Experiments.SD3TextEncodersLoader)
20260314 21.08.15 [ERR] Traceback (most recent call last):
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 216, in __call__
output = getattr(self, self.CALLBACK)(**self.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in execute
t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 465, in mm_exec
return memory_manager.exec(func, device, models, exclude, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 188, in exec
raise e
File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 172, in exec
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in
t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 52, in quantize
torchao(model, quant_type=torchao_quant_type)
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 70, in torchao
dtype = get_torchao_quant_method(quant_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 96, in get_torchao_quant_method
from torchao.quantization import (
ImportError: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1015, in execute_node
self.node_cache[id](**args)
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 220, in __call__
raise RuntimeError(f"Error executing {self.module_name}.{self.class_name}: {e}")
RuntimeError: Error executing modules.Experiments.SD3TextEncodersLoader: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)
20260314 21.08.15 [ERR] Error occurred in Traceback (most recent call last):
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 216, in __call__
output = getattr(self, self.CALLBACK)(**self.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in execute
t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/mellon/NodeBase.py", line 465, in mm_exec
return memory_manager.exec(func, device, models, exclude, args, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 188, in exec
raise e
File "/media/wasami/comfyui/Mellon/utils/memory_menager.py", line 172, in exec
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/modules/Experiments/StableDiffusion3.py", line 190, in
t5_encoder = self.mm_exec(lambda: quantize(t5_encoder, quantization, **kwargs), quant_device, exclude=[t5_encoder])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 52, in quantize
torchao(model, quant_type=torchao_quant_type)
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 70, in torchao
dtype = get_torchao_quant_method(quant_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/utils/quantization.py", line 96, in get_torchao_quant_method
from torchao.quantization import (
ImportError: cannot import name 'float8_dynamic_activation_float8_weight' from 'torchao.quantization' (/media/wasami/comfyui/Mellon/.venv/lib/python3.12/site-packages/torchao/quantization/__init__.py)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 318, in _main_worker
result = await self.loop.run_in_executor(None, partial(task, *args))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/wasami/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/concurrent/futures/thread.py", line 59, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 916, in execute_graph
self.execute_node(id, nodes[id], sid)
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1019, in execute_node
raise e
File "/media/wasami/comfyui/Mellon/mellon/server.py", line 1015, in execute_node
self.node_cache[id](**args)
```
### System Info
Ubuntu 24.04
RTX 5090
### Who can help?
_No response_
Beitragsleitfaden
Rechercherichtung
Beginne in utils/quantization.py, insbesondere bei get_torchao_quant_method und dem torchao-Aufruf in Zeile 70, und verfolge dann den SD3-Pfad in modules/Experiments/StableDiffusion3.py. Reproduziere den Fehler mit dem Mellon-Setup und dem SD3 Text Encoder Loader unter Verwendung von TorchAO int8wo. Fertig ist die Aufgabe, wenn der gemeldete ImportError für diese Konfiguration nicht mehr auftritt und der Quantisierungspfad erfolgreich ausgeführt wird.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- python, pytorch
- Bereich
- machine-learning
- Issue-Typ
- Bug
- Schwierigkeit
- 3/5
- Geschätzter Aufwand
- 1-2 Tage
- Aktivitätsstatus
- Veraltet
- Klarheit
- Größtenteils klar
- Anfängerfreundlichkeit
- 38/100