intel / intel/auto-round

[LLMC] llama3.1_W8A8_FP8_Block: ValueError: AutoRoundModifier only supports channel-wise and tensor-wise weight quantization

Open
#2,196 0 comments 0 reactions 1 assignee Claimed by @yiliu30 View on GitHub
Dominant language
Python
Stars
1.6k
Forks
175
Avg merge
1d 18h
Merged PRs (30d)
99

Description

[log](https://inteltf-jenk.sh.intel.com/job/AutoRound_LLMC_example_test/47//artifact/logs/llama3.1_W8A8_FP8_Block/quant.log)

```
(1/33): Calibrating: 0%| | 0/128 [00:00
oneshot(
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/entrypoints/oneshot.py", line 479, in oneshot
one_shot()
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/entrypoints/oneshot.py", line 206, in __call__
self.apply_recipe_modifiers(
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/entrypoints/oneshot.py", line 269, in apply_recipe_modifiers
pipeline(
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/pipelines/independent/pipeline.py", line 45, in __call__
pipeline(model, dataloader, dataset_args)
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/pipelines/sequential/helpers.py", line 488, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/pipelines/sequential/pipeline.py", line 160, in __call__
LifecycleCallbacks.sequential_epoch_end(subgraph.submodules(model))
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/core/session_functions.py", line 165, in sequential_epoch_end
return cls.event(EventType.SEQUENTIAL_EPOCH_END, modules=modules, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/core/session_functions.py", line 91, in event
return active_session().event(event_type, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/core/session.py", line 181, in event
mod_data = self._lifecycle.event(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/core/lifecycle.py", line 204, in event
data = mod.update_event(state=self.state, event=event, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/modifier.py", line 144, in update_event
self.on_sequential_epoch_end(state, event, **kwargs)
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/autoround/base.py", line 246, in on_sequential_epoch_end
self.apply_autoround(state, modules)
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/autoround/base.py", line 286, in apply_autoround
layer_config = self._build_layer_config_for_autoround(wrapped_model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/autoround/base.py", line 660, in _build_layer_config_for_autoround
default_config = self._quant_scheme_to_autoround_config(default_quant_scheme)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/autoround/base.py", line 584, in _quant_scheme_to_autoround_config
raise ValueError(
ValueError: AutoRoundModifier only supports channel-wise and tensor-wise weight quantization
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.