[LLMC] llama3_wNa16_FP8KV: AttributeError: 'NoneType' object has no attribute 'group_size'
- Dominant language
- Python
- Stars
- 1.6k
- Forks
- 175
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 99
Description
[log](https://inteltf-jenk.sh.intel.com/job/AutoRound_LLMC_example_test/47//artifact/logs/llama3_wNa16_FP8KV/quant.log)
```
Traceback (most recent call last):
File "/home/uttest/jenkins/mlp-dgx-01/workspace/AutoRound_LLMC_example_test/llm-compressor/examples/autoround/quantization_kv_cache/llama3_example.py", line 44, in
oneshot(
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/entrypoints/oneshot.py", line 479, in oneshot
one_shot()
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/entrypoints/oneshot.py", line 206, in __call__
self.apply_recipe_modifiers(
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/entrypoints/oneshot.py", line 269, in apply_recipe_modifiers
pipeline(
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/pipelines/independent/pipeline.py", line 45, in __call__
pipeline(model, dataloader, dataset_args)
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/pipelines/sequential/helpers.py", line 488, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/pipelines/sequential/pipeline.py", line 160, in __call__
LifecycleCallbacks.sequential_epoch_end(subgraph.submodules(model))
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/core/session_functions.py", line 165, in sequential_epoch_end
return cls.event(EventType.SEQUENTIAL_EPOCH_END, modules=modules, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/core/session_functions.py", line 91, in event
return active_session().event(event_type, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/core/session.py", line 181, in event
mod_data = self._lifecycle.event(
^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/core/lifecycle.py", line 204, in event
data = mod.update_event(state=self.state, event=event, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/modifier.py", line 144, in update_event
self.on_sequential_epoch_end(state, event, **kwargs)
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/autoround/base.py", line 246, in on_sequential_epoch_end
self.apply_autoround(state, modules)
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/autoround/base.py", line 286, in apply_autoround
layer_config = self._build_layer_config_for_autoround(wrapped_model)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/autoround/base.py", line 672, in _build_layer_config_for_autoround
layer_scheme = self._quant_scheme_to_autoround_config(quant_scheme)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/uttest/miniforge3/envs/autoround_v0.15.0_release/lib/python3.12/site-packages/llmcompressor/modifiers/autoround/base.py", line 574, in _quant_scheme_to_autoround_config
group_size = weight_args.group_size
^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'group_size'
```
Contributor guide
Assessment
This issue has not been assessed yet.