huggingface / huggingface/optimum-nvidia
ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`
- Dominant language
- Python
- Stars
- 1k
- Forks
- 103
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
I am trying to run [the example generate code provided](https://github.com/huggingface/optimum-nvidia#generate) with the docker image version 0.1.0b2. It gives me the following error:
```
File "/opt/optimum-nvidia/src/optimum/nvidia/runtime.py", line 154, in _from_pretrained
builder.build(engine_folder, optimization_level)
File "/opt/optimum-nvidia/src/optimum/nvidia/builder.py", line 378, in build
hf_model = AutoModelForCausalLM.from_pretrained(
File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 2863, in from_pretrained
raise ImportError(
ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`
```
Confirmed that the accelerate package has been installed.
```
root@r760xa-l40:/opt/optimum-nvidia/examples# pip list | grep accelerate
accelerate 0.20.3
```
It seems the `+ use_fp8=True,` is where the issue was generated. I can run it ok with the `transformers` lib without fp8.
Contributor guide
Assessment
This issue has not been assessed yet.