huggingface / huggingface/optimum-nvidia

ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`

Open
#56 0 comments 0 reactions 1 assignee Claimed by @mfuntowicz View on GitHub
bug
Dominant language
Python
Stars
1k
Forks
103
Avg merge
2m
Merged PRs (30d)
1

Description

I am trying to run [the example generate code provided](https://github.com/huggingface/optimum-nvidia#generate) with the docker image version 0.1.0b2. It gives me the following error:

```
File "/opt/optimum-nvidia/src/optimum/nvidia/runtime.py", line 154, in _from_pretrained
builder.build(engine_folder, optimization_level)
File "/opt/optimum-nvidia/src/optimum/nvidia/builder.py", line 378, in build
hf_model = AutoModelForCausalLM.from_pretrained(
File "/usr/local/lib/python3.10/dist-packages/transformers/models/auto/auto_factory.py", line 566, in from_pretrained
return model_class.from_pretrained(
File "/usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py", line 2863, in from_pretrained
raise ImportError(
ImportError: Using `low_cpu_mem_usage=True` or a `device_map` requires Accelerate: `pip install accelerate`
```

Confirmed that the accelerate package has been installed.
```
root@r760xa-l40:/opt/optimum-nvidia/examples# pip list | grep accelerate
accelerate 0.20.3
```
It seems the `+ use_fp8=True,` is where the issue was generated. I can run it ok with the `transformers` lib without fp8.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.