InternLM / InternLM/InternLM-XComposer
Can't run with multiple gpu
Open
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 175
- PR merge metrics
- No merged PRs in 30d
Description
Hello, thanks for the great work!
I refer to the `example_code/example_chat.py` to run the newest InternLM-XComposer-2.5 model using 4 NVIDIA 4090 GPUs. But still meet the OOM problem. It seems that although the weights are divided successfully, the first gpu always runs into OOM when `model.chat` is called.
Any response will be greatly appreciated!
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.