InternLM / InternLM/InternLM-XComposer

Can't run with multiple gpu

Open
#364 7 comments 0 reactions 1 assignee Claimed by @yhcao6 View on GitHub
Dominant language
Python
Stars
2.9k
Forks
175
PR merge metrics
No merged PRs in 30d

Description

Hello, thanks for the great work!

I refer to the `example_code/example_chat.py` to run the newest InternLM-XComposer-2.5 model using 4 NVIDIA 4090 GPUs. But still meet the OOM problem. It seems that although the weights are divided successfully, the first gpu always runs into OOM when `model.chat` is called.

Any response will be greatly appreciated!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.