deepspeedai / deepspeedai/DeepSpeedExamples
unable to load 4 7b size model in step3
Open
@samadejacobs is already working on this.
Since May 18, 2023.
deespeed chat
new-config
system
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
My actor model and critic model are both 7b in size. When I run the step3, there will be memory overflow, which consumes about 225G of memory space. Is there any solution to reduce memory consumption without changing the size of the model?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.