deepspeedai / deepspeedai/DeepSpeedExamples
Feature Request: add LazyPromptDataset to DeepSpeedChat
Open
@conglongli is already working on this.
Since Apr 27, 2023.
deespeed chat
enhancement
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
Currently, in our multi-GPU training process, every GPU process loads the train and evaluation datasets using the torch.load function. However, when the dataset is large, this can lead to an issue with CPU memory constraints.
To address this issue, I would like to suggest introducing the LazyPromptDataset functionality. This functionality will enable us to load the data on demand instead of pre-loading it into memory, which will save memory and improve overall performance.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.