How to run such a memory intensive model
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 174
- PR merge metrics
- No merged PRs in 30d
Description
May I ask how you managed to run this memory intensive model. Did you use 3D parallelism?
When I was using DeepSpeed's pipeline for parallel processing, I found that due to the Batch class of the model itself, it was not very suitable for the input and output of the DeepSpeed pipeline model (I consulted DeepSpeed's official documentation, and I found that the pipeline model requires the inputs and outputs of each layer must be either a single torch.Tensor or a tuple of tensors).
What approach would you recommend in this case?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no repository files, tests, or entry points. Start by reading the model's Batch class alongside DeepSpeed pipeline documentation, especially its input and output constraints and the discussion of 3D parallelism. Done would require a documented, repository-specific approach for running the memory-intensive model with compatible inputs and outputs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100