MoonInTheRiver / MoonInTheRiver/DiffSinger
Multi-GPU training & batchsize problem
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.9k
- Forks
- 826
- PR merge metrics
- No merged PRs in 30d
Description
Hi, I really appreciate your work and now I'm going to train the model on this pipeline. My issues are as follows:
- I note that you have adapted the code to multi-GPU versions with DDP, but I cant figure out how to train with multi-GPU. Maybe I should set
self.use_ddp = Truehere ? - In the paper you mentioned that you trained DiffSinger on 1 NVIDIA V100 GPU with 48 batch size. However, I can't find any customizable variable related to batch size. Is it necessary to set batch size to match the number of GPU, if I want to train with multi-GPU?
Any suggestion is welcome.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read docs/README-SVS-opencpop-cascade.md and tasks/base_task.py around line 88 to trace the training entry point, DDP setting, and batch-size configuration. Confirm the required multi-GPU setup and how batch size relates to GPU count, then document or test the setup so the intended training run is reproducible.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100