deepspeedai / deepspeedai/DeepSpeed
[REQUEST] Example of H5 dataloader based training on azure VM for multi-node
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43.1k
- Forks
- 5k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 112
Description
Is your feature request related to a problem? Please describe.
Deepspeed being a library for high speed training large model but most of the DL developers use Azure VMs and run multi-node training with their data being as H5 files. But there is no clear indication if H5 files are supported and how the training with deepspeed is being setup with H5 for multinode training. There is explanation on the communication time speed-ups when using multi-node training
Describe the solution you'd like
An example of deepspeed training with data being stored as H5 files and used for training on Azure VMs under multi-node scenario
Examples like this will help in wider adoption and building efficient training pipeline using deepspeed
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing DeepSpeed's existing distributed-training examples and the data-loading path relevant to H5 files. Document a runnable multi-node training example on Azure VMs, including setup and execution steps. Done means the example clearly shows how H5 data is used for training across multiple nodes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, python, pytorch
- Domain
- cloud, distributed-systems, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100