Lightning-AI / Lightning-AI/pytorch-lightning
Support for AdaptDL
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
Reporting from the idea-pool channel on slack, as discussed with @carmocca.
Hi there,
On the way to solve a OOM problem with dynamic batch sizes based on sequence length, I have just discovered AdaptDL. Might be an interesting library to support.
Some core features offered by AdaptDL are:
- Elastically schedule distributed DL training jobs in shared clusters.
- Cost-aware resource auto-scaling in cloud computing environments (e.g. AWS).
- Automatic batch size and learning rate scaling for distributed training.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the AdaptDL PyTorch documentation linked in the issue and reviewing how this project handles distributed deep-learning training. Determine the intended integration boundary for elastic scheduling, resource scaling, and automatic batch-size and learning-rate scaling. Done should mean the project has an agreed, tested support path for the requested AdaptDL capabilities.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- cloud, distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100