deepspeedai / deepspeedai/DeepSpeed
[REQUEST] Training with multiple optimizer
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 43.1k
- Forks
- 5k
- Avg merge
- 4d 15h
- Merged PRs (30d)
- 112
Description
Is your feature request related to a problem? Please describe.
It would be nice if we could train with multiple optimizers. That is currently not supported and that is very limiting in terms of model choice.
Describe the solution you'd like
Being able to train with multiple optimizers
Describe alternatives you've considered
No alternative. Otherwise model runs out of memory.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points, so start by locating the training and optimizer configuration paths in the repository. Define the supported multiple-optimizer behavior and verify that a representative training run can use it without the reported memory limitation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100