Lightning-AI / Lightning-AI/pytorch-lightning
Strategy for DeepSpeedRLHFEngine and DeepSpeed Hybrid Engine
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Description & Motivation
I want to train RLHF model with DeepSpeedRLHFEngine or DeepSpeed Hybrid Engine. My understanding is current DeepSpeedStrategy doesn't work with them. Would like to know any plan for new strategy to support them? If not in the near future, It's great if you could show me simple work around to do it. Thanks for reply!
### Pitch
_No response_
### Alternatives
_No response_
### Additional context
_No response_
cc @borda
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the current DeepSpeedStrategy and the DeepSpeedRLHFEngine and DeepSpeed Hybrid Engine mentioned in the issue. Determine whether supporting these engines requires a new strategy or a documented workaround; done means a clear implementation plan or usable workaround is available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100