Add activation checkpointing to DPO and GRPO
Open
✨ enhancement
🏋 DPO
🏋 GRPO
- Dominant language
- Python
- Stars
- 19.3k
- Forks
- 3k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 194
Description
### Feature request
Currently TRL only implemented activation checkpointing for SFT (https://github.com/huggingface/trl/pull/2954), can we also add it to DPO and GRPO? Will the same code changes from SFT also work for those?
### Motivation
Since SFT has activation checkpointing, it would be good that other training algorithms have the same feature.
### Your contribution
If the same code changes from SFT also work for DPO and GRPO, I can submit a PR.
Contributor guide
Assessment
This issue has not been assessed yet.