huggingface / huggingface/trl

Add activation checkpointing to DPO and GRPO

Open
#3,717 9 comments 0 reactions 0 assignees View on GitHub
✨ enhancement 🏋 DPO 🏋 GRPO
Dominant language
Python
Stars
19.3k
Forks
3k
Avg merge
1d 20h
Merged PRs (30d)
194

Description

### Feature request

Currently TRL only implemented activation checkpointing for SFT (https://github.com/huggingface/trl/pull/2954), can we also add it to DPO and GRPO? Will the same code changes from SFT also work for those?

### Motivation

Since SFT has activation checkpointing, it would be good that other training algorithms have the same feature.

### Your contribution

If the same code changes from SFT also work for DPO and GRPO, I can submit a PR.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.