Add multi-GPU precompute test coverage for KTO
- Dominant language
- Python
- Stars
- 19.3k
- Forks
- 3k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 194
Description
#6403 fixes precompute for DPO **and** KTO but only adds a DPO distributed test (`test_dpo_precompute_ref_log_probs`, ddp/zero2/zero3). Add the KTO mirror so the duplicated paths can't silently diverge: `trl/scripts/kto.py --precompute_ref_log_probs --eval_strategy epoch` (two-pass, the case that broke ZeRO-3), same configs. Check `kto.py` wires `eval_dataset` like `dpo.py`; exclude FSDP2 until Issue 1 lands.
Contributor guide
Research direction
Start with the existing test_dpo_precompute_ref_log_probs distributed coverage and compare its ddp, zero2, and zero3 configurations. Then inspect trl/scripts/kto.py alongside dpo.py for eval_dataset wiring, and run the KTO equivalent with --precompute_ref_log_probs --eval_strategy epoch. Done means the KTO two-pass case is covered for the same configurations while FSDP2 remains excluded.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100