[Refit] Support remote sparse refit in PPO and distillation
- Dominant language
- Python
- Stars
- 2k
- Forks
- 561
- Avg merge
- 4d 5h
- Merged PRs (30d)
- 145
Description
## Problem
The remote S3/ZeroMQ sparse-refit lifecycle introduced by PR #2444 is wired only
through GRPO setup: validation, baseline startup, synchronizer attachment, and
weight-version commit all live on that path. PPO and distillation currently use
the existing dense refit path.
PR #2444 adds a setup-time error when `policy.generation.refit_transport` is set
for PPO or distillation so the option cannot silently no-op or fall back to dense
NCCL.
## Desired support
Extend remote sparse refit to PPO and distillation with the same lifecycle and
correctness contract as GRPO.
## Acceptance criteria
- Both algorithms validate the remote transport configuration before allocating
workers.
- Baseline construction overlaps independent rollout-engine initialization.
- The sparse synchronizer is attached before the first refit.
- Failed syncs obey the same terminal/recovery contract as GRPO.
- End-to-end tests compare the sparse path with each algorithm's dense control.
Contributor guide
Research direction
Start by tracing the GRPO setup and remote sparse-refit lifecycle introduced by PR #2444, then compare the PPO and distillation setup paths. Verify validation, baseline initialization overlap, synchronizer attachment, and failed-sync recovery, and add end-to-end comparisons between each algorithm's sparse and dense paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100