alibaba / alibaba/ROLL

Colocated full-parameter weight sync: CPU fallback costs ~17% of every training step on seccomp-managed containers; request a pidfd_getfd-free GPU transport

Open
#485 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.4k
Forks
312
Avg merge
1h 2m
Merged PRs (30d)
2

Description

**Summary:** On containers where `pidfd_getfd` is blocked (e.g. AutoDL managed instances, seccomp filter mode), colocated weight sync must use the `ROLL_WEIGHT_SYNC_USE_CPU` CPU-staging path. For **full-parameter** models this costs a measured **~103 s per step (~17% of the step wall time)**. This is a feature request for a GPU-side transport that does not depend on `pidfd_getfd` (e.g. NCCL broadcast or pinned-memory staging). Related: #484 reports the same class of problem on the Megatron colocated LoRA path with a different root cause (`CUDA driver error` on an accelerator environment).

**Environment:**

- ROLL @ `370cb24c1036ea9145365478fcc40612b2186fc8`
- Actor: Qwen2.5-3B-Instruct, **full-parameter** PPO (agentic GRPO), bf16 weights ≈ **6.2 GB** per sync
- FSDP2 actor training + vLLM actor inference, colocated (train ⊕ infer on GPU 0/1; fixed-role services on GPU 2), 3× RTX 4090
- vLLM 0.10.2, torch 2.8.0+cu128, ray 2.48.0
- AutoDL managed container, `Seccomp: 2` (filter mode) — `pidfd_getfd` unavailable
- `ROLL_WEIGHT_SYNC_USE_CPU: "1"` (inherited through the example config chain)

**Why the CPU path is mandatory here:** `roll/utils/send_recv_utils.py:317-319` already documents this:

```
# Managed AutoDL containers block pidfd_getfd, which makes CUDA IPC
# deserialization fail even for colocated workers. Preserve the default zero-copy
# path unless this explicit portable transport is requested.
```

The opt-in CPU path (`send_recv_utils.py:320-321`, `bucket.detach().to("cpu").contiguous().numpy()` + `MultiprocessingSerializer`, called from `roll/third_party/fsdp2/model_update.py:229`) works correctly — this is a performance request, not a bug report.

**Measured cost (Qwen2.5-3B full-parameter, n=26 training steps):**

| Metric | Mean |
|---|---|
| `time/actor_train/model_update/total` | **103.1 s** |
| `time/step_model_update` (wall) | 117.0 s |
| `time/step_rollout` | 228.8 s |
| `time/step_train` | 215.3 s |

→ weight sync is **~17% of every training step**, and the single largest fixed per-step cost after rollout and the train pass. For LoRA-sized payloads this overhead is negligible (see #484: 169 MB), but it scales linearly with parameter count — at 7B+ full-parameter this would dominate.

**Request:** a colocated weight-sync transport that does not depend on `pidfd_getfd`, e.g.:

1. NCCL broadcast by extending the existing `_setup_broadcast_group` infrastructure in `roll/third_party/fsdp2/model_update.py` (currently separated/partial-overlap only) to fully-colocated mode, or
2. Pinned-memory staging with explicit `cudaMemcpy` (no IPC handles), or
3. Documented guidance on container runtimes that permit the syscalls CUDA IPC needs.

Happy to share the per-step metric dumps, the full step-time breakdown, or help benchmark alternatives — I have a reproducible 3×4090 colocated setup on the affected environment.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with roll/utils/send_recv_utils.py:317-321 and roll/third_party/fsdp2/model_update.py:229, then inspect the existing _setup_broadcast_group infrastructure. Compare a non-pidfd_getfd transport for fully colocated full-parameter sync and benchmark it on the documented 3×4090 setup; done means avoiding mandatory CPU staging without breaking weight updates in the affected container environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
distributed-systems, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.