pytorch / pytorch/pytorch

DistributedSampler(num_replicas=0) raises a rank error instead of rejecting num_replicas

Open Beginner friendly
#196,090 1 comment 0 reactions 0 assignees View on GitHub
bot-triaged module: dataloader module: error checking triaged
Dominant language
Python
Stars
103k
Forks
29.5k
PR merge metrics
PR metrics pending

Description

I confirmed this on current pytorch `main`. `num_replicas <= 0` is blamed on `rank`. Drafted with AI assistance; I reviewed the constructor checks.

> ### 🐛 Describe the bug
>
> `DistributedSampler(..., num_replicas=0, rank=0)` raises a `ValueError` about `rank`, not `num_replicas`.
>
> ```python
> import torch
> from torch.utils.data.distributed import DistributedSampler
>
> DistributedSampler(range(10), num_replicas=0, rank=0)
> # ValueError: Invalid rank 0, rank should be in the interval [0, -1]
> ```
>
> The constructor only checks `rank >= num_replicas or rank < 0`. When `num_replicas` is 0 or negative, that interval is meaningless and a valid `rank=0` is reported as invalid. Later `len(dataset) / num_replicas` would divide by zero if the rank check were reordered.
>
> ### Versions
>
> pytorch `main`.

cc @andrewkho @divyanshk @SsnL @VitalyFedyunin @dzhulgakov @scotts @malfet

Contributor guide

Open the contributing guide

Research direction

Start at the DistributedSampler constructor in torch.utils.data.distributed and reproduce the reported call with num_replicas=0. Review the constructor checks and verify that invalid num_replicas values are rejected directly rather than reported as an invalid rank. Done means the reported input produces a ValueError identifying num_replicas, without reaching division by zero.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
distributed-systems
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.