Lightning-AI / Lightning-AI/pytorch-lightning

Forcing deterministic behavior in Fabric

Open
#16,604 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

fabric feature
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

### Description & Motivation

PyTorch has a utility to force deterministic algorithms via

```py
torch.use_deterministic_algorithms(True)
torch.backends.cudnn.benchmark = False
```

This can also be conveniently added in the Trainer:

```py
Trainer(deterministic=True)
```
(which also deactivates benchmark).

Fabric doesn't have this yet.

### Pitch

**Option 1: Init argument**

Introduce `Fabric(deterministic=bool|str)` with identical implementation/behavior as in Trainer.
Pro:
- Easy to discover

Con:
- Cannot disable it later on unless I create a new Fabric object. Maybe I want to disable it during training but enable it during validation/test/prediction.

**Option 2: Functional call**

Introduce a simple function `deterministic(bool|str)`.
Pro:
- Can be used anywhere you want, disable and enable on the fly

Con:
- Maybe not as easy to discover, but we can show it in examples

**Option 2: Both**
Simply do both options for flexibility.

### Alternatives

_No response_

### Additional context

_No response_

cc @borda @carmocca @justusschock @awaelchli

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing Trainer(deterministic=True) behavior and its handling of torch.use_deterministic_algorithms and cudnn.benchmark. Decide whether Fabric should expose an initialization argument, a functional call, or both; done means the selected interface provides equivalent deterministic behavior and supports the intended enable/disable use cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.