Lightning-AI / Lightning-AI/pytorch-lightning
Forcing deterministic behavior in Fabric
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Description & Motivation
PyTorch has a utility to force deterministic algorithms via
```py
torch.use_deterministic_algorithms(True)
torch.backends.cudnn.benchmark = False
```
This can also be conveniently added in the Trainer:
```py
Trainer(deterministic=True)
```
(which also deactivates benchmark).
Fabric doesn't have this yet.
### Pitch
**Option 1: Init argument**
Introduce `Fabric(deterministic=bool|str)` with identical implementation/behavior as in Trainer.
Pro:
- Easy to discover
Con:
- Cannot disable it later on unless I create a new Fabric object. Maybe I want to disable it during training but enable it during validation/test/prediction.
**Option 2: Functional call**
Introduce a simple function `deterministic(bool|str)`.
Pro:
- Can be used anywhere you want, disable and enable on the fly
Con:
- Maybe not as easy to discover, but we can show it in examples
**Option 2: Both**
Simply do both options for flexibility.
### Alternatives
_No response_
### Additional context
_No response_
cc @borda @carmocca @justusschock @awaelchli
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing Trainer(deterministic=True) behavior and its handling of torch.use_deterministic_algorithms and cudnn.benchmark. Decide whether Fabric should expose an initialization argument, a functional call, or both; done means the selected interface provides equivalent deterministic behavior and supports the intended enable/disable use cases.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100