Lightning-AI / Lightning-AI/pytorch-lightning

Checkpoints Saving with different permissions to account defaults

Open
#20,243 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug ver: 2.4.x
Dominant language
Python
Stars
31.4k
Forks
3.8k
Avg merge
6d 7h
Merged PRs (30d)
6

Description

### Bug description

When working on a managed cluster all our files save with "-rw-r--r--" permissions. However the checkpoint files from lightning are being saved with "-rw-------" permissions. We've gone round in circles trying to debug this and do not understand. Is there anything in the lightning code base that is causing this/is it configurable?

### Environment

Current environment

```
#- PyTorch Lightning Version (e.g., 2.4.0): 2.4.0
#- PyTorch Version (e.g., 2.4): 2.4
#- Python version (e.g., 3.12): 3.12
#- OS (e.g., Linux): Ubuntu 22
#- CUDA/cuDNN version: 12.5
#- GPU models and configuration: `8xH100`
#- How you installed Lightning(`conda`, `pip`, source): `conda`
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce checkpoint saving with Lightning 2.4.0 in the reported Ubuntu 22, Python 3.12, PyTorch 2.4 environment and compare its permissions with other saved files. Trace the checkpoint-saving entry point to determine whether Lightning sets the restrictive permissions, then identify whether the behavior is configurable and document the observed result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.