Lightning-AI / Lightning-AI/pytorch-lightning
TensorboardLogger not handling remote filesystem paths correctly on Windows environment
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 31.4k
- Forks
- 3.8k
- Avg merge
- 6d 7h
- Merged PRs (30d)
- 6
Description
### Bug description
When attempting to use a remote filesystem, the TensorboardLogger does not handle the name and version parameters correctly. The issue seems to occur specifically when running training scripts from Windows environments. The problem arises because the os.path.join() function tries to concatenate the different URI parts using Windows backslash characters (\) instead of forward slashes (/).
One possible solution to avoid this issue is to first check the filesystem being used (while parsing the save_dir logger parameter) and then employ the appropriate path concatenation strategy if a remote filesystem is detected, regardless of the operating system executing the script.
### What version are you seeing the problem on?
v2.0
### How to reproduce the bug
Here's an example using S3 (in my case, self-hosted in a Minio Docker container):
1. Run the following code from a **Windows environment** (ensure that **my_bucket** is already created, as failure to format the URI correctly may result in an **InvalidBucketName** issue during bucket creation):
```python
logger = TensorBoardLogger(save_dir='s3://my_bucket/logs')
```
2. Notice that in your 'my_bucket' bucket, there is a single folder named 'logs\lightning_logs\version_0' instead of a sequence of nested folders ('logs', 'lightning_logs', and 'version_0').
### Error messages and logs
No error messages or logs are generated. In my Minio bucket, paths are not split into multiple directories as expected when using TensorboardLogger with S3. Although I haven't checked other filesystems, I believe they may suffer from the same issue. Instead of creating multiple directories, a single directory is created with a long name that includes all the subdirectories.
### Environment
Current environment
```
#- Lightning Componen: Trainer, TensorboardLogger:
#- PyTorch Lightning Version: 2.0.2
#- PyTorch Version: 1.13.1
#- Python version: 3.10.9
#- OS: Windows
#- How you installed Lightning : poetry
```
### More info
_No response_
cc @awaelchli @borda @Blaizzy
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with TensorboardLogger's save_dir parsing and reproduce the issue on Windows using the provided S3 example. Confirm that the remote path is split into nested logs, lightning_logs, and version_0 directories with forward slashes rather than one backslash-containing directory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100