# Fix Remote URI Corruption and Leading Slash Truncation in `storage.join()`
- Dominant language
- Go
- Stars
- 7.5k
- Forks
- 886
- Avg merge
- 1d 12h
- Merged PRs (30d)
- 120
Description
# Fix Remote URI Corruption and Leading Slash Truncation in `storage.join()`
## Background & Problem
In `src/flyte/storage/_storage.py`:
```python
def join(*paths: str) -> str:
"""
Join multiple paths together. This is a wrapper around os.path.join.
# TODO replace with proper join with fsspec root etc
Args:
paths: Paths to be joined.
"""
return str(os.path.join(*paths))
```
1. **Windows Path Corruption for Cloud URIs**: On Windows, `os.path.join("s3://bucket", "prefix", "file.txt")` uses `\` as separator, outputting `"s3://bucket\\prefix\\file.txt"`. Cloud storage services and object stores reject backslashes in keys.
2. **Subpaths with Leading Slash Wipe Out Prefix**: In Python, `os.path.join` treats any argument beginning with `/` (e.g. `storage.join("s3://bucket/dir", "/subfile.txt")`) as an absolute root path and discards all preceding components, returning `"/subfile.txt"`.
3. **Empty arguments**: Calling `storage.join()` with empty args should return `""` cleanly.
## User Review Required
> [!NOTE]
> For remote paths (checked via `is_remote(paths[0])`), components will be joined using forward slashes (`/`), stripping redundant slashes from child components so prefixes are never dropped. For local paths, standard `os.path.join` behavior is preserved.
## Proposed Changes
### Core Library
#### [MODIFY] [src/flyte/storage/_storage.py](file:///c:/Users/Aayush%20Shankar/OneDrive/Desktop/flyte-sdk/flyte-sdk/src/flyte/storage/_storage.py)
- Update `join(*paths: str) -> str` to check if `paths[0]` is a remote URI (using `is_remote(paths[0])`).
- If remote: join using forward slashes `/`, stripping leading and trailing `/` from inner segments, preserving the protocol scheme.
- If local: use `os.path.join(*paths)`.
### Tests
#### [MODIFY] [tests/internal/storage/test_storage.py](file:///c:/Users/Aayush%20Shankar/OneDrive/Desktop/flyte-sdk/flyte-sdk/tests/internal/storage/test_storage.py)
- Add `test_storage_join()` test covering:
- S3 / GCS / ABFS / Flyte remote URIs with multiple path components.
- Remote URIs where subpaths contain leading slashes (`/sub/file.txt`).
- Local relative and absolute paths.
- Empty call `storage.join()`.
## Verification Plan
### Automated Tests
- Run Python verification tests exercising `storage.join` across remote URIs, Windows path styles, and leading-slash subpaths.
Contributor guide
Research direction
Start in src/flyte/storage/_storage.py by reading storage.join() and the existing is_remote() behavior. Add or update coverage in tests/internal/storage/test_storage.py for remote URI components, leading slashes, local paths, and an empty call; the work is done when those cases pass without changing local os.path.join behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100