apache / apache/arrow-rs-object-store
LocalFileSystem rename fails with "No such file or directory" when source path contains hash fragment (#) on blobfuse mount
- Dominant language
- Rust
- Stars
- 322
- Forks
- 212
- Avg merge
- 5d 2h
- Merged PRs (30d)
- 10
Description
## Bug Report
### What happened
When using `object_store` with `LocalFileSystem` on a blobfuse-mounted Azure storage, file rename operations fail with "No such file or directory (os error 2)" when the source file path contains a hash fragment (`#`).
### Environment
- `object_store` version: 0.12.1
- Operating System: Linux (blobfuse)
- Storage backend: Azure Blob Storage via blobfuse mount
- Related library: deltalake 1.0.1 (Python bindings)
### To Reproduce
```python
import pandas as pd
import pyarrow as pa
import tempfile
import os
from pathlib import Path
try:
df = pd.DataFrame({'x': [1, 2, 3], 'y': ['a', 'a', 'b']})
arrow_table = pa.Table.from_pandas(df)
delta_table_path = "/lakehouse/default/Tables/xy_table2"
write_deltalake(
delta_table_path,
arrow_table,
mode="overwrite",
storage_options={"mount_allow_unsafe_rename": "true"}
)
delta_table = DeltaTable(delta_table_path)
print(f"{len(delta_table.to_pandas())}")
except Exception as e:
print(f"❌ error: {e}")
```
```
Error:
❌ error: Generic LocalFileSystem error
↳ Unable to rename file
↳ No such file or directory (os error 2)
```
1. Mount Azure Blob Storage using blobfuse
2. Attempt to rename a file with a path containing `#` character
3. The rename operation fails with `LocalFileSystem` error
### Expected behavior
File rename should succeed, or provide a more descriptive error message indicating the issue with hash fragments in paths.
### Actual behavior
Generic LocalFileSystem error ↳ Unable to rename file ↳ No such file or directory (os error 2)
### Additional context
This appears to be related to how blobfuse handles file paths with hash fragments. The blobfuse logs show:
Failed request, request_id: status: 404 response: {"error":{"code":"SourcePathNotFound","message":"The source path for a rename operation does not exist."}}
The source path in the blobfuse log shows a `#1` suffix that seems to be causing the issue:
src = /path/to/file.parquet#1
### Possible solutions
1. Add validation to reject file paths with hash fragments when using `LocalFileSystem` with a clear error message
2. Escape or handle hash fragments properly in file paths
3. Provide better error messaging to indicate the specific issue with path characters
### Related
- This issue is related to Delta Lake operations on Azure storage via blobfuse mounts
- May be related to: https://github.com/apache/arrow-rs/pull/5094
This issue focuses on the core problem: the object_store LocalFileSystem doesn't handle file paths with hash fragments properly when working with blobfuse mounts, and provides insufficient error messaging to diagnose the issue.
Contributor guide
Research direction
Start at the object_store LocalFileSystem rename path and reproduce the failure on an Azure blobfuse mount with a source path containing #. Compare the behavior with the related arrow-rs pull request 5094 and inspect any existing LocalFileSystem tests. Done means the rename handles such paths correctly or reports a specific, actionable error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, linux, rust
- Domain
- backend, cloud, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100