apache / apache/arrow-rs-object-store

LocalFileSystem rename fails with "No such file or directory" when source path contains hash fragment (#) on blobfuse mount

Open
#397 1 comment 0 reactions 0 assignees View on GitHub
question
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

## Bug Report

### What happened
When using `object_store` with `LocalFileSystem` on a blobfuse-mounted Azure storage, file rename operations fail with "No such file or directory (os error 2)" when the source file path contains a hash fragment (`#`).

### Environment
- `object_store` version: 0.12.1
- Operating System: Linux (blobfuse)
- Storage backend: Azure Blob Storage via blobfuse mount
- Related library: deltalake 1.0.1 (Python bindings)

### To Reproduce
```python
import pandas as pd
import pyarrow as pa
import tempfile
import os
from pathlib import Path

try:
df = pd.DataFrame({'x': [1, 2, 3], 'y': ['a', 'a', 'b']})
arrow_table = pa.Table.from_pandas(df)
delta_table_path = "/lakehouse/default/Tables/xy_table2"
write_deltalake(
delta_table_path,
arrow_table,
mode="overwrite",
storage_options={"mount_allow_unsafe_rename": "true"}
)
delta_table = DeltaTable(delta_table_path)
print(f"{len(delta_table.to_pandas())}")

except Exception as e:
print(f"❌ error: {e}")
```
```
Error:
❌ error: Generic LocalFileSystem error
↳ Unable to rename file
↳ No such file or directory (os error 2)
```
1. Mount Azure Blob Storage using blobfuse
2. Attempt to rename a file with a path containing `#` character
3. The rename operation fails with `LocalFileSystem` error

### Expected behavior
File rename should succeed, or provide a more descriptive error message indicating the issue with hash fragments in paths.

### Actual behavior
Generic LocalFileSystem error ↳ Unable to rename file ↳ No such file or directory (os error 2)

### Additional context
This appears to be related to how blobfuse handles file paths with hash fragments. The blobfuse logs show:
Failed request, request_id: status: 404 response: {"error":{"code":"SourcePathNotFound","message":"The source path for a rename operation does not exist."}}
The source path in the blobfuse log shows a `#1` suffix that seems to be causing the issue:
src = /path/to/file.parquet#1

### Possible solutions
1. Add validation to reject file paths with hash fragments when using `LocalFileSystem` with a clear error message
2. Escape or handle hash fragments properly in file paths
3. Provide better error messaging to indicate the specific issue with path characters

### Related
- This issue is related to Delta Lake operations on Azure storage via blobfuse mounts
- May be related to: https://github.com/apache/arrow-rs/pull/5094

This issue focuses on the core problem: the object_store LocalFileSystem doesn't handle file paths with hash fragments properly when working with blobfuse mounts, and provides insufficient error messaging to diagnose the issue.

Contributor guide

Open the contributing guide

Research direction

Start at the object_store LocalFileSystem rename path and reproduce the failure on an Azure blobfuse mount with a source path containing #. Compare the behavior with the related arrow-rs pull request 5094 and inspect any existing LocalFileSystem tests. Done means the rename handles such paths correctly or reports a specific, actionable error.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, linux, rust
Domain
backend, cloud, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.