apache / apache/arrow-rs-object-store

[ObjectStore] MultipartUpload::complete fails on WSL when writing to /mnt/

Open
#275 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

**Describe the bug**
Using LocalFileSystem on WSL + BufWriter I get the following error:
`file.metadata()` fails after `std::fs::rename` with FileNotFound but is reported as an aborted upload

**To Reproduce**
Use `object_store::buffered::BufWriter` to write a big enough file to the windows filesystem when running though WSL.

**Expected behavior**
File write succeeds.

**Additional context**
Using object_store-0.11.0.
Our services are actually writing to S3, but the local dev environment writes to the local filesystem. I could not reproduce the bug when writing to the linux filesystem, but it happens every time I write a 50MB+ file to the windows file system with the exe running on the linux side.

Contributor guide

Open the contributing guide

Research direction

Start at the LocalFileSystem multipart upload completion path used by object_store::buffered::BufWriter, focusing on the std::fs::rename followed by file.metadata() behavior on WSL /mnt paths. Reproduce with a file larger than 50 MB, then verify that completion succeeds and the file can be written to the Windows filesystem without being reported as an aborted upload.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, operating-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.