apache / apache/arrow-rs-object-store

Difficulty working with Windows paths when using url > 2.5.4

Open
#499 1 comment 2 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

**Describe the bug**
If I am on Windows, and using the 2.5.7 version of the `url` package, then I encounter errors due to a behavior change in the `url` package. I have filed an upstream issue (https://github.com/servo/rust-url/issues/1077) but would like to open this issue in case there are recommendations for a different approach or the upstream crate suggests a different approach.

**To Reproduce**

Here is a test that reproduces the issue. It only relies on the `tempfile` and `tokio` crates.

```
#[tokio::test]
async fn test_obj_store() {
use std::io::Write;

let tmpdir = tempfile::tempdir().unwrap();
let tmp_file = tmpdir.path().join("test.file");

let mut rust_file = std::fs::File::create(tmp_file.clone()).unwrap();
write!(rust_file, "test").unwrap();
drop(rust_file);

let tmp_path = tmp_file.to_str().unwrap();
// File parses ok
let obj_path = Path::parse(tmp_path).unwrap();

// This fails if url is at version 2.5.7 and passes if url is at version 2.5.4
LocalFileSystem::new().get_opts(&obj_path, GetOptions {
head: true,
..Default::default()
}).await.unwrap();
}
```

**Expected behavior**
The above test should pass

**Additional context**
This was encountered because datafusion version 50 forced an upgrade to 2.5.7 of the url package: https://github.com/apache/datafusion/blob/10343c18292fc3a9ca62b59c8bb530169fe79b82/Cargo.toml#L181

Contributor guide

Open the contributing guide

Research direction

Run the provided async test on Windows with the current url dependency and compare behavior with url 2.5.4 and 2.5.7. Read Path::parse and LocalFileSystem::new().get_opts, then review upstream rust-url issue 1077; done means the reproduction passes without regressing local filesystem access.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.