developmentseed / developmentseed/obstore

LocalStore file handling of empty paths is inconsistent with S3Store, GCSStore, etc.

Open
#635 1 comment 0 reactions 0 assignees View on GitHub
upstream
Dominant language
Python
Stars
810
Forks
42
Avg merge
1d 15h
Merged PRs (30d)
6

Description

When using an http-based store or s3store with a prefix that points directly to an object/file, calling get("")
will retrieve the object, i.e.

```py
store = obstore.store.from_url("s3://my-bucket/bar.txt")
store.get("") # <-- returns bar.txt object
```

however for file-based paths the following does not work

```py
store = obstore.store.from_url("file:///home/foo/bar.txt")
store.get("") # <-- throws error!
```
this throws the error
```py
Generic LocalFileSystem error: Filenames containing trailing '/#\d+/' are not supported:
Debug source:
Generic {
store: "LocalFileSystem",
source: InvalidPath {
path: "",
},
}
```

Ideally LocalStore would also support "". This makes tooling for downloading a file://, s3://, gs:// url easy.

Alternatively this issue could be worked around if there were a way of constructing a store from a url, but unprefixed and returning the prefix separately, i.e. something like

```py
store, prefix = obstore.store.unprefixed_from_url("file://foo/bar.txt")
store.get(prefix) # <-- works for file:// now!
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the store.from_url and get("") entry points for file:// URLs, then compare LocalStore behavior with the S3Store and GCSStore examples in the issue. Done means a file-based URL pointing directly to an object accepts an empty path consistently, with the existing error reproduced or covered by regression verification.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, google-cloud, python, rust
Domain
backend, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.