developmentseed / developmentseed/obstore
LocalStore file handling of empty paths is inconsistent with S3Store, GCSStore, etc.
- Dominant language
- Python
- Stars
- 810
- Forks
- 42
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 6
Description
When using an http-based store or s3store with a prefix that points directly to an object/file, calling get("")
will retrieve the object, i.e.
```py
store = obstore.store.from_url("s3://my-bucket/bar.txt")
store.get("") # <-- returns bar.txt object
```
however for file-based paths the following does not work
```py
store = obstore.store.from_url("file:///home/foo/bar.txt")
store.get("") # <-- throws error!
```
this throws the error
```py
Generic LocalFileSystem error: Filenames containing trailing '/#\d+/' are not supported:
Debug source:
Generic {
store: "LocalFileSystem",
source: InvalidPath {
path: "",
},
}
```
Ideally LocalStore would also support "". This makes tooling for downloading a file://, s3://, gs:// url easy.
Alternatively this issue could be worked around if there were a way of constructing a store from a url, but unprefixed and returning the prefix separately, i.e. something like
```py
store, prefix = obstore.store.unprefixed_from_url("file://foo/bar.txt")
store.get(prefix) # <-- works for file:// now!
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by tracing the store.from_url and get("") entry points for file:// URLs, then compare LocalStore behavior with the S3Store and GCSStore examples in the issue. Done means a file-based URL pointing directly to an object accepts an empty path consistently, with the existing error reproduced or covered by regression verification.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, google-cloud, python, rust
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100