developmentseed / developmentseed/obstore

ObjectStore to FsspecStore

Open
#525 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
810
Forks
42
Avg merge
1d 15h
Merged PRs (30d)
6

Description

Can a capability be added to convert an `ObjectStore` to an `FsspecStore`

Use: This can allow seamless usage in a code that involves fsspec only parts and non fsspec parts. For example, in my project, there are cases where I have to interact with GCS and I use the async list, list_with_delimiter etc. focusing on performance, but in other parts of the same code base i use with duckdb/ pyarrow to read data as well from same GCS as duckdb/ pyarrow both talk fsspec. So right now I create 2 separate handles and maintain them but an API like below can be very useful.

wdyt?

```
from obstore.store import ObjectStore
from obstore.fsspec import FsspecStore

# Option A
some_store: ObjectStore = ...
fs: FsspecStore = some_store.to_fsspec()

# Option B
some_store: ObjectStore = ...
fs: FsspecStore = FsspecStore.from_store(some_store)

# Option C (change the constructor of FsspecStore completely)
some_store: ObjectStore = ...
fs: FsspecStore = FsspecStore(some_store)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.