developmentseed / developmentseed/obstore

Streaming copy example with aiohttp

Open
#399 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
810
Forks
42
Avg merge
1d 15h
Merged PRs (30d)
6

Description

Add example to docs of passing streaming download from aiohttp into streaming multipart upload

Ref https://github.com/developmentseed/obstore/issues/398#issuecomment-2766564086

```py
from aiohttp import ClientSession
from boto3.session import Session

from obstore.auth.boto3 import Boto3CredentialProvider
from obstore.store import S3Store

s = Session(profile_name="ds")
cp = Boto3CredentialProvider(Session(profile_name="ds"))
store = S3Store("bucket", credential_provider=cp, region="us-east-1")

url = "https://storage.googleapis.com/open-geodata/linz-examples/nz-buildings-outlines.parquet"

async with ClientSession() as session, session.get(url) as resp:
await store.put_async("test_upload.parquet", resp.content.iter_any())
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Locate the documentation examples and start from the async upload entry point, store.put_async, alongside aiohttp's resp.content.iter_any(). Add the requested streaming download-to-multipart-upload example using the shown setup, and consider the work done when the example is present and readable in the docs.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.