developmentseed / developmentseed/obstore

Infer default credential providers

Open
#414 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
810
Forks
42
Avg merge
1d 15h
Merged PRs (30d)
6

Description

> I'm thinking something like this in [store.py](https://github.com/developmentseed/obstore/blob/dfaef429a05241921aabe735381211b27fd34a87/obstore/python/obstore/store.py#L709):
>
> (Not sure if all these exist credential providers exist in obstore, but assume that when called with no arguments that these return the vendor's default credential.)
>
> ``` python
> def from_url(...):
> ...
> scheme = _parse_scheme(url)
> vendor_default_credential_providers = {
> "s3": S3CredentialProvider,
> "gcs": GoogleCredentialProvider,
> "azure": AzureCredentialProvider,
> }
>
> vendor_default_async_credential_providers = {
> "s3": S3AsyncCredentialProvider,
> "gcs": GoogleAsyncCredentialProvider,
> "azure": AzureAsyncCredentialProvider,
> }
>
> if credential_provider is None and vendor_default_credential_provider:
> if credential_provider := vendor_default_credential_providers.get(scheme):
> credential_provider = credential_provider()
> elif credential_provider is None and vendor_default_async_credential_provider:
> if credential_provider := vendor_default_async_credential_providers.get(scheme):
> credential_provider = credential_provider()
>
> if scheme == "s3":
> ...
> ```
>
> Implications:
>
> - Existing behaviour of `from_url` is unchanged when new keywords are omitted.
> - Two new keywords added to `from_url`: `vendor_default_credential_provider` and `vendor_default_async_credential_provider`. These allow the user to opt into using an automatically selected vendor default credential provider, and allow the user to specify whether they want it to be synchronous or asynchronous.
> - If one of the new keywords is specified, and `credential_provider` is not explicitly specified, then the appropriate vendor default credential provider will be used.
> - There is a one to one mapping from scheme to default vendor credential providers. Users must still explicitly specify if they need credential providers other than the vendor default credential provider.
>

_Originally posted by @daviewales in [#267](https://github.com/developmentseed/obstore/issues/267#issuecomment-2773995765)_

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in obstore/python/obstore/store.py at from_url and inspect _parse_scheme plus the available synchronous and asynchronous credential providers. Check how the existing S3, GCS, and Azure paths handle an omitted credential_provider. Done means the two opt-in keywords select the matching vendor provider without changing behavior when omitted, while explicit providers continue to take precedence.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, azure, gcp, python
Domain
backend, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.