Credential vending is AWS-only — GCS and Azure tables can't be read
- Dominant language
- C++
- Stars
- 110
- Forks
- 44
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 4
Description
## Summary
The extension only handles AWS credentials, so Unity Catalog tables backed by GCS or Azure can't be read. `ATTACH … (TYPE unity_catalog)` succeeds and `SHOW ALL TABLES` works — both are metadata-only — but `SELECT` fails once credentials are vended, because the vended credential is dropped and an S3 secret is created regardless of where the table actually lives.
## Detail
Unity Catalog vends a different credential shape per cloud (per the UC API spec, `TemporaryCredentials`):
| cloud | field | kind |
| --- | --- | --- |
| AWS | `aws_temp_credentials` | key + secret + session token |
| Azure | `azure_user_delegation_sas` | SAS token |
| GCP | `gcp_oauth_token` | OAuth bearer token |
The delta/v1 credentials endpoint uses a parallel shape: `s3.access-key-id` … for AWS, `gcs.oauth-token` for GCP.
Four touchpoints assume AWS:
| file:line | what |
| --- | --- |
| `src/uc_api.cpp:409` | reads only `aws_temp_credentials` |
| `src/uc_api.cpp:441` | reads only `s3.access-key-id` and siblings |
| `src/storage/uc_table_set.cpp:214` | `input.type = "s3"`, unconditionally |
| `src/storage/uc_table_entry.cpp:206` | `sec.type = "s3"`, unconditionally (scan-plan path) |
The `ATTACH` surface is AWS-shaped too — `AWS_REGION` is the only cloud parameter the `unity_catalog` secret accepts.
## Suggested change
Branch on the table's `storage_location` scheme and create a matching secret:
- `gs://` → `gcs` secret carrying `bearer_token` from `gcp_oauth_token.oauth_token` (or `gcs.oauth-token` on the managed path)
- `abfss://`/`az://` → `azure` secret from `azure_user_delegation_sas.sasToken`
- `s3://` → unchanged
Everything downstream is already cloud-agnostic: httpfs handles GCS and duckdb-azure handles Azure, and duckdb-delta has working branches for both.
## Dependency
For GCS the Delta *log* read additionally needs duckdb/duckdb-delta#356 — the bearer token can't currently reach delta-kernel-rs's object store. The data-file half already works through httpfs. Azure has no such blocker: object_store 0.13.2 already exposes a SAS config key.
I have the GCS half working end-to-end against a live UC workspace and am happy to contribute this change too, once the duckdb-delta side has a direction.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.