Feature request: explicit “no vended credentials / no SAS” mode for IAM‑only setups (esp. Azure)
- Dominant language
- Java
- Stars
- 2.1k
- Forks
- 522
- Avg merge
- 2d 16m
- Merged PRs (30d)
- 139
Description
### Is your feature request related to a problem? Please describe.
Hi All,
after PR [#4284](https://github.com/apache/polaris/pull/4284) and the related dev thread, `SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION` is now clearly positioned as **test/dev‑only**, and Polaris emits a WARN when it’s used. That change makes sense to me: forwarding the server’s ambient credentials to clients is not something I’d want to rely on in production.
However, the underlying need that some people were trying to solve with that flag still exists:
- There are environments (especially on Azure) where security and compliance rules **forbid SAS and other vended storage credentials** on production storage accounts.
- In those environments, all access to storage must go through the cloud provider’s **native identity + IAM/RBAC** model (for example, Managed Identities + Azure RBAC for ADLS Gen2).
- At the same time, Polaris is still wanted as the **Iceberg REST catalog and authorization layer**, but it must **not mint SAS** or similar storage credentials for clients.
Right now, there is no dedicated, explicit way in Polaris to say:
“Clients will always use their own cloud identity for storage; Polaris should never vend storage credentials.”
### Describe the solution you'd like
I’d like a **separate, clearly named configuration option** that enables a “no vended credentials / client‑managed credentials only” mode. Roughly:
1. **No credential vending**
- Polaris does **not** generate or return SAS (or equivalent storage credentials) to clients in this mode.
- Clients are expected to obtain their own storage credentials directly from the cloud provider (Managed Identity, service principal, workload identity, etc.).
2. **Storage authorization via IAM/RBAC only**
- Storage access is governed entirely by the cloud IAM/RBAC layer (e.g. Azure RBAC).
- Polaris controls:
- catalog‑level permissions (who can see/alter which tables), and
- Iceberg commits / metadata.
- It is clearly documented that Polaris cannot further restrict or time‑limit a client’s storage rights once those are granted in IAM/RBAC.
3. **Clear naming and documentation**
- The option name should make it obvious that this is a deliberate “no vended credentials / clients bring their own storage credentials” mode.
- Documentation should describe:
- when this mode is appropriate (e.g. environments where SAS is banned),
- the trade‑offs (less defense in depth at the storage layer, stronger reliance on IAM/RBAC),
- typical usage patterns (e.g. Polaris as catalog + Azure Managed Identity + ADLS Gen2).
4. **Guardrails (optional but helpful)**
- A production readiness WARN when this mode is enabled, explaining:
- that credential vending is disabled by design,
- that storage security is now purely governed by IAM/RBAC,
- with a link to the docs.
### Describe alternatives you've considered
- **Leaning on `SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION` anyway**
This was the most obvious “there used to be a flag for this” thought. But the dev discussion and PR [#4284](https://github.com/apache/polaris/pull/4284) are pretty clear why this isn’t acceptable for production:
- it forwards the server’s ambient credentials to clients,
- it undermines defense‑in‑depth,
- and it’s now explicitly documented and warned as test/dev‑only.
So I’ve ruled this out as a serious option.
- **Putting a custom proxy/sidecar in front of Polaris**
One can imagine a proxy in front of Polaris that strips any vended credentials from responses and forces all clients to use their own storage identity. This would technically work, but it adds another component to operate, secure and debug, and each user would end up reinventing a similar pattern instead of relying on a supported configuration in Polaris. (Just an idea probably bad)
### Additional context
To make the problem concrete, here is a typical cloud setup this is trying to cover (Azure as the main example):
- Storage accounts are configured with `allowSharedKeyAccess=false`, and internal policy explicitly **bans SAS** on those accounts.
- All access to ADLS Gen2 must use the cloud provider’s native identity and authorization:
- on Azure, that means **Managed Identity / workload identity or OAuth2 service principals (App Registrations + OAuth2 tokens)**, combined with **Azure RBAC**,
- shared keys and SAS are not allowed as a design choice.
- Engines like **PyIceberg**, **DuckDB**, **Spark** etc. can already talk to ADLS directly using these identities:
- via Managed Identity / workload identity,
- or via OAuth2 client credentials / service principals,
- with access controlled purely by Azure RBAC.
- Polaris is used as the central **Iceberg REST catalog and RBAC layer**:
- it authorizes who can create/alter/read tables in the catalog,
- it coordinates Iceberg commits and metadata.
- But Polaris must **never mint SAS** or other storage credentials for those clients, because that would violate the platform’s security and compliance rules.
Today, this ends up being approximated by convention (“clients promise to use their own identity and ignore any vended credentials”) or by custom proxies/wrappers. A built‑in, documented **“no vended credentials / IAM‑ or OAuth2‑only”** mode in Polaris would make this setup explicit and easier to reason about:
- Polaris controls catalog behavior and Iceberg commits,
- the cloud IAM/OAuth2 + RBAC layer controls storage,
- and there is **no SAS** or similar vended storage credential in the middle by design.
Contributor guide
Research direction
Start by reading the existing SKIP_CREDENTIAL_SUBSCOPING_INDIRECTION handling and the related changes in PR #4284. Trace how Polaris currently vends storage credentials, then define the configuration, guardrails, and documentation needed for an explicit client-managed-credentials mode; done means no SAS or equivalent credentials are returned and the IAM/RBAC trade-offs are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, java
- Domain
- backend-api-design, cloud, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100