apache / apache/gravitino

[Bug report] Web identity credentials are not applied to server-side Iceberg S3FileIO

Open
#11,850 1 comment 0 reactions 1 assignee Claimed by @yuqi1129 View on GitHub
2.0.0 bug
Dominant language
Java
Stars
3.2k
Forks
935
Avg merge
1d 16h
Merged PRs (30d)
298

Description

### Version

main branch

### Describe what's wrong

The `aws-irsa` credential provider can read a WebIdentity token through `FileWebIdentityTokenSource`, but the server-side Iceberg paths do not consistently apply the vended storage credentials to Iceberg backend/FileIO configuration.

As a result, Iceberg `S3FileIO` may fall back to the AWS default credential chain when creating/loading table metadata, and the workflow only succeeds if the Gravitino process is also given `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and `AWS_SESSION_TOKEN` manually.

The expected behavior is that `credential-providers=aws-irsa` plus `s3-web-identity-token-source=file` is enough for Gravitino to fetch temporary credentials and use them in:
- Iceberg REST server backend FileIO
- Lakehouse Iceberg catalog backend FileIO
- Fileset credential vending
- GVFS access through fileset credentials

### Error message and/or stacktrace

`S3FileIO` can fail with:

`Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(...)`

The failing chain includes system properties, environment variables, AWS web identity token file env, profile, container credentials, and instance profile. This indicates the vended `AwsIrsaCredential` was not injected into the server-side Iceberg FileIO config.

### How to reproduce

1. Configure an S3/Iceberg catalog with:
- `credential-providers=aws-irsa`
- `s3-web-identity-token-source=file`
- `s3-web-identity-token-file=`
- `s3-role-arn=`
- `io-impl=org.apache.iceberg.aws.s3.S3FileIO`
- `warehouse=s3:///`
2. Do not set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, or `AWS_SESSION_TOKEN`.
3. Create/load an Iceberg table through Gravitino/Iceberg REST.
4. The server-side FileIO may use AWS default credentials instead of the vended `AwsIrsaCredential`.

### Additional context

The existing file-based WebIdentity token source has unit coverage, but it should also have end-to-end coverage proving the whole chain works through Iceberg, Fileset, and GVFS.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.