AWS: client credentials provider NPE
- 主要言語
- Java
- スター
- 9.2k
- フォーク
- 3.5k
- 平均マージ
- 2日 11時間
- マージ済み PR(30日)
- 132
説明
### Apache Iceberg version
1.11.0 (latest release)
### Query engine
None
### Please describe the bug 🐞
As of 1.11.0 (https://github.com/apache/iceberg/pull/15249), `AwsClientProperties#credentialsProvider` unconditionally does a `clientCredentialsProviderProperties.put(VendedCredentialsProvider.URI, refreshCredentialsEndpoint)` if "client.credentials-provider" is set; this may result in `null` being put into the properties map. Other similar code paths check first to make sure the value is not `null` or empty before putting the value.
This does not play well with custom credentials providers / `org.apache.iceberg.aws.AwsClientProperties`:
```
java.lang.NullPointerException
at java.base/java.util.Objects.requireNonNull(Objects.java:233)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1858)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at org.apache.iceberg.util.PropertyUtil.filterProperties(PropertyUtil.java:191)
at org.apache.iceberg.aws.AwsClientProperties.(AwsClientProperties.java:116)
at io.deephaven.iceberg.util.DeephavenS3ClientCredentialsProvider.create(DeephavenS3ClientCredentialsProvider.java:29)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.apache.iceberg.common.DynMethods$UnboundMethod.invokeChecked(DynMethods.java:60)
at org.apache.iceberg.common.DynMethods$UnboundMethod.invoke(DynMethods.java:73)
at org.apache.iceberg.common.DynMethods$StaticMethod.invoke(DynMethods.java:186)
at org.apache.iceberg.aws.AwsClientProperties.createCredentialsProvider(AwsClientProperties.java:314)
at org.apache.iceberg.aws.AwsClientProperties.credentialsProvider(AwsClientProperties.java:296)
at org.apache.iceberg.aws.AwsClientProperties.credentialsProvider(AwsClientProperties.java:218)
at org.apache.iceberg.aws.s3.S3FileIOProperties.getCredentialsProvider(S3FileIOProperties.java:996)
at org.apache.iceberg.aws.s3.S3FileIOProperties.applyCredentialConfigurations(S3FileIOProperties.java:985)
at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.lambda$s3$0(AwsClientFactories.java:116)
at software.amazon.awssdk.utils.builder.SdkBuilder.applyMutation(SdkBuilder.java:61)
at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.s3(AwsClientFactories.java:115)
at io.deephaven.iceberg.util.DeephavenAwsClientFactory.s3(DeephavenAwsClientFactory.java:49)
```
I'm unclear the exact semantics of nulls in property maps, and if this should be considered an upstream issue (ie, `null` should not be put in the map), or a downstream issue (`AwsClientProperties` should be tolerant of entries that have `null` values).
### Willingness to contribute
- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
コントリビューションガイド
調査の方向性
org.apache.iceberg.aws.AwsClientProperties#credentialsProvider から始め、PropertyUtil.filterProperties が呼び出される前の clientCredentialsProviderProperties の処理を確認します。refresh エンドポイントがないカスタム credentials provider のケースを再現し、その後、報告された NullPointerException なしで AWS クライアントを作成できることを確認し、関連する AWS プロパティのテストにカバレッジを追加します。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- aws, java
- 領域
- authentication, cloud
- issue の種類
- バグ
- 難易度
- 2/5
- 見積もり時間
- 1〜3時間
- 活発さ
- 活発
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 75/100