AWS: client credentials provider NPE

Apache Iceberg version

1.11.0 (latest release)

Query engine

None

Please describe the bug 🐞

As of 1.11.0 (https://github.com/apache/iceberg/pull/15249), AwsClientProperties#credentialsProvider unconditionally does a clientCredentialsProviderProperties.put(VendedCredentialsProvider.URI, refreshCredentialsEndpoint) if "client.credentials-provider" is set; this may result in null being put into the properties map. Other similar code paths check first to make sure the value is not null or empty before putting the value.

This does not play well with custom credentials providers / org.apache.iceberg.aws.AwsClientProperties:

java.lang.NullPointerException
	at java.base/java.util.Objects.requireNonNull(Objects.java:233)
	at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180)
	at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
	at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1858)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.apache.iceberg.util.PropertyUtil.filterProperties(PropertyUtil.java:191)
	at org.apache.iceberg.aws.AwsClientProperties.<init>(AwsClientProperties.java:116)
	at io.deephaven.iceberg.util.DeephavenS3ClientCredentialsProvider.create(DeephavenS3ClientCredentialsProvider.java:29)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.apache.iceberg.common.DynMethods$UnboundMethod.invokeChecked(DynMethods.java:60)
	at org.apache.iceberg.common.DynMethods$UnboundMethod.invoke(DynMethods.java:73)
	at org.apache.iceberg.common.DynMethods$StaticMethod.invoke(DynMethods.java:186)
	at org.apache.iceberg.aws.AwsClientProperties.createCredentialsProvider(AwsClientProperties.java:314)
	at org.apache.iceberg.aws.AwsClientProperties.credentialsProvider(AwsClientProperties.java:296)
	at org.apache.iceberg.aws.AwsClientProperties.credentialsProvider(AwsClientProperties.java:218)
	at org.apache.iceberg.aws.s3.S3FileIOProperties.getCredentialsProvider(S3FileIOProperties.java:996)
	at org.apache.iceberg.aws.s3.S3FileIOProperties.applyCredentialConfigurations(S3FileIOProperties.java:985)
	at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.lambda$s3$0(AwsClientFactories.java:116)
	at software.amazon.awssdk.utils.builder.SdkBuilder.applyMutation(SdkBuilder.java:61)
	at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.s3(AwsClientFactories.java:115)
	at io.deephaven.iceberg.util.DeephavenAwsClientFactory.s3(DeephavenAwsClientFactory.java:49)

I'm unclear the exact semantics of nulls in property maps, and if this should be considered an upstream issue (ie, null should not be put in the map), or a downstream issue (AwsClientProperties should be tolerant of entries that have null values).

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

Bắt đầu tại org.apache.iceberg.aws.AwsClientProperties#credentialsProvider và kiểm tra cách xử lý clientCredentialsProviderProperties trước khi gọi PropertyUtil.filterProperties. Tái hiện trường hợp credentials provider tùy chỉnh không có endpoint refresh, sau đó xác minh rằng AWS client có thể được tạo mà không gặp NullPointerException đã được báo cáo, đồng thời bổ sung coverage trong các bài kiểm thử AWS properties liên quan.