apache / apache/iceberg

AWS: client credentials provider NPE

Ouverte Adaptée aux débutants
#17,536 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
bug
Langage dominant
Java
Étoiles
9.2k
Forks
3.5k
Merge moyen
2 j 16 h
PR mergées (30 j)
129

Description

### Apache Iceberg version

1.11.0 (latest release)

### Query engine

None

### Please describe the bug 🐞

As of 1.11.0 (https://github.com/apache/iceberg/pull/15249), `AwsClientProperties#credentialsProvider` unconditionally does a `clientCredentialsProviderProperties.put(VendedCredentialsProvider.URI, refreshCredentialsEndpoint)` if "client.credentials-provider" is set; this may result in `null` being put into the properties map. Other similar code paths check first to make sure the value is not `null` or empty before putting the value.

This does not play well with custom credentials providers / `org.apache.iceberg.aws.AwsClientProperties`:

```
java.lang.NullPointerException
at java.base/java.util.Objects.requireNonNull(Objects.java:233)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1858)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at org.apache.iceberg.util.PropertyUtil.filterProperties(PropertyUtil.java:191)
at org.apache.iceberg.aws.AwsClientProperties.(AwsClientProperties.java:116)
at io.deephaven.iceberg.util.DeephavenS3ClientCredentialsProvider.create(DeephavenS3ClientCredentialsProvider.java:29)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.apache.iceberg.common.DynMethods$UnboundMethod.invokeChecked(DynMethods.java:60)
at org.apache.iceberg.common.DynMethods$UnboundMethod.invoke(DynMethods.java:73)
at org.apache.iceberg.common.DynMethods$StaticMethod.invoke(DynMethods.java:186)
at org.apache.iceberg.aws.AwsClientProperties.createCredentialsProvider(AwsClientProperties.java:314)
at org.apache.iceberg.aws.AwsClientProperties.credentialsProvider(AwsClientProperties.java:296)
at org.apache.iceberg.aws.AwsClientProperties.credentialsProvider(AwsClientProperties.java:218)
at org.apache.iceberg.aws.s3.S3FileIOProperties.getCredentialsProvider(S3FileIOProperties.java:996)
at org.apache.iceberg.aws.s3.S3FileIOProperties.applyCredentialConfigurations(S3FileIOProperties.java:985)
at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.lambda$s3$0(AwsClientFactories.java:116)
at software.amazon.awssdk.utils.builder.SdkBuilder.applyMutation(SdkBuilder.java:61)
at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.s3(AwsClientFactories.java:115)
at io.deephaven.iceberg.util.DeephavenAwsClientFactory.s3(DeephavenAwsClientFactory.java:49)
```

I'm unclear the exact semantics of nulls in property maps, and if this should be considered an upstream issue (ie, `null` should not be put in the map), or a downstream issue (`AwsClientProperties` should be tolerant of entries that have `null` values).

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez dans org.apache.iceberg.aws.AwsClientProperties#credentialsProvider et examinez le traitement de clientCredentialsProviderProperties avant l'appel à PropertyUtil.filterProperties. Reproduisez le cas d'un fournisseur d'identifiants personnalisé sans endpoint de refresh, puis vérifiez que le client AWS peut être créé sans la NullPointerException signalée et ajoutez une couverture de tests dans les tests pertinents des propriétés AWS.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
aws, java
Domaine
authentication, cloud
Type d'issue
Bug
Difficulté
2/5
Temps estimé
1-3 heures
Activité
Active
Clarté
Plutôt claire
Accessibilité débutants
75/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.