apache / apache/iceberg

AWS: client credentials provider NPE

Offen Anfängerfreundlich
#17,536 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
bug
Vorherrschende Sprache
Java
Sterne
9.2k
Forks
3.5k
Ø Merge
2 T. 11 Std.
Gemergte PRs (30 T.)
132

Beschreibung

### Apache Iceberg version

1.11.0 (latest release)

### Query engine

None

### Please describe the bug 🐞

As of 1.11.0 (https://github.com/apache/iceberg/pull/15249), `AwsClientProperties#credentialsProvider` unconditionally does a `clientCredentialsProviderProperties.put(VendedCredentialsProvider.URI, refreshCredentialsEndpoint)` if "client.credentials-provider" is set; this may result in `null` being put into the properties map. Other similar code paths check first to make sure the value is not `null` or empty before putting the value.

This does not play well with custom credentials providers / `org.apache.iceberg.aws.AwsClientProperties`:

```
java.lang.NullPointerException
at java.base/java.util.Objects.requireNonNull(Objects.java:233)
at java.base/java.util.stream.Collectors.lambda$uniqKeysMapAccumulator$1(Collectors.java:180)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:179)
at java.base/java.util.HashMap$EntrySpliterator.forEachRemaining(HashMap.java:1858)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at org.apache.iceberg.util.PropertyUtil.filterProperties(PropertyUtil.java:191)
at org.apache.iceberg.aws.AwsClientProperties.(AwsClientProperties.java:116)
at io.deephaven.iceberg.util.DeephavenS3ClientCredentialsProvider.create(DeephavenS3ClientCredentialsProvider.java:29)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at org.apache.iceberg.common.DynMethods$UnboundMethod.invokeChecked(DynMethods.java:60)
at org.apache.iceberg.common.DynMethods$UnboundMethod.invoke(DynMethods.java:73)
at org.apache.iceberg.common.DynMethods$StaticMethod.invoke(DynMethods.java:186)
at org.apache.iceberg.aws.AwsClientProperties.createCredentialsProvider(AwsClientProperties.java:314)
at org.apache.iceberg.aws.AwsClientProperties.credentialsProvider(AwsClientProperties.java:296)
at org.apache.iceberg.aws.AwsClientProperties.credentialsProvider(AwsClientProperties.java:218)
at org.apache.iceberg.aws.s3.S3FileIOProperties.getCredentialsProvider(S3FileIOProperties.java:996)
at org.apache.iceberg.aws.s3.S3FileIOProperties.applyCredentialConfigurations(S3FileIOProperties.java:985)
at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.lambda$s3$0(AwsClientFactories.java:116)
at software.amazon.awssdk.utils.builder.SdkBuilder.applyMutation(SdkBuilder.java:61)
at org.apache.iceberg.aws.AwsClientFactories$DefaultAwsClientFactory.s3(AwsClientFactories.java:115)
at io.deephaven.iceberg.util.DeephavenAwsClientFactory.s3(DeephavenAwsClientFactory.java:49)
```

I'm unclear the exact semantics of nulls in property maps, and if this should be considered an upstream issue (ie, `null` should not be put in the map), or a downstream issue (`AwsClientProperties` should be tolerant of entries that have `null` values).

### Willingness to contribute

- [ ] I can contribute a fix for this bug independently
- [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginnen Sie bei org.apache.iceberg.aws.AwsClientProperties#credentialsProvider und prüfen Sie die Verarbeitung von clientCredentialsProviderProperties, bevor PropertyUtil.filterProperties aufgerufen wird. Reproduzieren Sie den Fall eines benutzerdefinierten Credentials-Providers ohne Refresh-Endpunkt. Verifizieren Sie anschließend, dass der AWS-Client ohne die gemeldete NullPointerException erstellt werden kann, und ergänzen Sie Testabdeckung in den relevanten AWS-Properties-Tests.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
aws, java
Bereich
authentication, cloud
Issue-Typ
Bug
Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Aktivitätsstatus
Aktiv
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
75/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.