aws / aws/aws-sdk-java-v2

Blocking call while resolving credentials inside an EKS cluster using DynamoDB async client (Blockhound)

Aperta
#2,360 2 commenti 1 reazione 0 assegnatari Vedi su GitHub
bug p2
Lingua principale
Java
Stelle
2.6k
Fork
1k
Merge medio
2g 9h
PR unite (30g)
51

Descrizione

## Describe the bug
While testing a spring boot webflux app inside a EKS cluster I've detected a blocking call within AWS libraries thanks to [BlockHound](https://github.com/reactor/BlockHound).

It happens while resolving the credentials right before the dynamodb call using the async client. This happens even when I try to provide a custom aws credentials provider:

```
@Bean
fun awsCredentialsProviderChain(): AwsCredentialsProviderChain {
return AwsCredentialsProviderChain.of(
ContainerCredentialsProvider.builder().asyncCredentialUpdateEnabled(true)
.build(),
InstanceProfileCredentialsProvider.builder()
.asyncCredentialUpdateEnabled(true)
.build()
)
}
```

My dynamodb client:
```

@Bean
fun dynamoDbAsyncClient(awsCredentialsProviderChain: AwsCredentialsProviderChain): DynamoDbAsyncClient =
DynamoDbAsyncClient
.builder().credentialsProvider(awsCredentialsProviderChain).build()
```

Locally everything works fine, I think the issue lies inside the _InstanceProfileCredentialsProvider_, the other providers seem to be working fine.

## Expected Behavior
Dynamodb aync client should be non blocking so I'd expect to use it without any blocking cals

## Current Behavior

Blockhounds detects a blocking call.
Stacktrace:

`reactor.blockhound.BlockingOperationError: Blocking call! java.net.SocketOutputStream#socketWrite0
at java.base/java.net.SocketOutputStream.socketWrite0(Unknown Source)
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
|_ checkpoint ⇢ com.mypackage.MyFilter [DefaultWebFilterChain]
|_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
|_ checkpoint ⇢ HTTP GET "/v1/trolleys/1001781417" [ExceptionHandlingWebHandler]
Stack trace:
at java.base/java.net.SocketOutputStream.socketWrite0(Unknown Source)
at java.base/java.net.SocketOutputStream.socketWrite(Unknown Source)
at java.base/java.net.SocketOutputStream.write(Unknown Source)
at java.base/java.io.BufferedOutputStream.flushBuffer(Unknown Source)
at java.base/java.io.BufferedOutputStream.flush(Unknown Source)
at java.base/java.io.PrintStream.flush(Unknown Source)
at java.base/sun.net.www.MessageHeader.print(Unknown Source)
at java.base/sun.net.www.http.HttpClient.writeRequests(Unknown Source)
at java.base/sun.net.www.http.HttpClient.writeRequests(Unknown Source)
at java.base/sun.net.www.protocol.http.HttpURLConnection.writeRequests(Unknown Source)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
at java.base/java.net.HttpURLConnection.getResponseCode(Unknown Source)
at software.amazon.awssdk.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:114)
at software.amazon.awssdk.regions.internal.util.EC2MetadataUtils.getToken(EC2MetadataUtils.java:442)
at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.getToken(InstanceProfileCredentialsProvider.java:83)
at software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider.getCredentialsEndpointProvider(InstanceProfileCredentialsProvider.java:69)
at software.amazon.awssdk.auth.credentials.HttpCredentialsProvider.refreshCredentials(HttpCredentialsProvider.java:74)
at software.amazon.awssdk.utils.cache.CachedSupplier.refreshCache(CachedSupplier.java:132)
at software.amazon.awssdk.utils.cache.CachedSupplier.get(CachedSupplier.java:89)
at java.base/java.util.Optional.map(Unknown Source)
at software.amazon.awssdk.auth.credentials.HttpCredentialsProvider.resolveCredentials(HttpCredentialsProvider.java:146)
at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:91)
at software.amazon.awssdk.awscore.client.handler.AwsClientHandlerUtils.createExecutionContext(AwsClientHandlerUtils.java:79)
at software.amazon.awssdk.awscore.client.handler.AwsAsyncClientHandler.createExecutionContext(AwsAsyncClientHandler.java:65)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.lambda$execute$1(BaseAsyncClientHandler.java:78)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.measureApiCallSuccess(BaseAsyncClientHandler.java:276)
at software.amazon.awssdk.core.internal.handler.BaseAsyncClientHandler.execute(BaseAsyncClientHandler.java:75)
at software.amazon.awssdk.awscore.client.handler.AwsAsyncClientHandler.execute(AwsAsyncClientHandler.java:52)
at software.amazon.awssdk.services.dynamodb.DefaultDynamoDbAsyncClient.getItem(DefaultDynamoDbAsyncClient.java:3256)
at software.amazon.awssdk.enhanced.dynamodb.internal.operations.CommonOperation.executeAsync(CommonOperation.java:140)
at software.amazon.awssdk.enhanced.dynamodb.internal.operations.TableOperation.executeOnPrimaryIndexAsync(TableOperation.java:81)
at software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbAsyncTable.getItem(DefaultDynamoDbAsyncTable.java:136)
at software.amazon.awssdk.enhanced.dynamodb.internal.client.DefaultDynamoDbAsyncTable.getItem(DefaultDynamoDbAsyncTable.java:143)`

## Steps to Reproduce
Run a spring boot webflux app with a dynamodb async client with blockound installed inside an EKS cluster. Using the client will throw an exception

## Context
Not sure if this is going to have any impact on the service perfomance. The main point here is that this issue prevents the client of being full non blocking.

Meanwhile, I can ignore this blocking call on the blockhound integration configurations.

## Your Environment
* AWS Java SDK version used: 2.16.25
* JDK version used: openjdk-11-jre-ubi-minimal:11.0.10-hotspot
* Kotlin version: 1.4.31
* Spring boot version: 2.4.4

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Riproduci il problema con un client asincrono DynamoDB e BlockHound all’interno di EKS, quindi traccia la risoluzione delle credenziali da AwsCredentialsProviderChain, attraverso InstanceProfileCredentialsProvider, fino a HttpResourcesUtils.readResource. Confronta lo stack che fallisce con il percorso di ContainerCredentialsProvider; il lavoro è completato quando la risoluzione delle credenziali non esegue più l’operazione bloccante segnalata sul percorso reattivo.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
aws, java
Ambito
cloud, databases
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.