aws / aws/aws-sdk-java-v2

ConnectionUtils should be configurable wrt to read and connect timeout

Open
#2,746 1 comment 0 reactions 0 assignees View on GitHub
feature-request p3
Dominant language
Java
Stars
2.6k
Forks
1k
Avg merge
2d 9h
Merged PRs (30d)
51

Description

While using the InstanceProfileCredentialsProvider to retrieve the token (using IAM roles) to access SQS we see frequently read timeout exceptions while initializing our service (`Unable to load credentials from service endpoint`).

## Describe the issue
The InstanceProfileCredentialsProvider uses internally the ConnectionUtils class to make its http request to the metadata service. From time to time this requests fails for a couple of minutes and since the read timeout is hardcoded to 1s, this fails continuously until it eventually succeeds. I guess having the option to increase the read timeout would be beneficial in this case. There is a TODO already in this class to refactor.

## Steps to Reproduce
Intermittent while trying to connect to SQS via IAM role.

## Current Behavior
```
Caused by: java.util.concurrent.ExecutionException: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from service endpoint.
at java.util.concurrent.CompletableFuture.reportGet(Unknown Source)
at java.util.concurrent.CompletableFuture.get(Unknown Source)
at xxxxx.JobQueue.getQueueUrl(JobQueue.java:xxx)
... 39 common frames omitted
Caused by: s.a.a.core.exception.SdkClientException: Unable to load credentials from service endpoint.
at s.a.a.c.e.SdkClientException$BuilderImpl.build(SdkClientException.java:98)
at s.a.a.a.c.HttpCredentialsProvider.refreshCredentials(HttpCredentialsProvider.java:110)
at s.a.awssdk.utils.cache.CachedSupplier.refreshCache(CachedSupplier.java:132)
at s.a.awssdk.utils.cache.CachedSupplier.get(CachedSupplier.java:89)
at java.util.Optional.map(Unknown Source)
at s.a.a.a.c.HttpCredentialsProvider.resolveCredentials(HttpCredentialsProvider.java:146)
at s.a.a.a.i.AwsExecutionContextBuilder.resolveCredentials(AwsExecutionContextBuilder.java:165)
at s.a.a.a.i.AwsExecutionContextBuilder.invokeInterceptorsAndCreateExecutionContext(AwsExecutionContextBuilder.java:102)
at s.a.a.a.c.handler.AwsAsyncClientHandler.invokeInterceptorsAndCreateExecutionContext(AwsAsyncClientHandler.java:65)
at s.a.a.c.i.handler.BaseAsyncClientHandler.lambda$execute$1(BaseAsyncClientHandler.java:77)
at s.a.a.c.i.handler.BaseAsyncClientHandler.measureApiCallSuccess(BaseAsyncClientHandler.java:282)
at s.a.a.c.i.handler.BaseAsyncClientHandler.execute(BaseAsyncClientHandler.java:75)
at s.a.a.a.c.handler.AwsAsyncClientHandler.execute(AwsAsyncClientHandler.java:52)
at s.a.a.services.sqs.DefaultSqsAsyncClient.getQueueUrl(DefaultSqsAsyncClient.java:925)
... 40 common frames omitted
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Unknown Source)
at java.net.SocketInputStream.socketRead(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.net.SocketInputStream.read(Unknown Source)
at java.io.BufferedInputStream.fill(Unknown Source)
at java.io.BufferedInputStream.read1(Unknown Source)
at java.io.BufferedInputStream.read(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
at s.n.www.protocol.http.HttpURLConnection.getInputStream0(Unknown Source)
at s.n.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source:33)
at java.net.HttpURLConnection.getResponseCode(Unknown Source:33)
at s.a.a.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:116)
at s.a.a.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:91)
at s.a.a.a.c.InstanceProfileCredentialsProvider$InstanceProviderCredentialsEndpointProvider.endpoint(InstanceProfileCredentialsProvider.java:150)
at s.a.a.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:112)
at s.a.a.regions.util.HttpResourcesUtils.readResource(HttpResourcesUtils.java:91)
at s.a.a.a.c.HttpCredentialsProvider.refreshCredentials(HttpCredentialsProvider.java:79)
... 52 common frames omitted
```

## Your Environment
* AWS Java SDK version used: 2.17.34
* JDK version used: Java 11
* Operating System and version: Linux

Contributor guide

Open the contributing guide

Research direction

Start by reading the ConnectionUtils TODO and trace how InstanceProfileCredentialsProvider and HttpResourcesUtils issue requests to the metadata service. Determine where read and connect timeouts are currently fixed, then verify that callers can configure them without changing the existing credential-loading behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
authentication, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.