aws / aws/aws-sdk-java-v2

"Unable to load credentials from any of the providers" for parallel calls to an S3Client with ProfileCredentialsProvider

Aperta
#6,827 2 commenti 0 reazioni 1 assegnatario Rivendicata da @debora-ito Vedi su GitHub
bug p2
Lingua principale
Java
Stelle
2.6k
Fork
1k
Merge medio
2g 9h
PR unite (30g)
51

Descrizione

### Describe the bug

I am still encountering the behavior from this closed issue https://github.com/aws/aws-sdk-java-v2/issues/5001.

In my case I have a list of S3-Objects that exist in a bucket and download & process them in a parallel stream.

It sometimes manages some objects, but **always** fails with `Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain...` (see stacktrace) eventually. The number of elements doesn't seem to have an impact.

In my local testing the same program worked flawlessly when either the stream was sequential instead of parallel or the S3Client has already executed another call before the parallel stream starts.

This makes the analysis from that previous issue seem likely, that there's a race condition for `resolveCredentials` in ProfileCredentialProvider.

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Expected Behavior

The parallel stream should be processed normally.

### Current Behavior

> Exception in thread "main" software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=_**my profile name**_, profileFile=ProfileFile(sections=[profiles, sso-session, services], profiles=[Profile(name=default, properties=[region, output, ca_bundle]), Profile(name=_**my profile name**_, properties=[role_arn, region, source_profile])])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=_**my profile name**_): Cannot invoke "software.amazon.awssdk.profiles.ProfileFile.profile(String)" because "profileFile" is null, ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]
at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:130)
at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:130)
at software.amazon.awssdk.auth.credentials.internal.LazyAwsCredentialsProvider.resolveCredentials(LazyAwsCredentialsProvider.java:45)
at software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.resolveCredentials(DefaultCredentialsProvider.java:134)
at software.amazon.awssdk.auth.credentials.AwsCredentialsProvider.resolveIdentity(AwsCredentialsProvider.java:54)
at software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor.lambda$trySelectAuthScheme$6(S3AuthSchemeInterceptor.java:181)
at software.amazon.awssdk.core.internal.util.MetricUtils.reportDuration(MetricUtils.java:81)
at software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor.trySelectAuthScheme(S3AuthSchemeInterceptor.java:181)
at software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor.selectAuthScheme(S3AuthSchemeInterceptor.java:99)
at software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor.beforeExecution(S3AuthSchemeInterceptor.java:71)
at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.lambda$beforeExecution$1(ExecutionInterceptorChain.java:59)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.beforeExecution(ExecutionInterceptorChain.java:59)
at software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.runInitialInterceptors(AwsExecutionContextBuilder.java:319)
at software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.invokeInterceptorsAndCreateExecutionContext(AwsExecutionContextBuilder.java:155)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.invokeInterceptorsAndCreateExecutionContext(AwsSyncClientHandler.java:67)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$0(BaseSyncClientHandler.java:62)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:60)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:60)
at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:6837)
at software.amazon.awssdk.services.s3.S3Client.getObject(S3Client.java:11977)
at software.amazon.awssdk.services.s3.S3Client.getObject(S3Client.java:12243)
at Test.lambda$main$1(Test.java:89)

### Reproduction Steps

A simple version of my program could look like this:
```
try (S3Client s3Client = S3Client.builder().region(Region.EU_CENTRAL_1).credentialsProvider(DefaultCredentialsProvider.builder().build()).build()) {
// works if s3Client is already called here with e.g:
// s3Client.listBuckets().buckets().forEach(IO::println);

String bucketName = "...";
List s3Keys = List.of("...", "...", "...", ...);
s3Keys.parallelStream() // works if this is just .stream()
.map(s3Key -> {
try {
return new String(s3Client.getObject(b -> b.bucket(bucketName).key(s3Key)).readAllBytes());
} catch (IOException e) {
throw new RuntimeException(e);
}
})
.map(String::length)
.forEach(System.out::println);
}
```

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### AWS Java SDK version used

2.42.4

### JDK version used

amazon-corretto-jdk25.0.0_36

### Operating System and version

Windows 11

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.