"Unable to load credentials from any of the providers" for parallel calls to an S3Client with ProfileCredentialsProvider
- Dominant language
- Java
- Stars
- 2.6k
- Forks
- 1k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 51
Description
### Describe the bug
I am still encountering the behavior from this closed issue https://github.com/aws/aws-sdk-java-v2/issues/5001.
In my case I have a list of S3-Objects that exist in a bucket and download & process them in a parallel stream.
It sometimes manages some objects, but **always** fails with `Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain...` (see stacktrace) eventually. The number of elements doesn't seem to have an impact.
In my local testing the same program worked flawlessly when either the stream was sequential instead of parallel or the S3Client has already executed another call before the parallel stream starts.
This makes the analysis from that previous issue seem likely, that there's a race condition for `resolveCredentials` in ProfileCredentialProvider.
### Regression Issue
- [ ] Select this option if this issue appears to be a regression.
### Expected Behavior
The parallel stream should be processed normally.
### Current Behavior
> Exception in thread "main" software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(profileName=_**my profile name**_, profileFile=ProfileFile(sections=[profiles, sso-session, services], profiles=[Profile(name=default, properties=[region, output, ca_bundle]), Profile(name=_**my profile name**_, properties=[role_arn, region, source_profile])])), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(profileName=_**my profile name**_): Cannot invoke "software.amazon.awssdk.profiles.ProfileFile.profile(String)" because "profileFile" is null, ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]
at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:130)
at software.amazon.awssdk.auth.credentials.AwsCredentialsProviderChain.resolveCredentials(AwsCredentialsProviderChain.java:130)
at software.amazon.awssdk.auth.credentials.internal.LazyAwsCredentialsProvider.resolveCredentials(LazyAwsCredentialsProvider.java:45)
at software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider.resolveCredentials(DefaultCredentialsProvider.java:134)
at software.amazon.awssdk.auth.credentials.AwsCredentialsProvider.resolveIdentity(AwsCredentialsProvider.java:54)
at software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor.lambda$trySelectAuthScheme$6(S3AuthSchemeInterceptor.java:181)
at software.amazon.awssdk.core.internal.util.MetricUtils.reportDuration(MetricUtils.java:81)
at software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor.trySelectAuthScheme(S3AuthSchemeInterceptor.java:181)
at software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor.selectAuthScheme(S3AuthSchemeInterceptor.java:99)
at software.amazon.awssdk.services.s3.auth.scheme.internal.S3AuthSchemeInterceptor.beforeExecution(S3AuthSchemeInterceptor.java:71)
at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.lambda$beforeExecution$1(ExecutionInterceptorChain.java:59)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1604)
at software.amazon.awssdk.core.interceptor.ExecutionInterceptorChain.beforeExecution(ExecutionInterceptorChain.java:59)
at software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.runInitialInterceptors(AwsExecutionContextBuilder.java:319)
at software.amazon.awssdk.awscore.internal.AwsExecutionContextBuilder.invokeInterceptorsAndCreateExecutionContext(AwsExecutionContextBuilder.java:155)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.invokeInterceptorsAndCreateExecutionContext(AwsSyncClientHandler.java:67)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.lambda$execute$0(BaseSyncClientHandler.java:62)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.measureApiCallSuccess(BaseSyncClientHandler.java:182)
at software.amazon.awssdk.core.internal.handler.BaseSyncClientHandler.execute(BaseSyncClientHandler.java:60)
at software.amazon.awssdk.core.client.handler.SdkSyncClientHandler.execute(SdkSyncClientHandler.java:52)
at software.amazon.awssdk.awscore.client.handler.AwsSyncClientHandler.execute(AwsSyncClientHandler.java:60)
at software.amazon.awssdk.services.s3.DefaultS3Client.getObject(DefaultS3Client.java:6837)
at software.amazon.awssdk.services.s3.S3Client.getObject(S3Client.java:11977)
at software.amazon.awssdk.services.s3.S3Client.getObject(S3Client.java:12243)
at Test.lambda$main$1(Test.java:89)
### Reproduction Steps
A simple version of my program could look like this:
```
try (S3Client s3Client = S3Client.builder().region(Region.EU_CENTRAL_1).credentialsProvider(DefaultCredentialsProvider.builder().build()).build()) {
// works if s3Client is already called here with e.g:
// s3Client.listBuckets().buckets().forEach(IO::println);
String bucketName = "...";
List s3Keys = List.of("...", "...", "...", ...);
s3Keys.parallelStream() // works if this is just .stream()
.map(s3Key -> {
try {
return new String(s3Client.getObject(b -> b.bucket(bucketName).key(s3Key)).readAllBytes());
} catch (IOException e) {
throw new RuntimeException(e);
}
})
.map(String::length)
.forEach(System.out::println);
}
```
### Possible Solution
_No response_
### Additional Information/Context
_No response_
### AWS Java SDK version used
2.42.4
### JDK version used
amazon-corretto-jdk25.0.0_36
### Operating System and version
Windows 11
Contributor guide
Assessment
This issue has not been assessed yet.