"authentication/authorization failure" while connecting to OpenSearch Serverless using IAM role

Steps to reproduce:

Attach an IAM role with all permissions.

**aoss:CreateCollectionItems
aoss:DeleteCollectionItems
aoss:UpdateCollectionItems
aoss:DescribeCollectionItems

aoss:CreateIndex
aoss:DeleteIndex
aoss:UpdateIndex
aoss:DescribeIndex
aoss:ReadDocument
aoss:WriteDocument**

Run below Java class within a container.

import java.io.IOException;
import java.util.Optional;

import org.opensearch.client.opensearch.OpenSearchClient;
import org.opensearch.client.opensearch._types.OpenSearchException;
import org.opensearch.client.opensearch.indices.CreateIndexRequest;
import org.opensearch.client.transport.aws.AwsSdk2Transport;
import org.opensearch.client.transport.aws.AwsSdk2TransportOptions;

import software.amazon.awssdk.auth.credentials.AwsCredentialsProvider;
import software.amazon.awssdk.auth.credentials.AwsSessionCredentials;
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.auth.signer.params.Aws4SignerParams;
import software.amazon.awssdk.http.SdkHttpClient;
import software.amazon.awssdk.http.apache.ApacheHttpClient;
import software.amazon.awssdk.regions.Region;

public class IndexDocument {

    private static final String host = "<test-domain>.us-west-2.aoss.amazonaws.com";
    private static Region region = Region.US_WEST_2;

    public static void main(String[] args) {
        SdkHttpClient httpClient = ApacheHttpClient.builder().build();
        try {
        	
        //testing out the default credential chain
            final AwsCredentialsProvider credentials = Optional.ofNullable(AwsSdk2TransportOptions.builder().build())
                    .map(o -> o instanceof AwsSdk2TransportOptions ? ((AwsSdk2TransportOptions) o) : null)
                    .map(AwsSdk2TransportOptions::credentials)
                    .or(() -> Optional.ofNullable(AwsSdk2TransportOptions.builder().build().credentials()))
                    .orElse(DefaultCredentialsProvider.create());

            Aws4SignerParams signerParams = Aws4SignerParams.builder()
                    .awsCredentials(credentials.resolveCredentials())
                    .signingName("aoss")
                    .signingRegion(region)
                    .build();
           //pring cred values
            AwsSessionCredentials cred = (AwsSessionCredentials) signerParams.awsCredentials();
            System.out.println(cred.);
            System.out.println(cred.secretAccessKey());
            System.out.println(cred.sessionToken());

            OpenSearchClient client = new OpenSearchClient(
                    new AwsSdk2Transport(
                            httpClient,
                            host,
                            "aoss",
                            region,
                            AwsSdk2TransportOptions.builder().build()));

            try {
				try {
					client.cat().indices();
				} catch (IOException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
				}
			} catch (OpenSearchException e) {
				// TODO Auto-generated catch block
				System.out.println("ERROR" + e.error().reason());
			}
            
        } finally {
            httpClient.close();
        }
    }
}

AWS supplies session access & secret ids along with session token. That will get printed. But final result would be a 403 error with message "authentication/authorization failure".

This not a permission issue in IAM role. If we pass access key & secret key (generate with same permissions) in environment variable, the client works fine.

Also Python opensearch-py library works just fine within the same container with session credentials from same IAM role.

Python code:

from opensearchpy import OpenSearch, RequestsHttpConnection
from requests_aws4auth import AWS4Auth
import boto3 
client = boto3.client('opensearchserverless')
service = 'aoss'
region = "us-west-2"
credentials = boto3.Session().get_credentials()
awsauth = AWS4Auth(credentials.access_key, credentials.secret_key,region, service, session_token=credentials.token)
client = OpenSearch(hosts=[{'host': "<test>.us-west-2.aoss.amazonaws.com", "port": 443}],http_auth=awsauth,use_ssl=True,verify_certs=True, connection_class=RequestsHttpConnection,timeout=300)
client.indices.get_mapping()

Have anyone faced this issue while accessing OpenSearch serverless with opensearch-java 2.2.0? Is there any workaround or planned fix?

Start with the AwsSdk2Transport and AwsSdk2TransportOptions usage in the supplied Java class, then run the reproduction against OpenSearch Serverless with the IAM role's session credentials. Compare its credential and request-signing path with the working opensearch-py example; done means the Java client can call client.cat().indices() without the 403 authentication/authorization failure.