Permissions issues with AOS and composable index template

Title
OpenSearch sink fails with generic security_exception: authentication/authorization failure on composable index template check, despite maximum IAM/OpenSearch permissions

Describe the bug

The OpenSearch sink fails to initialize with a retryable security_exception: authentication/authorization failure specifically on the composable index template existence check (existsIndexTemplate), even after granting maximum permissions on both the OpenSearch internal security role and the IAM execution role.

To Reproduce

Steps to reproduce the behavior:

Create an OpenSearch Ingestion pipeline with an opensearch sink using template_type: index-template (composable index template) against an OpenSearch 3.3 domain with fine-grained access control enabled.
Map the pipeline's IAM execution role as a backend role to an OpenSearch internal security role with cluster_all + indices_all permissions.
Attach an IAM policy to the execution role granting es:* on the domain resource (wildcard, for testing).
Start the pipeline and observe sink initialization logs.

Expected behavior

Sink should initialize successfully and proceed to create/verify the index template.

Actual behavior

Sink repeatedly fails with:

org.opensearch.client.opensearch._types.OpenSearchException: Request failed: [security_exception] authentication/authorization failure
at org.opensearch.client.transport.aws.AwsSdk2Transport.parseResponse(AwsSdk2Transport.java:541)
at org.opensearch.client.transport.aws.AwsSdk2Transport.executeSync(AwsSdk2Transport.java:440)
at org.opensearch.client.transport.aws.AwsSdk2Transport.performRequest(AwsSdk2Transport.java:217)
at org.opensearch.client.opensearch.indices.OpenSearchIndicesClient.existsIndexTemplate(OpenSearchIndicesClient.java:571)
at org.opensearch.dataprepper.plugins.sink.opensearch.index.ComposableTemplateAPIWrapper.getTemplate(ComposableTemplateAPIWrapper.java:45)
at org.opensearch.dataprepper.plugins.sink.opensearch.index.ComposableIndexTemplateStrategy.getExistingTemplateVersion(ComposableIndexTemplateStrategy.java:28)
at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.shouldCreateTemplate(AbstractIndexManager.java:292)
at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.checkAndCreateIndexTemplate(AbstractIndexManager.java:246)
at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.checkAndCreateIndexTemplate(AbstractIndexManager.java:234)
at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.setupIndex(AbstractIndexManager.java:224)
at org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink.doInitializeInternal(OpenSearchSink.java:173)
at org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink.doInitialize(OpenSearchSink.java:119)
at org.opensearch.dataprepper.model.sink.AbstractSink.initialize(AbstractSink.java:59)
at org.opensearch.dataprepper.core.pipeline.Pipeline.isReady(Pipeline.java:237)

The exact same underlying operation succeeds when run manually via OpenSearch Dashboards Dev Tools (as the domain's master user):

GET _index_template/main-fashchat-vectorindex
→ 404 resource_not_found_exception (expected, template doesn't exist yet)

Troubleshooting already performed:

Confirmed OpenSearch role mapping (backend role → security role) is correct, under "Backend roles" not "Users"
Escalated OpenSearch internal role permissions to cluster_all + indices_all on index pattern * — no change
Escalated IAM policy on the pipeline's execution role to es:* wildcard on arn:aws🇪🇸::domain// — no change
Purged OpenSearch security plugin cache — no change
Fully stopped/restarted the pipeline (multiple times) — no change
Confirmed domain access policy is fully open (Principal: , Allow, es:)
Confirmed cluster health is green, no relocating/unassigned shards

Environment

OpenSearch version: 3.3
Deployment: Amazon OpenSearch Ingestion (managed Data Prepper), domain with fine-grained access control enabled, VPC-based
Sink config: aws.serverless: false, template_type: index-template, custom template_content with knn_vector mapping

Additional context

Given that maximum permissions on both the IAM and OpenSearch security layers don't resolve this, and the identical operation succeeds via a different auth path (master user through Dev Tools), this looks like it could be specific to how the sink signs/sends this particular existsIndexTemplate request (possibly using an HTTP verb or header not covered by expectations), or an incompatibility between the Data Prepper client version bundled in OpenSearch Ingestion and OpenSearch 3.3's composable template API. Any guidance on what this specific request looks like on the wire (method, headers) would help us pinpoint the mismatch.

Start by tracing ComposableTemplateAPIWrapper.getTemplate and ComposableIndexTemplateStrategy.getExistingTemplateVersion, then follow the existsIndexTemplate call in the OpenSearch client stack shown in the report. Compare the sink request with the successful Dev Tools request, including its method and headers. Done means the sink can initialize and create or verify the composable index template under the reported managed OpenSearch setup.