[BUG] Failed to initialize OpenSearch sink, retrying: Forbidden access

Describe the bug
add roles and users to opensearch

"log_collector_prepper" : {
    "reserved" : false,
    "hidden" : false,
    "cluster_permissions" : [
      "cluster_monitor"
    ],
    "index_permissions" : [
      {
        "index_patterns" : [
          "logs-*",
          "nodes-*",
          "kube-*",
          "logstash-*",
          "pacman-*"
        ],
        "fls" : [ ],
        "masked_fields" : [ ],
        "allowed_actions" : [
          "crud",
          "create_index"
        ]
      },
      {
        "index_patterns" : [
          ".ds-logs-*",
          ".ds-nodes-*",
          ".ds-kube-*",
          ".ds-logstash-*",
          ".ds-pacman-*"
        ],
        "fls" : [ ],
        "masked_fields" : [ ],
        "allowed_actions" : [
          "indices:admin/mapping/put"
        ]
      }
    ],
    "tenant_permissions" : [
      {
        "tenant_patterns" : [
          "*"
        ],
        "allowed_actions" : [
          "kibana_all_write"
        ]
      }
    ],
    "static" : false
  }

install data prepper via helm

- chart: opensearch/data-prepper
    version: 0.1.0
    name: data-prepper
    namespace: logging
    values:
      - pipelineConfig:
          config:
            simple-sample-pipeline:
              source:
                http:
              buffer:
                bounded_blocking:
                  buffer_size: 1024     # max number of records the buffer accepts
                  batch_size: 256       # max number of records the buffer drains after each read
              processor:
              route:
              sink:
                - opensearch:
                    hosts: ["https://opensearch-cluster-master:9200"]
                    cert: /usr/share/data-prepper/config/logstash-tls/ca.crt
                    username: dataprepper
                    password: dataprepper
                    index_type: custom
                    index: kube-2
                    max_retries: 4

and got errors

2024-08-21T09:37:16,656 [simple-sample-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Initializing OpenSearch sink
2024-08-21T09:37:16,656 [simple-sample-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the username provided in the config.
2024-08-21T09:37:16,656 [simple-sample-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the cert provided in the config.
2024-08-21T09:37:16,659 [simple-sample-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the username provided in the config.
2024-08-21T09:37:16,659 [simple-sample-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the cert provided in the config.
2024-08-21T09:37:16,678 [simple-sample-pipeline-sink-worker-2-thread-1] WARN  org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Failed to initialize OpenSearch sink, retrying: Forbidden access 

To Reproduce
I've test the rights from conteiner and index created

curl -XPUT "https://opensearch-cluster-master:9200/kube-011" -u 'dataprepper:dataprepper' --cacert /usr/share/data-prepper/config/logstash-tls/ca.crt

so after some searching I found this article
and add this role

"data_prepper" : {
    "reserved" : false,
    "hidden" : false,
    "cluster_permissions" : [
      "cluster_all",
      "indices:admin/template/put",
      "indices:admin/template/get"
    ],
    "index_permissions" : [
      {
        "index_patterns" : [
          "otel-v1*"
        ],
        "dls" : "",
        "fls" : [ ],
        "masked_fields" : [ ],
        "allowed_actions" : [
          "indices_all"
        ]
      },
      {
        "index_patterns" : [
          ".opendistro-ism-config"
        ],
        "dls" : "",
        "fls" : [ ],
        "masked_fields" : [ ],
        "allowed_actions" : [
          "indices_all"
        ]
      },
      {
        "index_patterns" : [
          "*"
        ],
        "dls" : "",
        "fls" : [ ],
        "masked_fields" : [ ],
        "allowed_actions" : [
          "manage_aliases"
        ]
      }
    ],
    "tenant_permissions" : [
      {
        "tenant_patterns" : [
          "*"
        ],
        "allowed_actions" : [
          "kibana_all_write"
        ]
      }
    ],
    "static" : false
  }

but same behavior
if I add all_access role it works fine

Expected behavior
to work well under a limited role
or to provide information which role should we create

Screenshots

Environment (please complete the following information):

• Version 0.1.0

Additional context

Start with data-prepper-plugins/opensearch/opensearch_security.md and reproduce the OpenSearch sink initialization using the limited role and Helm configuration shown. Compare the permissions required by the sink with the observed Forbidden access; done means the sink works with a documented least-privilege role or clearly identifies the missing permission.