[BUG] Dots Discovered Key Names

Describe the bug
Keys with "." in them are not able to be processed.

When ingesting logs from FluentBit -> S3 -> SQS -> Data Prepper / OSIS -> OpenSearch any key that has a dot "." in it is throwing an error on ingestion, see below error from OSIS. I believe this is because the Kubernetes metadata in labels contains dots.

2024-09-24T14:08:46.611 [s3-log-pipeline-sink-worker-2-thread-2] WARN  org.opensearch.dataprepper.plugins.sink.opensearch.BulkRetryStrategy - operation = Index, status = 400, error = can't merge a non object mapping [kubernetes.labels.app] with an object mapping

The JSON blob looks as such

    "labels": {
      "app": "fooservice",
      "app.kubernetes.io/component": "foo",
      "app.kubernetes.io/instance": "foo-in-cluster",
      "app.kubernetes.io/managed-by": "Helm",
      "app.kubernetes.io/name": "fooservice",
      "app.kubernetes.io/version": "somelonghash",

If these labels aren't in the log ingestion succeeds. One challenge is that the labels vary from service to service so predicting what they will be is difficult. It would be preferable if there was a way to say "If the key found has a "." (or some other char) substitute it with "_" or whatever the user chooses.

It is possible that this is able to be done and I am unaware on how to do so.

To Reproduce

Attempt to process and ingest a log file to OpenSearch with Data Prepper with a log that has Keys that contain dots "."

Such as:

    "labels": {
      "app": "fooservice",
      "app.kubernetes.io/component": "foo",
      "app.kubernetes.io/instance": "foo-in-cluster",
      "app.kubernetes.io/managed-by": "Helm",
      "app.kubernetes.io/name": "fooservice",
      "app.kubernetes.io/version": "somelonghash",

Expected behavior
The key in double quotes is processed as a key even when dots are present.

Environment (please complete the following information):

• AWS Managed OpenSearch Ingestion Service

Additional context
Seems this is related and was merged with a Fix. But it is unclear on how to resolve this issue.

https://github.com/opensearch-project/data-prepper/issues/450

No source files or tests are identified in the issue. Start by reviewing the linked issue 450 and reproducing ingestion with the provided dotted Kubernetes label keys; determine the existing handling and configuration options before scoping a change. Done means these keys can be processed without the OpenSearch mapping error, with the expected key-preservation or substitution behavior defined.