opensearch-project / opensearch-project/data-prepper

[BUG] lack of opensearch sink initialization error logs when data-prepper is spinning up

Open
#2,877 2 comments 1 reaction 1 assignee View on GitHub

@kkondaka is already working on this.

Since Jul 5, 2023.

bug
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug
A user reported lack of opensearch sink initialization error logs when data-prepper is spinning up.

2023-06-12T22:52:21,382 [entry-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [entry-pipeline] - sink is not ready for execution, retrying
2023-06-12T22:52:21,382 [service-map-pipeline-sink-worker-4-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [service-map-pipeline] - sink is not ready for execution, retrying
2023-06-12T22:52:21,383 [entry-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [entry-pipeline] Waiting for Sink to be ready
2023-06-12T22:52:21,383 [service-map-pipeline-sink-worker-4-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Initializing OpenSearch sink
2023-06-12T22:52:21,383 [raw-pipeline-sink-worker-6-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [raw-pipeline] - sink is not ready for execution, retrying
2023-06-12T22:52:21,383 [raw-pipeline-sink-worker-6-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Initializing OpenSearch sink
2023-06-12T22:52:21,407 [main] WARN  org.opensearch.dataprepper.pipeline.server.HttpServerProvider - Creating Data Prepper server without TLS. This is not secure.
2023-06-12T22:52:21,407 [main] WARN  org.opensearch.dataprepper.pipeline.server.HttpServerProvider - In order to set up TLS for the Data Prepper server, go here: https://github.com/opensearch-project/data-prepper/blob/main/docs/configuration.md#server-configuration
2023-06-12T22:52:21,409 [raw-pipeline-sink-worker-6-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - aws_sigv4 is set, will sign requests using AWSRequestSigningApacheInterceptor
2023-06-12T22:52:21,409 [service-map-pipeline-sink-worker-4-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - aws_sigv4 is set, will sign requests using AWSRequestSigningApacheInterceptor
2023-06-12T22:52:21,425 [main] INFO  org.opensearch.dataprepper.pipeline.server.DataPrepperServer - Data Prepper server running at :4900
2023-06-12T22:52:21,883 [service-map-pipeline-sink-worker-4-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the trust all strategy
2023-06-12T22:52:21,883 [raw-pipeline-sink-worker-6-thread-1] INFO  org.opensearch.dataprepper.plugins.sink.opensearch.ConnectionConfiguration - Using the trust all strategy
2023-06-12T22:52:22,383 [entry-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [entry-pipeline] - sink is not ready for execution, retrying
2023-06-12T22:52:23,383 [entry-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [entry-pipeline] - sink is not ready for execution, retrying
2023-06-12T22:52:24,384 [entry-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [entry-pipeline] - sink is not ready for execution, retrying
2023-06-12T22:52:25,384 [entry-pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [entry-pipeline] - sink is not ready for execution, retrying

Still need to reproduce the issue

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: [e.g. Ubuntu 20.04 LTS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.