opensearch-project / opensearch-project/data-prepper

[BUG] Understanding what caused a pipeline shutdown can be difficult.

Open
#3,986 0 comments 0 reactions 1 assignee View on GitHub

@dlvenable is already working on this.

Since Jan 18, 2024.

bug
Dominant language
Java
Stars
374
Forks
355
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug

Sometimes Data Prepper shuts down without a clear reason as to why.

2024-01-18T06:23:32.523 [pool-33-thread-1] INFO    org.opensearch.dataprepper.DataPrepper - Shutting down pipeline: my-pipeline
2024-01-18T06:23:32.524 [pool-33-thread-1] INFO  org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [my-pipeline] - Received shutdown signal with buffer drain timeout PT0S, processor shutdown timeout PT5M, and sink shutdown timeout PT30S. Initiating the shutdown process
2024-01-18T06:23:34.564 [pool-33-thread-1] INFO  org.opensearch.dataprepper.plugins.source.loghttp.HTTPSource - Stopped http source.

Expected behavior

I'd like to see clearer logging when the pipeline shuts down. Flor example, was it from a call to POST /shutdown?

Environment (please complete the following information):

  • Data Prepper: 2.6.1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.