opensearch-project / opensearch-project/data-prepper
[BUG] Docker container does not stop running after Data Prepper shuts down
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Describe the bug
I'm using the Docker version of Data Prepper (opensearch-data-prepper:2.4.0-SNAPSHOT) built from source (./gradlew :dockerSolution:buildDockerImages). When using a pipeline that includes an Opensearch source, invoking Data Prepper's shutdown API appears to terminate the Data Prepper server but does not stop the Docker container.
Snippet of Data Prepper logs:
...
Found openjdk version of 17.0
2023-08-10T23:16:46,589 [main] INFO org.opensearch.dataprepper.parser.PipelineParser - Building [opensearch] as source component for the pipeline [test-pipeline]
....
2023-08-10T23:16:46,940 [main] INFO org.opensearch.dataprepper.parser.PipelineParser - Building [opensearch] as sink component
....
2023-08-10T23:16:47,223 [main] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [test-pipeline] - Initiating pipeline execution
....
2023-08-10T23:16:47,488 [main] INFO org.opensearch.dataprepper.pipeline.server.DataPrepperServer - Data Prepper server running at :4900
....
2023-08-10T23:16:48,143 [test-pipeline-sink-worker-2-thread-1] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [test-pipeline] - Submitting request to initiate the pipeline processing
....
2023-08-10T23:16:51,810 [pool-4-thread-1] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [test-pipeline] - Received shutdown signal with processor shutdown timeout PT30S and sink shutdown timeout PT30S. Initiating the shutdown process
....
2023-08-10T23:17:21,813 [pool-4-thread-1] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [test-pipeline] - Shutting down processor process workers.
....
2023-08-10T23:17:23,026 [pool-4-thread-1] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [test-pipeline] - Shutting down sink process workers.
2023-08-10T23:17:23,027 [pool-4-thread-1] INFO org.opensearch.dataprepper.pipeline.Pipeline - Pipeline [test-pipeline] - Pipeline fully shutdown.
2023-08-10T23:17:23,057 [pool-4-thread-1] INFO org.opensearch.dataprepper.pipeline.server.DataPrepperServer - Data Prepper server stopped
Command line:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
08319e699cae opensearch-data-prepper:2.4.0-SNAPSHOT "/entrypoint.sh bin/…" 23 minutes ago Up 22 minutes 0.0.0.0:4900->4900/tcp epic_jackson
$ docker top 0831
UID PID PPID C STIME TTY TIME CMD
root 48665 48639 3 23:16 ? 00:00:43 java -Dlog4j.configurationFile=/usr/share/data-prepper/config/log4j2-rolling.properties -Ddata-prepper.dir=/usr/share/data-prepper -cp /usr/share/data-prepper/lib/* org.opensearch.dataprepper.DataPrepperExecute
root 49046 48639 0 23:19 ? 00:00:00 /bin/bash
Pipeline configuration:
test-pipeline:
sink:
- opensearch:
bulk_size: 2
document_id_field: getMetadata("opensearch-document_id")
hosts:
- [redacted]
index: ${getMetadata("opensearch-index")}
password: [redacted]
username: [redacted]
source:
opensearch:
disable_authentication: true
hosts:
- [redacted]
indices:
exclude:
- index_name_regex: \.*
include:
- index_name_regex: [redacted]
- index_name_regex: [redacted]
This is possibly an issue only with the opensearch source plugin - I used the following pipeline and the Docker container stopped correctly (the sink endpoint in both cases is the same).
http-test-pipeline:
source:
http:
sink:
- opensearch:
hosts: [redacted]
username: [redacted]
password: [redacted]
index: "httptest"
To Reproduce
See above
Expected behavior
The Docker container stops after the Data Prepper server has shut down
Screenshots
N/A
Environment (please complete the following information):
- OS: MacOS 12.6.6
- Version: Data Prepper 2.4.0-SNAPSHOT
Additional context
N/A
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with the Docker image built by ./gradlew :dockerSolution:buildDockerImages, comparing the opensearch-source pipeline with the working http-source pipeline. Inspect /entrypoint.sh and the DataPrepperExecute process shown by docker top, then trace shutdown handling for the opensearch source. Done means the container exits after the Data Prepper server reports that it has stopped.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, java
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100