opensearch-project / opensearch-project/data-prepper
[BUG] Ingesting example doesn't work, crashes with ERROR com.amazon.dataprepper.plugin.PluginCreator - Encountered exception while instantiating the plugin OpenSearchSink
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Describe the bug
When you are attempting to docker run by guide or docker-compose-up it is crashed with
data-prepper | 2022-09-10T23:18:28,864 [main] ERROR com.amazon.dataprepper.plugin.PluginCreator - Encountered exception while instantiating the plugin OpenSearchSink
To Reproduce
Steps to reproduce the behavior:
-
Clone repo
-
GOTO examples
-
GOTO log-ingestion
-
Follow instructions
-
By instruction enter the command docker run --name data-prepper -v /full/path/to/log_pipeline.yaml:/usr/share/data-prepper/pipelines.yaml --network "data-prepper_opensearch-net" opensearchproject/data-prepper:latest with corresponding replacement (in my case the network name was d61fd085926e log-ingestion_opensearch-net bridge local )

-
Data-Prepper crashes with exception data-prepper | 2022-09-10T23:18:28,864 [main] ERROR com.amazon.dataprepper.plugin.PluginCreator - Encountered exception while instantiating the plugin OpenSearchSink
(See full log attached)
CMD.report.txt
Expected behavior
Data Prepper is running correctly, you should see something similar to the following line as the latest output in your terminal.
INFO com.amazon.dataprepper.pipeline.ProcessWorker - log-pipeline Worker: No records received from buffer
Screenshots

Environment (please complete the following information):
- OS: Windows 11
- Docker Desktop 4.12.0
- VSCode
Additional context
I also did docker compose to reproduce the issue, behavior is the same with command line used and full path specified
docker-compose.txt
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in examples/log-ingestion and reproduce the failure using the documented Docker command or docker-compose setup, then inspect CMD.report.txt and the attached compose configuration. Done means the OpenSearchSink no longer crashes and the container reaches the expected “No records received from buffer” output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose, java
- Domain
- data-engineering, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100