opensearch-project / opensearch-project/data-prepper
[BUG] DataPrepper crashes when following quickstart instructions
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Describe the bug
DataPrepper crashes when following quickstart instructions
The stack trace is quite long, but the relevant part is (not terribly decipherable, but anyway)...
Reading pipelines and data-prepper configuration files from Data Prepper home directory.
/opt/java/openjdk/bin/java
Found openjdk version of 17.0
2023-11-08T19:02:11,203 [main] INFO org.opensearch.dataprepper.DataPrepperArgumentConfiguration - Command line args: /usr/share/data-prepper/pipelines,/usr/share/data-prepper/config/data-prepper-config.yaml
2023-11-08T19:02:11,440 [main] INFO org.opensearch.dataprepper.parser.PipelinesDataflowModelParser - Reading pipeline configuration from pipelines.yaml
2023-11-08T19:02:11,782 [main] WARN org.springframework.context.support.AbstractApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dataPrepper' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/DataPrepper.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'pipelineParser' defined in org.opensearch.dataprepper.parser.config.PipelineParserConfiguration: Unsatisfied dependency expressed through method 'pipelineParser' parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionsApplier' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionsApplier.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionLoader' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionLoader.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionPluginConfigurationConverter' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionPluginConfigurationConverter.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionPluginConfigurationResolver' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionPluginConfigurationResolver.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataPrepperConfiguration' defined in org.opensearch.dataprepper.parser.config.DataPrepperAppConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.opensearch.dataprepper.parser.model.DataPrepperConfiguration]: Factory method 'dataPrepperConfiguration' threw exception; nested exception is java.lang.IllegalArgumentException: Invalid DataPrepper configuration file.
To Reproduce
Follow the instructions: https://opensearch.org/docs/latest/data-prepper/getting-started/
pipelines.yaml
simple-sample-pipeline:
workers: 2
delay: "5000"
source:
random:
sink:
- stdout:
data-prepper-config.yaml
This is empty since it doesn't seem to require anything by default
docker command
docker run --name data-prepper -p 4900:4900 -v ${PWD}/pipelines.yaml:/usr/share/data-prepper/pipelines/pipelines.yaml -v ${PWD}/data-prepper-config.yaml:/usr/share/data-prepper/config/data-prepper-config.yaml opensearchproject/data-prepper:latest
Expected behavior
Should have the output of UUIDs as described.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked quickstart instructions and reproduce the failure using pipelines.yaml, the empty data-prepper-config.yaml, and the provided Docker command. Compare the configuration and image behavior with the expected setup; done means DataPrepper starts successfully and outputs UUIDs as described.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, java
- Domain
- backend, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100