opensearch-project / opensearch-project/data-prepper

[BUG] DataPrepper crashes when following quickstart instructions

Open
#3,612 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug documentation
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug
DataPrepper crashes when following quickstart instructions

The stack trace is quite long, but the relevant part is (not terribly decipherable, but anyway)...

Reading pipelines and data-prepper configuration files from Data Prepper home directory.
/opt/java/openjdk/bin/java
Found openjdk version of 17.0
2023-11-08T19:02:11,203 [main] INFO org.opensearch.dataprepper.DataPrepperArgumentConfiguration - Command line args: /usr/share/data-prepper/pipelines,/usr/share/data-prepper/config/data-prepper-config.yaml
2023-11-08T19:02:11,440 [main] INFO org.opensearch.dataprepper.parser.PipelinesDataflowModelParser - Reading pipeline configuration from pipelines.yaml
2023-11-08T19:02:11,782 [main] WARN org.springframework.context.support.AbstractApplicationContext - Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'dataPrepper' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/DataPrepper.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'pipelineParser' defined in org.opensearch.dataprepper.parser.config.PipelineParserConfiguration: Unsatisfied dependency expressed through method 'pipelineParser' parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionsApplier' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionsApplier.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionLoader' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionLoader.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionPluginConfigurationConverter' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionPluginConfigurationConverter.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'extensionPluginConfigurationResolver' defined in URL [jar:file:/usr/share/data-prepper/lib/data-prepper-core-2.5.0.jar!/org/opensearch/dataprepper/plugin/ExtensionPluginConfigurationResolver.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataPrepperConfiguration' defined in org.opensearch.dataprepper.parser.config.DataPrepperAppConfiguration: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.opensearch.dataprepper.parser.model.DataPrepperConfiguration]: Factory method 'dataPrepperConfiguration' threw exception; nested exception is java.lang.IllegalArgumentException: Invalid DataPrepper configuration file.

To Reproduce

Follow the instructions: https://opensearch.org/docs/latest/data-prepper/getting-started/

pipelines.yaml

simple-sample-pipeline:
  workers: 2
  delay: "5000"
  source:
    random:
  sink:
    - stdout:

data-prepper-config.yaml
This is empty since it doesn't seem to require anything by default

docker command

docker run --name data-prepper -p 4900:4900 -v ${PWD}/pipelines.yaml:/usr/share/data-prepper/pipelines/pipelines.yaml -v ${PWD}/data-prepper-config.yaml:/usr/share/data-prepper/config/data-prepper-config.yaml opensearchproject/data-prepper:latest

Expected behavior
Should have the output of UUIDs as described.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked quickstart instructions and reproduce the failure using pipelines.yaml, the empty data-prepper-config.yaml, and the provided Docker command. Compare the configuration and image behavior with the expected setup; done means DataPrepper starts successfully and outputs UUIDs as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, java
Domain
backend, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.