opensearch-project / opensearch-project/data-prepper

[BUG] Data Prepper allows creating pipelines with multiple sources

Open
#5,708 0 comments 0 reactions 1 assignee View on GitHub

@chenqi0805 is already working on this.

Since May 20, 2025.

bug
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug

Data Prepper only supports a single source. But, the configuration will pass if multiple are provided. It appears that the first is chose.

To Reproduce
Steps to reproduce the behavior:

  1. Create a pipeline like this:
pipeline:
  workers: 2
  source:
    http:
    otel_trace:
  sink:
    - stdout:
  1. Run Data Prepper

See that the http source starts.

2025-05-20T19:17:13,762 [pipeline-sink-worker-2-thread-1] INFO  org.opensearch.dataprepper.plugins.source.loghttp.HTTPSource - Started http source on port 2021...

Expected behavior

Data Prepper should fail indicating that only one source may be defined.

Screenshots
N/A

Environment (please complete the following information):

Data Prepper 2.11.0

Additional context

N/A

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.