opensearch-project / opensearch-project/data-prepper
[BUG] Data Prepper allows creating pipelines with multiple sources
Open
@chenqi0805 is already working on this.
Since May 20, 2025.
bug
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Describe the bug
Data Prepper only supports a single source. But, the configuration will pass if multiple are provided. It appears that the first is chose.
To Reproduce
Steps to reproduce the behavior:
- Create a pipeline like this:
pipeline:
workers: 2
source:
http:
otel_trace:
sink:
- stdout:
- Run Data Prepper
See that the http source starts.
2025-05-20T19:17:13,762 [pipeline-sink-worker-2-thread-1] INFO org.opensearch.dataprepper.plugins.source.loghttp.HTTPSource - Started http source on port 2021...
Expected behavior
Data Prepper should fail indicating that only one source may be defined.
Screenshots
N/A
Environment (please complete the following information):
Data Prepper 2.11.0
Additional context
N/A
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.