opensearch-project / opensearch-project/data-prepper
Provide a configuration for OpenSearch Data Streams
@dlvenable is already working on this.
Since May 6, 2026.
- Dominant language
- Java
- Stars
- 374
- Forks
- 355
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
In #6249, we added support for the OpenSearch sink to automatically detect if an index is a data stream.
In some cases, users are seeing inconsistencies. This could be because the permissions to detect are not present, or if the index is not always a data stream (perhaps on other clusters)
Describe the solution you'd like
Provide a new sink configuration to control data streams named data_stream.
The possible values are: automatic (the default), always, never.
opensearch:
index: "my-index"
data_stream: "automatic" # or "always" / "never"
The enum would look something like this:
public enum DataStreamStatus {
AUTOMATIC("automatic"),
ALWAYS("always"),
NEVER("never");
}
Describe alternatives you've considered (Optional)
N/A
Additional context
Issues: #2037, #2038
PR: #6249
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.