opensearch-project / opensearch-project/data-prepper

Provide hold_forever sink to help with testing and experimenting

Open
#4,737 0 comments 0 reactions 1 assignee View on GitHub

@dlvenable is already working on this.

Since Jul 16, 2024.

plugin - sink
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Is your feature request related to a problem? Please describe.

Sometimes when running data through Data Prepper, I'd like to test the behavior when data is not sent to a sink.

Describe the solution you'd like

Provide a new sink - hold_forever.

sink:
- hold_forever:

This sink would not send the data, would not acknowledge to acknowledgements.

This sink could provide a warning that this is only for testing purposes.

Describe alternatives you've considered (Optional)

It would be interesting to consider having a CLI flag for debugMode. Then when this flag is provided, this sink would be available. If it isn't provided, the sink is unavailable. This could prevent misuse of it.

However, Data Prepper's current arguments are not very robust. So I'd like to propose that something like this come later.

Additional context

I already have this implemented and keep it in my development environment. But, it may be useful beyond this, so I'd like to make it part of the standard sinks.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.