opensearch-project / opensearch-project/data-prepper

Ruleset to decide a Dynamic Pipeline Config update is acceptable/possible given the new version of pipeline config

Open
#5,717 0 comments 0 reactions 1 assignee View on GitHub

@san81 is already working on this.

Since May 22, 2025.

enhancement
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Is your feature request related to a problem? Please describe.
A flexible rule based approach to be able to make a decision as to the feasibility of dynamically updating a pipeline config given the new version.

Describe the solution you'd like
A simple ruleset that is extensible in the future to add/remove any rules that controls this decision.

public interface PipelineUpdatableRule<T> {
    boolean evaluate(T input);
    String getDescription();
}

Define all the rules that we would like to apply and execute them to make the decision.

Describe alternatives you've considered (Optional)
Define a new method in the Plugin interface called IsDynamicallyUpdatable

Make every plugin implement IsDynamicallyUpdatable method and let each plugin define their own logic of whether the given new state is acceptable to update dynamically or not.

    boolean IsDynamicallyUpdatable(T input);

For the all the Source, Processors and Sink in the original pipeline, we will invoke this method to make the decision. Any new addition of Processors should be acceptable unconditionally.

Additional context
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.