opensearch-project / opensearch-project/data-prepper

A common type for plugin configurations to support if ,else if ,else format

Open
#4,408 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Is your feature request related to a problem? Please describe.
As a user of data prepper, I would like to be able to set different property values in the configuration depending on certain conditions.

Describe the solution you'd like
Similar to how the opensearch sink supports the actions parameter for conditionally choosing an action

actions:
  - when: "condition"
     type: "delete" 
  - when: "condition_two"
     type: "update"
  - type: "index" # default case

I would like to extract this out for common use with any parameter, without requiring duplicate code to evaluate the result. This could be by specifying this common type in the config like this.

In data-prepper-api

public SwitchWhen<T> {
    private List<T> configList; # contains 
    private List<String> whenConditions;

    private T getConfigForEvent(final Event event);
    
    
}
SomePluginConfig {
  
   @JsonProperty("actions")
   private SwitchWhen<ActionClass> actionsWhen;

}

public ActionClass {
    @JsonProperty("type")
    String type;
}

this would make it easy to add support for other parameters that need this same functionality, for example for s3 sink thresholds

public S3SinkConfig {
   
   @JsonProperty("thresholds")
   private SwitchWhen<ThresholdOptions> thresholdOptions; 

}

Describe alternatives you've considered (Optional)
Only support expressions to fit this use case, but this required add_entries processors with when conditions to some sort of metadata or part of the Event data and then using that in expression value. This is a more confusing user experience.

Additional context
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the existing opensearch sink actions configuration and the proposed data-prepper-api SwitchWhen shape, then compare how plugin configuration classes represent actions and the proposed S3 sink thresholds. Done means a common conditional configuration type can support when clauses and a default case for multiple plugin parameters without duplicating condition evaluation.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.