opensearch-project / opensearch-project/data-prepper
Support dynamic indexAlias through dynamic index patterns in ism policy using DP
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
Currently data-prepper is managing dynamic policy creation and template creation but not ISM policy. When user supplies the custom policy there is no way currently update the index-patterns. Use can't supply dynamic index patterns unless user manages policies manually which will remove the user of data-prepper.
Describe the solution you'd like
Using DP we shall create index pattern/update index-pattern when user supplies ISM policy based on indexAlias*. So, that even if there is another dynamic index is create this will take care of the policy, template and index (including rollover).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing Data Prepper's existing dynamic policy and template creation, then inspect how custom ISM policies and indexAlias patterns are represented. Done means dynamically creating or updating the ISM index pattern when new dynamic indexes are created, while preserving policy, template, and rollover behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100