opensearch-project / opensearch-project/data-prepper
DataPrepper validator should validating the expressions and functions inside the expressions
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 374
- Forks
- 354
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 8
Description
Is your feature request related to a problem? Please describe.
DataPrepper validations are not validating or catching unknown functions inside an expression. For example
processor:
- drop_events:
drop_when: "not exists(/index)"
handle_failed_events: drop
DataPrepper validation pass above configuration. But it could possibly validate the function given in the expression as well.
Describe the solution you'd like
Expressions are validate at the pipeline configuration validation step itself.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin at the pipeline configuration validation step and trace how expressions and their functions are currently handled. The issue names no files or tests, so locate the relevant validation entry point and existing expression-validation coverage first. Done means the shown drop_events configuration rejects unknown functions inside its expression during pipeline validation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100