opensearch-project / opensearch-project/data-prepper

[BUG] Nested configuration validation annotations are not getting triggered

Open
#2,743 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Java
Stars
374
Forks
354
Avg merge
3d 18h
Merged PRs (30d)
8

Description

Describe the bug
Any nested configurations in a pipeline configuration will not run the validations of the nested class without the @Valid annotation.

For example, the aws parameter gets validated in the s3 source because it has the @Valid annotation (https://github.com/opensearch-project/data-prepper/blob/d0d7ba01d179b60d153ff54865c7924a133a7358/data-prepper-plugins/s3-source/src/main/java/org/opensearch/dataprepper/plugins/source/S3SourceConfig.java#L42)

While the sqs options do not get validated (https://github.com/opensearch-project/data-prepper/blob/d0d7ba01d179b60d153ff54865c7924a133a7358/data-prepper-plugins/s3-source/src/main/java/org/opensearch/dataprepper/plugins/source/S3SourceConfig.java#L38)

To Reproduce
Steps to reproduce the behavior:
Run a pipeline configuration like the following and you will get a NPE at runtime and not get an exception for this validation until you add the @Valid annotation

s3-scan-pipeline:
  source:
    s3:
      sqs:
        queue_url: null
      codec:
        newline:
      aws:
        region: "us-west-2"
        sts_role_arn: "arn:aws:iam::870201406020:role/s3-to-osis-role"
  sink:
    - stdout:

Expected behavior
These should all be validated

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: [e.g. Ubuntu 20.04 LTS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the linked S3SourceConfig.java references and run the provided pipeline configuration to reproduce the validation failure and resulting NPE. Trace how nested configuration validation is invoked, then verify that invalid nested options such as a null queue_url are rejected before runtime use and that the affected nested configurations are covered.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.