apache / apache/hop

[Feature Request]: Add conditional applicability per validation rule in Data Validator

Open
#7,022 3 comments 0 reactions 0 assignees View on GitHub
awaiting triage P3 Transforms
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 32m
Merged PRs (30d)
216

Description

### What would you like to happen?

Description
Today, the Data Validator step applies each configured validation rule to every incoming row.

In some cases, this creates unnecessary complexity in the pipeline. A common scenario is when most validation rules must be applied to all rows, but one or more specific rules should only be applied when a condition is met.

For example:

Validation A: apply to all rows
Validation B: apply to all rows
Validation C: apply only when a given field or business condition matches

At the moment, the workaround is usually to split the flow with Filter Rows and use additional Data Validator steps. This works, but it makes the pipeline more verbose, harder to maintain, and can duplicate validation logic.

Proposal
Add an optional condition for each validation rule inside the Data Validator step.

Suggested behavior:

If no condition is defined, the validation rule applies to all rows
If a condition is defined, that validation rule is evaluated only when the condition is true
If the condition is false, that rule is skipped for the current row

This is not a request to add general row filtering inside Data Validator.
The goal is specifically to support conditional validation per rule.

Example use cases

Validate CPF only when person_type = 'F'
Validate CNPJ only when person_type = 'J'
Require cancellation date only when status = 'CANCELLED'
Apply state-specific validation only when state = 'SP'

Benefits

Reduces the need for extra Filter Rows steps
Reduces the need for duplicated Data Validator steps
Keeps related validation rules together in one place
Improves pipeline readability and maintainability
Better supports business rules where some validations are only applicable in specific contexts

Possible UI approach

Add an optional field per validation rule, such as:

Apply when
Condition
Validation condition

If empty, the rule applies to all rows.

Notes

A key design point is to keep the feature scoped to rule applicability, not general flow control.
This would preserve the current responsibility of the Data Validator step while making it more flexible for conditional business validations.

### Issue Priority

Priority: 3

### Issue Component

Component: Transforms

Contributor guide

Open the contributing guide

Research direction

Start by locating the Data Validator transform and its validation-rule configuration and UI entry points. Review how rules are evaluated for each incoming row and how conditions are represented elsewhere, then define and test the per-rule behavior: no condition applies universally, a true condition evaluates the rule, and a false condition skips it.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.