GoogleCloudPlatform / GoogleCloudPlatform/cloud-data-quality
Allow multiple row_filters and params to row_filters
- Dominant language
- Python
- Stars
- 283
- Forks
- 59
- PR merge metrics
- No merged PRs in 30d
Description
Is there any reason that not allow multiple row_filters and params like rule_id?
While using clouddq, i got some needs of this in some situation.
if allow params, then row_filters can be used in general. I mean one row_filter can be used in many entities.
Or sometime, we want to filter by a column other than the one used for rule_binding.
I suggest modifying like below, then filtering email row_filter can be use any entity with any given column.
```yaml
row_filters:
NONE:
filter_sql_expr: |-
True
DATA_TYPE_EMAIL:
params: |-
- column
filter_sql_expr: |-
$column = 'email'
rule_bindings:
T2_DQ_1_EMAIL:
entity_id: TEST_TABLE
column_id: VALUE
row_filter_ids:
- DATA_TYPE_EMAIL:
column: contact_type
rule_ids:
- NOT_NULL_SIMPLE
- REGEX_VALID_EMAIL
- CUSTOM_SQL_LENGTH:
upper_bound: 30
- NOT_BLANK
metadata:
brand: one
```
Is there any way to use it in this situation? if not, can i adding this?
Contributor guide
Assessment
This issue has not been assessed yet.