apache / apache/hamilton

`check_output_custom` could pass the information about the feature definition to the output validators

Open
#809 5 comments 0 reactions 0 assignees View on GitHub
data quality
Dominant language
Jupyter Notebook
Stars
2.6k
Forks
213
PR merge metrics
No merged PRs in 30d

Description

hello, we're using the decorator `check_output_custom` to add custom validation to our features, but while the validator class is defined only once, each feature has its own rules for validation

the problem is that the validator class does not receive any information on the feature it will be validating, only the resulting pd.Series, and we would need at least the name of the feature to be able to validate it correctly without explicitly using the name of the feature on the validator

the `DataValidator` child class could receive either in the `__init__` or in the `validate` method the function whose results it should validate

I'll glady open a PR with the change

I'd like to be possible to do this:
```python
class MyValidator(DataValidator):
def validate(self, dataset, fn):
validator = VALIDATORS[fn.name]
return validator(dataset)

@check_output_custom(MyValidator("warn"))
def sum_feature(x, y):
return x + y
```

instead of needing to do this:
```python
class MyValidator(DataValidator):
def __init__(self, importance, fn_name):
self.super().__init__(importance=importance)
self.fn_name = fn_name

def validate(self, dataset):
validator = VALIDATORS[self.fn_name]
return validator(dataset)

@check_output_custom(MyValidator("warn", "sum_feature"))
def feature(x, y):
return x + y
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.