Helpful ideas for feature validation
- Dominant language
- Jupyter Notebook
- Stars
- 201
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
I recently stumble several times with this error in triage:
```
ValueError: Missing columns in get_impute_create: {'demographics_entity_id_all_primarypayercode_country_indigent_max,
}
```
And I noticed that those names have longer lengths that the ones admitted for column names in PostgreSQL. Since `collate` is looking up the column names in the `features_aggregation` table by creating a `{feature_group}_{group}_{column_name}_{aggregation_fx}` naming, it's not able to grab the truncated names that PostgreSQL is using. In my case: `demographics_entity_id_all_primarypayercode_countr`.
I would add a feature into the validator to make a column name length calculation using the elements from the `feature_aggregation` elements in the triage's configuration file. We can use the `choices` or the `choice_query` to grab these errors before running triage. Is this a good idea? I'm the only one with long name problems?
Contributor guide
Assessment
This issue has not been assessed yet.