GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator
Nullable numeric/float fields can cause issues as being part of a primary key
- Dominant language
- Python
- Stars
- 524
- Forks
- 171
- Avg merge
- 5d 15h
- Merged PRs (30d)
- 4
Description
Nullable numeric/float fields can cause issues as being part of a primary key since they can be `null` and they appear in the filters as `nan`
Example:
```
filters:
- source: (opportunity_id > '132252' OR opportunity_id = '132252' AND (stagename
> 'In Progress' OR stagename = 'In Progress' AND (stage_change_datetime > '2023-01-02
11:09:20' OR stage_change_datetime = '2023-01-02 11:09:20' AND (chain_name >
'(Not available)' OR chain_name = '(Not available)' AND (amount >= nan)))))
AND (opportunity_id < '134269' OR opportunity_id = '134269' AND (stagename <
'Onboarding' OR stagename = 'Onboarding' AND (stage_change_datetime < '2020-06-29
15:04:37' OR stage_change_datetime = '2020-06-29 15:04:37' AND (chain_name <
'241 Pizza' OR chain_name = '241 Pizza' AND (amount < nan)))))
target: (opportunity_id > '132252' OR opportunity_id = '132252' AND (stagename
> 'In Progress' OR stagename = 'In Progress' AND (stage_change_datetime > '2023-01-02
11:09:20' OR stage_change_datetime = '2023-01-02 11:09:20' AND (chain_name >
'(Not available)' OR chain_name = '(Not available)' AND (amount >= nan)))))
AND (opportunity_id < '134269' OR opportunity_id = '134269' AND (stagename <
'Onboarding' OR stagename = 'Onboarding' AND (stage_change_datetime < '2020-06-29
15:04:37' OR stage_change_datetime = '2020-06-29 15:04:37' AND (chain_name <
'241 Pizza' OR chain_name = '241 Pizza' AND (amount < nan)))))
type: custom
```
(see `(amount < nan)`)
As far as I know it's not possible to apply a COALESCE or type-cast in fields when generating partitions
Contributor guide
Assessment
This issue has not been assessed yet.