GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator
YAML file generate by generate-partitions does not work for some backends
- Dominant language
- Python
- Stars
- 524
- Forks
- 171
- Avg merge
- 5d 15h
- Merged PRs (30d)
- 4
Description
**Describe the bug**
The yaml files generated by generate-partitions does not work for some backends.
The following backends successfully validated the yaml files generated by generate-partitions
1. BigQuery
2. Hive
3. MySQL
4. Oracle (only when primary key does not use date and timestamp columns)
5. Postgres
6. Snowflake
7. Teradata (only when strings don't have embedded quotes - see 1221)
For most of these backends the commands used to test are `data-validation generate-table-partitions -sc $ENGINE -tc $ENGINE -tbls=pso_data_validator.test_generate_partitions_v2 -pk=course_id,quarter_id,recd_timestamp,registration_date,approved -cdir . -pn 2 -hash='*' --filter-status=fail` followed by `data-validation configs run -cdir pso_data_validator.test_generate_partitions_v2`
The following backends are not able to validate yaml files generated by generate-partitions
1. Microsoft SQLServer (issue with the "false" literal, other challenges)
2. Impala - issues
Contributor guide
Assessment
This issue has not been assessed yet.