GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

YAML file generate by generate-partitions does not work for some backends

Open
#1,504 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

**Describe the bug**
The yaml files generated by generate-partitions does not work for some backends.

The following backends successfully validated the yaml files generated by generate-partitions
1. BigQuery
2. Hive
3. MySQL
4. Oracle (only when primary key does not use date and timestamp columns)
5. Postgres
6. Snowflake
7. Teradata (only when strings don't have embedded quotes - see 1221)

For most of these backends the commands used to test are `data-validation generate-table-partitions -sc $ENGINE -tc $ENGINE -tbls=pso_data_validator.test_generate_partitions_v2 -pk=course_id,quarter_id,recd_timestamp,registration_date,approved -cdir . -pn 2 -hash='*' --filter-status=fail` followed by `data-validation configs run -cdir pso_data_validator.test_generate_partitions_v2`

The following backends are not able to validate yaml files generated by generate-partitions
1. Microsoft SQLServer (issue with the "false" literal, other challenges)
2. Impala - issues

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.