GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

Generate table partitions failures for boolean types

Open
#1,618 3 comments 0 reactions 0 assignees View on GitHub
priority: p3
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

**Describe the bug**
Our test table for `generate-table-partitions` contains a boolean column. For many situations this fails to validate. Mostly I think when one engine supports booleans and the other doesn't. For example SQL Server and `bit` or Oracle and use of a string or numeric type.

Example Oracle vs PostgreSQL:
```
$ data-validation generate-table-partitions -sc=ora -tc=pg \
-tbls=pso_data_validator.test_generate_partitions_v2 \
--primary-keys="recd_timestamp,registration_date,approved" \
--concat='*' --partition-num=2 -cdir=/tmp
Traceback (most recent call last):
...
raise TypeError(
TypeError: Arguments approved:!boolean and Literal(N):string are not comparable
```

BigQuery vs Sybase (and SQL Server):
```
$ data-validation generate-table-partitions -sc=bq -tc=syb \
-tbls=pso_data_validator.test_generate_partitions_v2 \
--primary-keys="recd_timestamp,registration_date,approved" \
--concat='*' --partition-num=2 -cdir=/tmp
Traceback (most recent call last):
...
Error '(pyodbc.ProgrammingError) ('42S22', "[42S22] [FreeTDS][SQL Server]Invalid column name 'false'.\n (207) (SQLExecDirectW)")
```

**What version of DVT are you using?**
8.2.0

**What type of connections are you using for source and target?**
Several different engine pairings.

**Additional context**
It is extremely unlikely a boolean will ever be in a primary key so this can be low priority to resolve.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.