GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator
Identical strings failing validation when TD uses UNICODE encoding
- Dominant language
- Python
- Stars
- 524
- Forks
- 171
- Avg merge
- 5d 15h
- Merged PRs (30d)
- 4
Description
A TD and BQ table have the same string and the validation is failing.
```
# TD
create table unicode_col_example (
id INTEGER,
str_col CHAR(10) CHARACTER SET UNICODE NOT CASESPECIFIC COMPRESS
);
INSERT INTO unicode_col_example (id, str_col) VALUES (1, 'weiß');
```
```
# BQ
create table dataset.unicode_col_example (
id INT64,
str_col STRING
);
INSERT INTO dataset.unicode_col_example (id, str_col) VALUES (1, 'weiß');
```
```
# DVT
data-validation -v validate row \
-sc teradata \
-tc bq-conn \
-tbls unicode_col_example=dataset.unicode_col_example \
--primary-keys id \
--hash '*' \
-bqrh project.dataset.dvt_results
```
Contributor guide
Assessment
This issue has not been assessed yet.