GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

Identical strings failing validation when TD uses UNICODE encoding

Open
#1,429 6 comments 0 reactions 0 assignees View on GitHub
type: question
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

A TD and BQ table have the same string and the validation is failing.

```
# TD
create table unicode_col_example (
id INTEGER,
str_col CHAR(10) CHARACTER SET UNICODE NOT CASESPECIFIC COMPRESS
);

INSERT INTO unicode_col_example (id, str_col) VALUES (1, 'weiß');
```

```
# BQ
create table dataset.unicode_col_example (
id INT64,
str_col STRING
);

INSERT INTO dataset.unicode_col_example (id, str_col) VALUES (1, 'weiß');
```

```
# DVT
data-validation -v validate row \
-sc teradata \
-tc bq-conn \
-tbls unicode_col_example=dataset.unicode_col_example \
--primary-keys id \
--hash '*' \
-bqrh project.dataset.dvt_results
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.