GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

Errors validating tables with special characters in identifiers

Open
#1,271 0 comments 0 reactions 0 assignees View on GitHub
priority: p2
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

I am documenting issues discovered while working on issue https://github.com/GoogleCloudPlatform/professional-services-data-validator/issues/1192 that are unrelated to the original issue I was fixing.

Working on this issue is blocked by https://github.com/GoogleCloudPlatform/professional-services-data-validator/issues/1192 because the test tables/tests mentioned below are to be delivered as part of that issue.

**Oracle**
Test `test_row_validation_identifiers` is skipped because it fails with:
```
WITH t0 AS
*
ERROR at line 1:
ORA-04054: database link AT does not exist
```
This is because the `concat__all` expression does not quote column names:
```
...
, t2."RSTRIP__IFNULL__CAST__COL-HYPHEN" AS "RSTRIP__IFNULL__CAST__COL-HYPHEN"
, t2."RSTRIP__IFNULL__CAST__COL@AT" AS "RSTRIP__IFNULL__CAST__COL@AT"
, t2.rstrip__ifnull__cast__other_data AS rstrip__ifnull__cast__other_data
, rstrip__ifnull__cast__id || rstrip__ifnull__cast__col#hash || rstrip__ifnull__cast__col$dollar || rstrip__ifnull__cast__col-hyphen || rstrip__ifnull__cast__col@at || rstrip__ifnull__cast__other_data AS concat__all
FROM t2) t3
...
```

**BigQuery**
Tests `test_column_validation_identifiers` and `test_row_validation_identifiers` are skipped because they fail with:
```
google.api_core.exceptions.BadRequest: 400 Name length__col#hash not found inside t1 at [7:17]
```

**Teradata**
All three `test_*_validation_identifiers` tests fail with:
```
E pandas.errors.DatabaseError: Execution failed on sql '
E HELP COLUMN udf.dvt-identifier$_#.*;
E ': [Version 20.0.0.12] [Session 3718] [Teradata Database] [Error 3807] Object 'udf' does not exist.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.