GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

Oracle exception "ORA-12704: character set mismatch" NVARCHAR (SQL Server also)

Open
#1,406 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

During a row validation using the --hash "*" option we observed, that the SQL that is sent to Oracle behind the scenes produces an exception if the table being validated contains NVARCHAR columns.

The reason seems to be that the coalesce function used in the SQL generated by DVT that is called to replace null values with a non-empty String like 'DEFAULT_REPLACEMENT_STRING' requires two parameters of the same datatype . In case of a NVARCHAR column one parameter of the coalesce function is of type NVARCHAR using the configured NCHAR character set for that database and the second parameter (the replacement String) is of type CHAR using the default database charset which may be different from the NCHAR character set.

Example code to reproduce this exception:

CREATE TABLE nchartest (id NUMBER(5) PRIMARY KEY, content nvarchar2(10));

INSERT INTO nchartest VALUES (1, 'one');
INSERT INTO nchartest VALUES (2, 'two');
INSERT INTO nchartest VALUES (3, null);

SELECT id, coalesce(content, 'DEFAULT_REPLACEMENT_STRING') FROM nchartest;

The SELECT statement causes the observed character-set-mismatch exception. The exception disappears, if either the value of the content column is converted to the default database charset using to_char or the replacement String is marked as NCHAR value with a leading 'N'. So both of the following statements seem to work.

SELECT id, coalesce(**to_char(content)**, 'DEFAULT_REPLACEMENT_STRING') FROM nchartest;

or

SELECT id, coalesce(content, **N**'DEFAULT_REPLACEMENT_STRING') FROM nchartest;

Can one of these solutions be applied to the SQL that is generated by DVT whenever COALESCE will get called on such nvarchar columns ? Is there any other solution or workaround to make concat or hash validations work with NCHAR columns ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.