GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator
Oracle exception "ORA-12704: character set mismatch" NVARCHAR (SQL Server also)
- Dominant language
- Python
- Stars
- 524
- Forks
- 171
- Avg merge
- 5d 15h
- Merged PRs (30d)
- 4
Description
During a row validation using the --hash "*" option we observed, that the SQL that is sent to Oracle behind the scenes produces an exception if the table being validated contains NVARCHAR columns.
The reason seems to be that the coalesce function used in the SQL generated by DVT that is called to replace null values with a non-empty String like 'DEFAULT_REPLACEMENT_STRING' requires two parameters of the same datatype . In case of a NVARCHAR column one parameter of the coalesce function is of type NVARCHAR using the configured NCHAR character set for that database and the second parameter (the replacement String) is of type CHAR using the default database charset which may be different from the NCHAR character set.
Example code to reproduce this exception:
CREATE TABLE nchartest (id NUMBER(5) PRIMARY KEY, content nvarchar2(10));
INSERT INTO nchartest VALUES (1, 'one');
INSERT INTO nchartest VALUES (2, 'two');
INSERT INTO nchartest VALUES (3, null);
SELECT id, coalesce(content, 'DEFAULT_REPLACEMENT_STRING') FROM nchartest;
The SELECT statement causes the observed character-set-mismatch exception. The exception disappears, if either the value of the content column is converted to the default database charset using to_char or the replacement String is marked as NCHAR value with a leading 'N'. So both of the following statements seem to work.
SELECT id, coalesce(**to_char(content)**, 'DEFAULT_REPLACEMENT_STRING') FROM nchartest;
or
SELECT id, coalesce(content, **N**'DEFAULT_REPLACEMENT_STRING') FROM nchartest;
Can one of these solutions be applied to the SQL that is generated by DVT whenever COALESCE will get called on such nvarchar columns ? Is there any other solution or workaround to make concat or hash validations work with NCHAR columns ?
Contributor guide
Assessment
This issue has not been assessed yet.