GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

Oracle Test fails after using duckdb backend to process pandas dataframe

Open
#1,763 1 comment 0 reactions 0 assignees View on GitHub
type: bug
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

**Describe the bug**
Tests `test_column_validation_all_null_oracle_to_postgres` and `test_row_validation_comp_fields_core_types` fail due to overflow of DECIMAL(18,3). There appear to be more than 18 digits in the value - which means the overflow is legitimate.
```
FAILED tests/system/data_sources/test_oracle.py::test_column_validation_all_null_oracle_to_postgres - ibis.common.exceptions.IbisTypeError: Arguments differences_target_value:float64 and differences_source_value:string are not...
FAILED tests/system/data_sources/test_oracle.py::test_row_validation_comp_fields_core_types - sqlalchemy.exc.ProgrammingError: (duckdb.duckdb.InvalidInputException) Invalid Input Error: Attempting to execute an unsucce...
```
**What version of DVT are you using?**
Latest version based on Ibis 7.1

**What type of connections are you using for source and target?**
Oracle

**Steps to reproduce the behavior**
Run the tests - after removing the statement to skip this test.

**Expected behavior**
Complete successfully.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**How are you executing DVT?**
Ex: CLI commands on local computer or VM, within a DAG, Cloud Run job, etc

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.