GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

SQL Server Test fail after migration to Duckdb Backend for Pandas dataframes

Open
#1,764 0 comments 0 reactions 0 assignees View on GitHub
type: bug
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

**Describe the bug**
Tests `test_column_validation_ss_types_to_bigquery` and `test_row_validation_comp_fields_ss_types_to_bigquery` fail due to overflow of DECIMAL(18,3). There appear to be more than 18 digits in the value - which means the overflow is legitimate. Also see issue 1763
```
FAILED tests/system/data_sources/test_sql_server.py::test_column_validation_ss_types_to_bigquery - sqlalchemy.exc.ProgrammingError: (duckdb.duckdb.InvalidInputException) Invalid Input Error: Attempting to execute an unsucce...
FAILED tests/system/data_sources/test_sql_server.py::test_row_validation_comp_fields_ss_types_to_bigquery - sqlalchemy.exc.ProgrammingError: (duckdb.duckdb.InvalidInputException) Invalid Input Error: Attempting to execute an unsucce...
```

**What version of DVT are you using?**
Ex: 7.1.0

**What type of connections are you using for source and target?**
SQL Server and BigQuery

**Steps to reproduce the behavior**
Ex: 1. Add '...'
2. Execute '....'
3. Scroll down to '....'
4. See error

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**How are you executing DVT?**
Ex: CLI commands on local computer or VM, within a DAG, Cloud Run job, etc

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.