GoogleCloudPlatform / GoogleCloudPlatform/professional-services-data-validator

Misleading "Column DNE in source" exception

Open
#1,663 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
524
Forks
171
Avg merge
5d 15h
Merged PRs (30d)
4

Description

When a table without a primary key is migrated using GCP DMS service a decimal `rowid` column is added to the target as a primary key. This column does not exist in the source.

If the user runs validation on one of these tables with no primary key then they, quite accurately, get the error:
```
raise ValueError(f"Column DNE in source: {column}")
ValueError: Column DNE in source: rowid
```

This is a little misleading though and can cause wasted time trying to understand why the rowid column is part of the `--hash=*` expansion. It turns out it there because of primary key automation.

It would be better if target primary keys were skipped if the columns are not in the source table. Then the user would see a more easily understood exception:
```
No primary keys were provided and neither the source or target tables have primary keys. Please include --primary-keys argument
```

This is a bit of an edge case so not high priority.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.