GoogleCloudPlatform / GoogleCloudPlatform/spanner-migration-tool

Improve error messages during data conversion.

Open
#85 0 comments 0 reactions 0 assignees View on GitHub
p4
Dominant language
PLpgSQL
Stars
155
Forks
83
Avg merge
3d 7h
Merged PRs (30d)
6

Description

In general, HarbourBridge provides a summary of errors and unusual conditions as part of the 'Unexpected Conditions" section of report.txt. However, this section does not including errors encountered while writing rows to Spanner.

Note that such errors are accumulated in BatchWriter and are available via the Errors() method, but HarbourBridge does not call Error and so these errors are essentially dropped.

There are several options to improve this:

a) Improve error handling in data.go so that more errors are caught before we call Spanner (e.g. we could catch common errors like non-matching column/value counts, and field size mismatches). This has other benefits: we can provide more succinct error messages and avoid batch retries/splits.

b) Change report.go to include errors from BatchWriter (requires some minor refactoring of main.go).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.