GoogleCloudPlatform / GoogleCloudPlatform/spanner-migration-tool

Show more diverse set of rows in dropped.txt

Open
#242 0 comments 0 reactions 0 assignees View on GitHub
p4
Dominant language
PLpgSQL
Stars
155
Forks
83
Avg merge
3d 7h
Merged PRs (30d)
6

Description

Currently, we keep appending bad rows to conv till we hit the byte limit and then dump them to dropped.txt. When dealing with large tables, usually we end up storing all rows from one table in the dropped.txt because a single issue is occuring across many rows.
There is scope for improvement by adding bad rows from different tables by removing some of the earlier ones, as more rows caused by the same error does not provide more information. It is more efficient to report a few samples of multiple types of bad rows.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.