cockroachdb / cockroachdb/cockroach
logical: perform validation on source tables before ingestion
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
Currently LDR performs validation that the destination tables are valid targets for replication via `CheckLogicalReplicationCompatibility`. This validation has a few gaps for the CREATE TABLE flow:
1. Create table has stricter requirements on what constitutes a valid replication target. Consider UDTs, these are supported in the non create table flow, but not in the create table flow as we need to validate the referenced UDTs exist in the replication set and rewrite their IDs. Currently attempting to replicate a table with a UDT will fail with a non specific `cross database type references are not supported` error. Another edge case is VIEW statements, which will silently fail for the same reason.
2. This validation happens after we ingest the source tables and create the destination tables, i.e. after we rewrite IDs. Ideally we would like the rewrite to throw an error if we ever hit an unmapped ID, as this likely means we will have a corrupted descriptor. However because we don't perform validation until after, throwing an error in rewrite would obscure the specific "TRIGGER not supported" errors.
Instead, we should perform validation on the source tables before ingestion. This would allow us to throw explicit errors up front, and add the more general fallback in Rewrite.
Jira issue: CRDB-63348
Contributor guide
Research direction
Trace CheckLogicalReplicationCompatibility and Rewrite from the logical replication ingestion path, focusing on where source tables are available before ingestion. Done means validation reports explicit errors for unsupported UDT, VIEW, and TRIGGER cases before destination creation, while Rewrite provides a general fallback for unmapped IDs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100