debezium / debezium/dbz

Support table.include/exclude.list and column.include/exclude.list options in Spanner connectors [DBZ-7990]

Open
#995 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

component/spanner-connector migrated-from-jira type/enhancement
Dominant language
HTML
Stars
6
Forks
9
Avg merge
2d 19h
Merged PRs (30d)
1

Description

Migrated from DBZ-7990

We would like to be able to specify table.include/exclude.list and column.include/exclude.list options in the Spanner connector like we can specify in the MySQL and PostgreSQL connectors.

[https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-property-table-include-list|http://example.com/]
[https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-table-exclude-list|http://example.com/]

The use case is that when creating a changestream, we would like to monitor all changes as follows, and the connector would be used to exclude tables and columns that would contain PII.
{code:java}
CREATE CHANGE STREAM EverythingStream
FOR ALL;{code}
If PII is later included in a table or column, it should be very convenient to be able to quickly exclude it without any changes on the changestream side, just by changing the connector settings.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No implementation files or tests are named. Start by locating the Spanner connector configuration and its table and column filtering entry points, then compare the existing MySQL and PostgreSQL include/exclude options. Done means the connector can exclude configured tables and columns while the Spanner change stream continues monitoring all changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.