ConduitIO / ConduitIO/conduit-connector-postgres

Feature: connector should handle `TRUNCATE` messages from the server

Open
#168 0 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Go
Stars
19
Forks
12
Avg merge
12h 20m
Merged PRs (30d)
5

Description

### Feature description

Currently, `TRUNCATE` messages are ignored. This can provide unexpected behaviour because the stream will not capture all records which have been deleted by the operation.

Each truncate message, contains the number of rels (tables) being truncated in the same LSN. This is tricky because:
* The truncate message does not carry the keys of each row which needs to be deleted
* It all happens in the same LSN
* Conduit only supports Snapshot, Create, Delete and Update operations. Truncate does not fit in any of these

Some systems ignore it, others (like [dbz](https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-truncate-events)) have an additional `truncate` operation to handle this type of operation.

Since its introduction in 2008, `TRUNCATE` is an optional feature in SQL. However, other systems do support way to empty a collection entirely without bother to emit individual event per delete.

The least work involved here will be to warn/block/something so the user is aware this is happening.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.