ConduitIO / ConduitIO/conduit-connector-postgres
Feature: connector should handle `TRUNCATE` messages from the server
- Dominant language
- Go
- Stars
- 19
- Forks
- 12
- Avg merge
- 12h 20m
- Merged PRs (30d)
- 5
Description
### Feature description
Currently, `TRUNCATE` messages are ignored. This can provide unexpected behaviour because the stream will not capture all records which have been deleted by the operation.
Each truncate message, contains the number of rels (tables) being truncated in the same LSN. This is tricky because:
* The truncate message does not carry the keys of each row which needs to be deleted
* It all happens in the same LSN
* Conduit only supports Snapshot, Create, Delete and Update operations. Truncate does not fit in any of these
Some systems ignore it, others (like [dbz](https://debezium.io/documentation/reference/stable/connectors/postgresql.html#postgresql-truncate-events)) have an additional `truncate` operation to handle this type of operation.
Since its introduction in 2008, `TRUNCATE` is an optional feature in SQL. However, other systems do support way to empty a collection entirely without bother to emit individual event per delete.
The least work involved here will be to warn/block/something so the user is aware this is happening.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.