cockroachdb / cockroachdb/cockroach

pglogical: SQL "frontend" related changes meta issue

Open
#105,130 2 comments 1 reaction 0 assignees View on GitHub
A-cdc C-enhancement meta-issue T-cdc
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

* [ ] Add the system `replication_slots` table + migration
* [x] Add replication role option #106082
* [x] Add `pg_lsn` data type #105031
* [x] Add requisite version gating https://github.com/cockroachdb/cockroach/pull/105391
* [x] Version gate `pg_lsn` type in cast expressions
* [x] Add pgwire encoding tests for `pg_lsn` to `pkg/cmd/generate-binary/main.go`
* [x] add pg_lsn operators https://github.com/cockroachdb/cockroach/pull/105326
* [ ] Implement the PG [streaming replication protocol](https://www.postgresql.org/docs/current/protocol-replication.html)
* [x] Implement the parser #104938
* [x] Allow `replication={database,0,1,on,off}` as an argument into the connection string #105400
* [x] Disable extended protocol when `replication` mode enabled #106131
* [x] Disable logging in unless you have the replication roleoption #106082
* [ ] (optional) Disable changing `replication` session variable
* [ ] Implement & use the streaming replication protocol commands
* [x] IDENTIFY_SYSTEM #106131
* [ ] START_REPLICATION -- follow https://github.com/postgres/postgres/blob/master/src/backend/replication/walsender.c
* [ ] CREATE_REPLICATION_SLOT (not-AWS DMS blocking)
* [ ] DROP_REPLICATION_SLOT (not-AWS DMS blocking)
* [x] Add unimplemented errors to unsupported replication protocol commands (i.e. TIMELINE_HISTORY, CREATE_REPLICATION_SLOT for PHYSICAL, BASE_BACKUP, READ_REPLICATION_SLOT) #104938
* [ ] Implement the pg_catalog metadata tables
* [ ] Implement the functions
* [ ] pg_create_logical_replication_slot
* [ ] pg_drop_replication_slot
* [ ] pg_logical_slot_get_changes (not-AWS DMS blocking)
* [ ] pg_logical_slot_peek_changes (not-AWS DMS blocking)
* [ ] pg_logical_slot_get_binary_changes (stretch/not-AWS DMS blocking)
* [ ] pg_logical_slot_peek_binary_changes (stretch/not-AWS DMS blocking)
* [ ] pg_get_replication_slots (not-AWS DMS blocking)
* [ ] pg_replication_slot_advance (not-AWS DMS blocking)
* [ ] Add unimplemented errors to [unsupported commands](https://docs.google.com/document/d/1ZvkBweK8fWtceDI-UKCpygWqQxKyc_GuAjWSCqkUhtc/edit#heading=h.z7g5d1i4oupy)
* [ ] Add mixed version testing for v22.2+v23.1 / v23.2
* [ ] Properly [figure out LSNs](https://docs.google.com/document/d/1VrGN0bC7ZQQXwd1T1doWh4KzCZDR7F7T42y9Ew7guHc/edit)
* [ ] Add `max_replication_slots`, `max_wal_senders`, `wal_level`, `wal_sender_timeout` session variables
* [ ] (stretch) extend SHOW SYNTAX to support pg repl commands
* [ ] (stretch) CREATE PUBLICATION

----

x-ref #93404 for prototype PR
DMS meta issue https://github.com/cockroachdb/cockroach/issues/84505
Epic: https://cockroachlabs.atlassian.net/browse/CRDB-26486
test with https://github.com/jackc/pglogrepl

Jira issue: CRDB-28886

Epic CRDB-26486

Contributor guide

Open the contributing guide

Research direction

This is a broad meta issue with many unchecked replication, catalog, function, testing, and session-variable tasks. Start by selecting one unchecked item, then read the linked PostgreSQL walsender.c reference or the pglogrepl client; pkg/cmd/generate-binary/main.go is mentioned for pg_lsn encoding tests. Done means the selected checklist item works and is validated with the relevant protocol or mixed-version testing.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, postgresql
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.