xataio / xataio/pgstream

[Snapshots] Option to use PK instead of ctid ranges

Open
#601 0 comments 1 reaction 1 assignee View on GitHub

@kvch is already working on this.

Since Jun 24, 2026.

enhancement snapshot
Dominant language
Go
Stars
1.2k
Forks
82
Avg merge
18h 22m
Merged PRs (30d)
51

Description

Is your feature request related to a problem? Please describe.
Have the option to provide a PK column for a table to be used during the snapshot process instead of using the ctid, which forces a range over a stable view of the database (transaction snapshot). This is helpful for enabling resumable snapshots at row level as well as reconnection attempts.

Describe the solution you'd like
Have a data snapshot generator implementation that uses PK to read the source table instead of relying on transaction snapshots and ctid ranges.

Configuration

Proposed (under source.postgres.snapshot.data):

strategy: ctid | pk            # default ctid
strategy_overrides:            # per-table strategy
  public.audit: ctid
primary_keys:                  # per-table key override
  public.orders: [tenant_id, id]

Env equivalents: PGSTREAM_POSTGRES_SNAPSHOT_DATA_STRATEGY, ..._STRATEGY_OVERRIDES, ..._PRIMARY_KEYS (final names to confirm). Key-resolution precedence: primary_keys override → detected primary key → NOT NULL unique index.

Notes
The PK keyset reader gives per-batch eventual consistency: each batch is internally consistent, but there is no cross-batch or cross-table point-in-time guarantee. Rows read at different times may reflect concurrent writes. This is acceptable for the initial-snapshot to CDC path because replication resumes from the slot LSN and ON CONFLICT upserts reconcile any drift. Under the snapshot command (no CDC follows), the result may not be point-in-time consistent.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.