citusdata / citusdata/citus

Block reads until 2PC recovery after commit prepared failure

Open
#3,780 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

Currently when Citus fails to commit one of the prepared transactions, we return a commit with warnings to the client. However, clients will typically just treat that as a regular commit and rightfully expect all the data to be immediately readable.

After a commit failure, we should:
1) Do 2PC recovery, at least on the node with the commit failure, at a high frequency.
2) Block further queries on the node that had a commit failure until 2PC recovery succeeds.

A more fine-grained approach could be to keep track of what data needs to be committed (e.g. which tables were modified by the transactions).

In a typical scenario, the commit failed because the node was rebooting or failing over and momentarily unable to serve queries, hence the loss of availability is already there. We mainly want to ensure that 2PC recovery happens before any queries that depend on the commit.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.