canonical / canonical/dqlite

Dqlite returns node join success before the join was successful in the distributed context

Open
#799 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C
Stars
4.4k
Forks
255
PR merge metrics
No merged PRs in 30d

Description

When we add a new node via a Dqlite clients request, Dqlite is returning before the cluster membership state is fully committed. Our client has no way of knowing whether the cluster membership addition was successful or not. The first red arrow marks the point we currently return at. In the distributed sense our add node operation would only be completed when we have confirmation from the followers that the join was successfully committed (second red arrow).

@just-now's diagram illustrates the issue:

Image

Related to this issue is the Dqlite cluster.yaml only being synced every second: https://github.com/canonical/go-dqlite/issues/388.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the Dqlite client add-node request through cluster membership handling, using the diagram's first and second red arrows as the timing points to compare. Review the related cluster.yaml synchronization issue for context. Done means the client receives success only after followers confirm that the membership addition has been committed.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, sqlite
Domain
databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.