citusdata / citusdata/citus

The citus cluster has collapsed

Open
#7,667 9 comments 0 reactions 0 assignees View on GitHub
Support
Dominant language
C
Stars
12.8k
Forks
794
Avg merge
2d 14h
Merged PRs (30d)
31

Description

For an unknown reason, the cluster stopped working.
His condition is:
**Coordinator:**
nodeid groupid nodename hasmetadata isactive metadatasynced shouldhaveshards
54 11 worker21 TRUE TRUE **FALSE** TRUE
**51** 12 **worker23** TRUE TRUE **TRUE** TRUE

pg_dist_node is locked by ExclusiveLock with process without pid (because of that it is not possible to do anything with metadata: meta sync for example )
It is advisory lock - I do not now how to kill that lock ?

**Worker21:**
nodeid groupid nodename isactive metadatasynced shouldhaveshards
54 11 worker21 TRUE TRUE TRUE
**51** 12 **worker22** TRUE TRUE TRUE

the worker22 is not active (existing) worker.
postgres log: WARNING: connection to the remote node worker22:**** failed with the following error: server closed the connection unexpectedly

**Worker23:**
nodeid groupid nodename isactive metadatasynced shouldhaveshards
54 11 worker21 TRUE **FALSE** TRUE
**51** 12 worker23 TRUE TRUE TRUE

It is possible build the cluster?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.