ClusterLabs / ClusterLabs/resource-agents

Galera resource file issue

Open
#770 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
519
Forks
608
Avg merge
6d 1h
Merged PRs (30d)
7

Description

I'm using release v3.9.6.
I think release v3.9.6 have the same issue too.

The function

`detect_first_master`

Compare the last commit, to select best node.
If Node1 and Node2 as the same last_commit.
Then best_node have two different value.
Then Node1 and Node2 all the bootstrap node.
Then Node1 mysql got error:

`due to conflicting prims: older overrides`

And the last, Node1 status looks like fine, but can't INSERT or UPDATE, there raise
`1213, 'Deadlock found when trying to get lock; try restarting transaction'`

At that time, mysql wsrep status like this:
`+---------------+-------+`
`| Variable_name | Value |`
`+---------------+-------+`
`| wsrep_last_committed | 448630 |`
`| wsrep_incoming_addresses | 172.17.105.66:3306 |`
`| wsrep_cluster_status | Primary |`
`| wsrep_connected | ON |`
`| wsrep_ready | ON |`

There is node1 log:
`Feb 23 08:31:03 cvm66 galera(db)[5516]: INFO: Enter galera_start`
`Feb 23 08:31:04 cvm66 galera(db)[5516]: INFO: MySQL is not running`
`....`
`Feb 23 08:31:14 cvm66 galera(db)[5516]: INFO: attempting to detect last commit version`
`Feb 23 08:31:14 cvm66 galera(db)[5516]: INFO: Last commit version found: 448630`
`Feb 23 08:31:14 cvm66 galera(db)[5516]: INFO: Waiting on node <172.17.105.67> to report database status before Master instances can start.`
`Feb 23 08:31:15 cvm66 galera(db)[5516]: INFO: Leave galera_start 0`
`...`
`Feb 23 08:31:32 cvm66 pengine[4337]: notice: LogActions: Promote db:0#011(Slave -> Master 172.17.105.66)`
`Feb 23 08:31:32 cvm66 pengine[4337]: notice: LogActions: Promote db:1#011(Slave -> Master 172.17.105.67)`
`Feb 23 08:31:32 cvm66 galera(db)[6578]: INFO: Enter galera_promote`
`Feb 23 08:31:32 cvm66 galera(db)[6578]: INFO: Node <172.17.105.66> is bootstrapping the cluster`
`...`
`Feb 23 08:31:45 cvm66 galera(db)[6578]: INFO: Leave galera_promote 0`

There is node2 log:
`Feb 23 08:31:19 cvm67 galera(db)[5893]: INFO: Enter galera_start`
`Feb 23 08:31:20 cvm67 galera(db)[5893]: INFO: MySQL is not running`
`...`
`Feb 23 08:31:32 cvm67 galera(db)[5893]: INFO: attempting to detect last commit version`
`Feb 23 08:31:32 cvm67 galera(db)[5893]: INFO: Last commit version found: 448630`
`Feb 23 08:31:32 cvm67 galera(db)[5893]: INFO: Promoting 172.17.105.67 to be our bootstrap node`
`Feb 23 08:31:32 cvm67 galera(db)[5893]: INFO: Leave galera_start 0`
`...`
`Feb 23 08:31:32 cvm67 galera(db)[6452]: INFO: Enter galera_promote`
`Feb 23 08:31:32 cvm67 galera(db)[6452]: INFO: Node <172.17.105.67> is bootstrapping the cluster`
`...`
`Feb 23 08:31:46 cvm67 galera(db)[6452]: INFO: Bootstrap complete, promoting the rest of the galera instances.`
`Feb 23 08:31:46 cvm67 galera(db)[6452]: INFO: Leave galera_promote 0`

There is node1 mysql log:
`...`
`160223 8:31:55 [Warning] WSREP: a37e2635 conflicting prims: my prim: view_id(PRIM,a37e2635,11) other prim: view_id(PRIM,e76d9671,11)`
`...`
`160223 8:31:55 [ERROR] WSREP: exception from gcomm, backend must be restarted: a37e2635 aborting due to conflicting prims: older overrides (FATAL)`
`...`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the detect_first_master function in the Galera resource agent and compare its last-commit selection with the node1 and node2 logs. Reproduce or test the equal-last-commit case, then verify that only one node bootstraps and that conflicting primary views no longer occur.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, shell
Domain
databases, distributed-systems, infrastructure
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.