canonical / canonical/microceph

How can I update the IP addresses for each node after the rack is relocated?

Open
#570 5 comments 0 reactions 0 assignees View on GitHub
bug documentation
Dominant language
Go
Stars
396
Forks
74
Avg merge
2d 20h
Merged PRs (30d)
7

Description

I was using a 4-node(3-mon within) MicroCeph cluster for a while in my project.
But after the rack is relocated and the IP addresses is changed, MicroCeph cluster can not startup.
And I've modified all IP addresses in config files. It still doesn't work.
How to solve this problem? Thank you.

microceph status:
> Error: failed listing disks: Database is not yet initialized

ceph-mon.node1.log:

> 2025-06-14T04:57:58.691+0800 702a7dc5ba80 0 mon.node1@-1(???).osd e7064 crush map has features 432629239337189376, adjusting msgr requires
> 2025-06-14T04:57:58.692+0800 702a7dc5ba80 1 mon.node1@-1(???).paxosservice(auth 4751..4837) refresh upgraded, format 0 -> 3
> 2025-06-14T04:57:58.694+0800 702a7dc5ba80 -1 Processor -- bind unable to bind to v2:10.109.80.35:3300/0: (99) Cannot assign requested address
> 2025-06-14T04:57:58.694+0800 702a7dc5ba80 -1 Processor -- bind was unable to bind. Trying again in 5 seconds
> 2025-06-14T04:58:03.695+0800 702a7dc5ba80 -1 Processor -- bind unable to bind to v2:10.109.80.35:3300/0: (99) Cannot assign requested address
> 2025-06-14T04:58:03.695+0800 702a7dc5ba80 -1 Processor -- bind was unable to bind. Trying again in 5 seconds
> 2025-06-14T04:58:08.695+0800 702a7dc5ba80 -1 Processor -- bind unable to bind to v2:10.109.80.35:3300/0: (99) Cannot assign requested address
> 2025-06-14T04:58:08.695+0800 702a7dc5ba80 -1 Processor -- bind was unable to bind after 3 attempts: (99) Cannot assign requested address
> 2025-06-14T04:58:08.695+0800 702a7dc5ba80 -1 unable to bind monitor to [v2:10.109.80.35:3300/0,v1:10.109.80.35:6789/0]

where 10.109.80.35 is still the old IP address.

Contributor guide

Open the contributing guide

Research direction

Start with the reported `microceph status` failure and `ceph-mon.node1.log`, especially the bind errors for the old address 10.109.80.35. Trace where the monitor endpoint is retained after the rack relocation and determine the supported recovery path; done means the monitors bind to their current addresses and `microceph status` works again.

Written by the indexing model from the issue text.

Assessment

Domain
distributed-systems, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.