canonical / canonical/microceph

Unable to change monitor IP on secondary node; permission errors during mon IP modification

Open
#568 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Go
Stars
396
Forks
74
Avg merge
2d 20h
Merged PRs (30d)
7

Description

I'm setting up a 3-node MicroCeph cluster across three different subnets. After bootstrap and node join operations, I discovered slow operation errors in logs. Investigation revealed inconsistent IP binding between nodes:

1. **First Node** (Bootstrap):
```bash
sudo microceph cluster bootstrap --cluster-network 0.0.0.0/0 --mon-ip --public-network 0.0.0.0/0 --microceph-ip
```
- Binds correctly to public IP for both services:
- `7443` (MicroCeph): ``
- `6789` (Mon): ``

2. **Second Node** (Joined via token):
```bash
sudo microceph cluster join --microceph-ip
```
- Binds inconsistently:
- `7443` (MicroCeph): `` (correct)
- `6789` (Mon): `192.168.1.137` (private IP, incorrect)

Attempted to change the monitor IP following [Ceph's documentation](https://docs.ceph.com/en/latest/rados/operations/add-or-rm-mons/#changing-a-monitor-s-ip-address-preferred-method) but encountered permission errors:
```bash
sudo ceph mon -i {mon-id} --mkfs --monmap /tmp/mapfile --keyring /tmp/keyfile
# Error:
2025-06-10T07:32:13.032+0000 7f2b467fc6c0 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2,1]
[errno 13] RADOS permission denied (error connecting to the cluster)
```

### Key Questions
1. Is there a way to specify the monitor IP (`--mon-ip`) during `microceph cluster join`?
2. If not, what's the correct procedure to change the monitor IP post-join without triggering permission errors?

### Additional Context
- Cluster network configuration uses `0.0.0.0/0` (all interfaces)
- Nodes reside in **different subnets**, requiring explicit public IP binding
- Observed "slow ops" errors in logs likely due to cross-subnet communication issues

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.